Reckless adoption of AI can help deep fakes grab business data

A couple of different things. For voice deep fakes, we usually recommend that there are shared key phrases between people within the company. If I called you, I need to use the word "banana" in our conversation so that you know that it is me calling you. Alternatively, after you receive a call and you are on that call, interacting with the person, when you hang up, call the person back -- proactively take the action of calling the number that you know is good and confirm with the person – were we just on the phone? 

Those are some of the recommendations we give for deep fakes, voice-related deep fakes. On email, take really strong precautions on things that seem too good to be true. Do second checks. For example, if you get an email that says, "I'm your vendor, here's the invoice, you owe me $10,000," confirm with a second person from accounting that we do owe that vendor $10,000. Don't just transfer the money without a second check. We are seeing a little bit of evolution where defenders themselves are starting to use LLMs [large language models] to fight LLMs, but it's going to be a constant cat and mouse game. It's going to be an interesting evolution to monitor, but it's going to be hard. 

What we announced is called affirmative coverage, meaning it's not necessarily a new coverage. It's just affirmatively saying that for these cases, we do cover it. We have lots of clients that are embedding LLMs into their products. There used to be something called a SQL injection, where an attacker would put [in] a very specific string to get a database to dump all the content that it has. 

What attackers now use against LLMs is a prompt injection, in which they can put in a very specific string to try to get the bot or the LLM to access some of their customer data. That would be covered. We affirmatively say, if you are a victim of a prompt injection attack, it is covered under your cyber insurance policy. This isn't new. This is a malicious attack, attacking a web application that happens to use an LLM.

We are seeing customers adopt more and more AI, either for day-to-day use, or embedding it into their products. What we're not seeing is people being very careful. A lot of companies have customer data, and they embed these AI products without thinking about data privacy issues or how providers use data to retrain their own models. 

Make sure you read all of the terms and conditions and data privacy policies of your providers, and how they're going to use all of that data. Do you really need to send all of the data to the LLM? Can you separate customer data from generic data? The EU has tight AI data laws so it's really important to get that right.