Artificial intelligence (AI) seems to be everywhere these days, from marketing programs to diagnostic laboratories. It’s now increasingly common to build a custom AI model or buy commercial offerings powered by AI. But before you set that AI loose in the world and into your core business, make sure you understand the potential security pitfalls and take steps for responsible adoption of AI.
AI 101
Machine learning (ML) is the most common form of AI and is the process of training a machine to make future predictions based on historical data. AI/ML exists in two phases. The first is training, which is when historical data is being used to "train" a model. To train a model, you need data that is labeled, or in other words, data that is paired with the answer. For example, an image of a cat is labeled as "cat." The more data you use, and the more realistic that data is, the stronger the model will be. The second phase is inferencing, in which the model is now given real data and asked to output what it believes the answer is based on its model.
Stop or full speed ahead?
Practically any 4-year-old can tell you what a stop sign is. Would it surprise you to know that a trained AI misidentified one as a speed limit sign? A few rectangles of black and white tape were enough to fool it. I'll dig into how later.
Where AI Meets Security
While AI has been a game-changer for solving complex business problems, there is a security risk landscape associated with it. Let's explore three areas.
Model Training Poisoning: If an adversary has access to the training data, then an attacker can train in potential backdoors, i.e., flawed logic paths known to the attacker. Take image recognition that is used to identify abandoned luggage at an airport, for example. A bad actor could introduce wrongly labeled images into the training set so that a suitcase with a specific graphic on it would be misidentified as something benign, such as a television—ensuring that the threat would go unnoticed in a facility depending on AI identification.
Continue reading:
https://www.forbes.com/sites/forbestechcouncil/2022/08/11/implementing-ai-youd-better-think-about-security-first/?sh=1b2a10315ea4
AI 101
Machine learning (ML) is the most common form of AI and is the process of training a machine to make future predictions based on historical data. AI/ML exists in two phases. The first is training, which is when historical data is being used to "train" a model. To train a model, you need data that is labeled, or in other words, data that is paired with the answer. For example, an image of a cat is labeled as "cat." The more data you use, and the more realistic that data is, the stronger the model will be. The second phase is inferencing, in which the model is now given real data and asked to output what it believes the answer is based on its model.
Stop or full speed ahead?
Practically any 4-year-old can tell you what a stop sign is. Would it surprise you to know that a trained AI misidentified one as a speed limit sign? A few rectangles of black and white tape were enough to fool it. I'll dig into how later.
Where AI Meets Security
While AI has been a game-changer for solving complex business problems, there is a security risk landscape associated with it. Let's explore three areas.
Model Training Poisoning: If an adversary has access to the training data, then an attacker can train in potential backdoors, i.e., flawed logic paths known to the attacker. Take image recognition that is used to identify abandoned luggage at an airport, for example. A bad actor could introduce wrongly labeled images into the training set so that a suitcase with a specific graphic on it would be misidentified as something benign, such as a television—ensuring that the threat would go unnoticed in a facility depending on AI identification.
Continue reading:
https://www.forbes.com/sites/forbestechcouncil/2022/08/11/implementing-ai-youd-better-think-about-security-first/?sh=1b2a10315ea4
