AI Software

Overview

Adversaries may target software packages that are commonly used in AI-enabled systems or are part of the AI DevOps lifecycle. This can include deep learning frameworks used to build AI models (e.g. PyTorch, TensorFlow, Jax), generative AI integration frameworks (e.g. LangChain, LangFlow), inference engines, and AI DevOps tools. They may also target the dependency chains of any of these software packages Compromised PyTorch-nightly dependency chain between December 25th and December 30th, 2022.. Additionally, adversaries may target specific components used by AI software such as configuration files New Vulnerability in GitHub Copilot and Cursor: How Hackers Can Weaponize Code Agents or example usage of AI packages, which may be distributed in Jupyter notebooks Careful Who You Colab With: abusing google colaboratory.

Adversaries may compromise legitimate packages Security Update for Amazon Q Developer Extension for Visual Studio Code (Version #1.84) or publish malicious software to a namesquatted location Compromised PyTorch-nightly dependency chain between December 25th and December 30th, 2022.. They may target package names that are hallucinated by large language models Slopsquatting: When AI Agents Hallucinate Malicious Packages (see: Publish Hallucinated Entities). They may also perform a AI Supply Chain Rug Pull in which they first publish a legitimate package and then publish a malicious version once they reach a critical mass of users.