Malicious attack method on hosted ML models now targets PyPI

Artificial intelligence (AI) and machine learning (ML) are now inextricably linked to the software supply chain. ML models, which are based on large language models (LLMs), are powering the enterprise — and offer an infinite number of solutions to organizations’ mission-critical needs. The widespread and increasing use of generative AI tools like OpenAI’s ChatGPT, in addition to developer community resources like Hugging Face – a platform dedicated to collaboration and sharing of ML projects – show how software, coding and AI/ML are now one and the same. 

But as with any new technological advancement, the pressing need for ML models has created a new and ever-evolving attack surface that the cybersecurity industry is racing to understand and mitigate. Recognizing the convergence of AI and the software supply chain, ReversingLabs (RL) researchers and engineers have taken steps to better understand the threat posed by malicious ML models. 

One such threat that RL researchers have previously flagged is the Pickle file format, a popular but insecure Python module that is used widely for serializing and deserializing ML model data. Dhaval Shah, RL’s senior director of product management, wrote recently that Pickle files open the door to malicious actors who can abuse it to inject harmful code into the model files.

That warning proved true with the discovery of nullifAI, discovered by RL threat researchers in February, in which threat actors abused ML models in the Pickle file format to distribute malicious ML models on Hugging Face. With this latest discovery, RL researchers uncovered a new malicious campaign that further proves threat actors’ newly favored method of exploiting the Pickle file format — this time on the Python Package Index (PyPI). 

Last Tuesday, RL researchers detected three, newly uploaded malicious packages that pose as a “Python SDK for interacting with Aliyun AI Labs services.” As the package description indicates, this is an attack that targets users of Alibaba AI labs. Once installed, the malicious package delivers an infostealer payload hidden inside a PyTorch model loaded from the initialization script. (PyTorch models are basically zipped Pickle files.) The malicious payload exfiltrates basic information about the infected machine and the content of the .gitconfig file.

Here’s what RL researchers discovered — and what this new malicious campaign means for the security of ML models, as well as how open-source software (OSS) platforms are still a favored supply chain attack vector.

The post Malicious attack method on hosted ML models now targets PyPI appeared first on Security Boulevard.