Model theft / extraction
Model theft and extraction means an attacker obtains a model an organization trained, bought or adapted to its own needs. That can be direct theft of the files, the weights and the configuration, but also controlled extraction through an API, where the model is queried enough times to build an approximate copy.
Term
description · examples · notesModel theft and extraction means an attacker obtains a model an organization trained, bought or adapted to its own needs. That can be direct theft of the files, the weights and the configuration, but also controlled extraction through an API, where the model is queried enough times to build an approximate copy.
Description
Direct theft resembles ordinary data theft: badly protected storage, a repository, a developer's laptop or a cloud bucket. Extraction is different. The attacker sends many queries, records the answers, and trains their own model that imitates the behavior of the original.
The consequence is not only a loss of intellectual property. A copy of the model can help an attacker find weaknesses, build evasive examples, or draw out traces of the data the model was trained on.
Examples
- Model files are left in public cloud storage and the attacker downloads them.
- A scoring API has no limit on the number of queries, so the attacker systematically reconstructs the model's logic.
- A model trained on sensitive data reveals, through carefully chosen queries, information it should not.
Notes
- A model does not have to be stolen as a file to be stolen.
- Rate limiting, query monitoring and access control protect the asset here, but not availability.
Techniques
carried out with 2Techniques used to carry it out. Select one to open its page.
Defenses
countered by 5How it is defended against. Select one to open its page.