An AI model can be given a hidden backdoor by editing fewer than 10 of its rarely used neurons, without retraining the model, Microsoft researchers have shown.
When a chosen trigger appears, the model outputs an attacker’s command, which an agent using that model would run with the agent’s own permissions. The attacker must be able to change the model’s weights.
The team tested the method on An AI model can be given a hidden backdoor by editing fewer than 10 of its rarely used neurons, without retraining the model, Microsoft researchers have shown.
When a chosen trigger appears, the model outputs an attacker’s command, which an agent using that model would run with the agent’s own permissions. The attacker must be able to change the model’s weights.
The team tested the method on The Hacker News