Data Poisoning vs Facial Recognition Technologies

This post provides an explanation of a research paper titled Data Poisoning Won’t Save You From Facial Recognition by Evani Radiya-Dixit and Florian Tramer who both work at Stanford University. This paper was submitted on June 28, 2021. The authors argue that poisoning-based methods for facial images uploaded online will not serve as a good defense in protecting user privacy.

Context:

Data poisoning is a situation where you alter (or perturb) your image before you post it online with the hopes that facial recognition models that are trained on web scraped pictures can’t detect who you are. Perturbing your pictures should theoretically safeguard your online privacy. That is unless the models that scrape the images are really good at facial recognition even when you perturb an image, which they are as I found out! The authors set up experiments, called poisoning attack games, between the user who uploads perturbed images and the model that detects them. They refer to the user as the attacker and the model as the defender (even though the defender is the one that is aiming to breach the online privacy of the attacker).

Defenses are actions taken by the defender to counter the attacker who poisoned the image. There were two defense strategies employed in the attack games, but first, let's look at how the authors poisoned the images for experimentation:

Attack methods

There are two facial recognition tools that were used for experimentation called Fawkes and LowKey. Fawkes was created by scientists at the University of Chicago’s Sand Lab and Lowkey was created by a group of computer scientists who wrote a paper to describe their work. While the technology behind the tools is a bit complicated, the goal of the two tools is the same: to prevent the images you post online from being used to track you. These tools were used as attacks on the facial recognition tool to see how well the pictures did at being undetected. An example of how images get poisoned is below:

First row: Original images; Second row: Images protected with LowKey (medium); Third row: Images protected with LowKey (large)

Defenses

Oblivious defense: The idea behind the oblivious defense is to wait until a new facial recognition model is invented that can better detect poisoned images using more advanced techniques and train your model on the images that were already scraped. Then employ your model on new images uploaded.

For an attacker to bypass this defense, the attacker will need to poison an image that fools not only today's facial recognition models but those that will be built in the future, which is nearly impossible! The facial recognition technology we have today is way more advanced than it was just a few years ago.

Adaptive defense: This defense is when you can access how the poisoning of the image occurred, apply that to images, and then augment your dataset with perturbed and unperturbed images. So the defender is basically using the strategies employed by the attacker to create poisoned images to train the model. An adaptive model could also aim to detect whether a picture is perturbed or not. This would be useful because it can filter out perturbed images, and retain only unperturbed images of a user

The authors note there has not been an effective counterattack on this defense.

Figure and comments below taken from the research paper

(1) Users perturb their pictures before posting them online.

(2) A model trainer continuously scrapes the Web for pictures.

(3–4)The model trainer builds a model from collected pictures and evaluates it on unperturbed pictures. With no defense strategy, the poisoned model fails to recognize users whose online pictures were perturbed. An “oblivious” model trainer can wait until a better facial recognition model is discovered and retroactively train it on past pictures to resist poisoning. An adaptive model trainer with black-box access to the attack employed by users can immediately train a robust model that resists poisoning.

Outcomes

To conclude, the authors state that “poisoning attacks against facial recognition will not lead to an “arms race”, where new attacks can continuously counteract new defenses. Since the perturbation applied to a picture cannot be changed once the picture is scraped, a successful poisoning attack has to remain effective against all future models, even models trained adaptively against the attack or models that use new techniques discovered only after the attack.”

So are we doomed against safeguarding our online privacy? Well, I think the authors would argue that we are. The only way to not have your photos be tracked is if you never post a photo of yourself in the first place, which is highly unlikely in this age of social media. The only other thing that may help with privacy safeguards is if there is legislation passed that makes it illegal to use mass facial recognition software that in turn gets used by public and private entities.

Why you should care about this

There are major ethical questions that haunt facial recognition research. For a facial recognition model to work well, it needs to be trained and tested on tons of labeled images. This is usually done by scraping social media sites without asking for your permission and it is more than likely that your face is labeled in a database somewhere being used to build a product for profit that you know nothing about.

However, it's not all bad! Facial recognition technology can be used by police to identify a suspect in a crime, or used in airports to check travelers' identities or to simply unlock your iPhone. Facial recognition is an awesome subject in data science, but like all new technologies, you have to think about the ethical consideration of the new technology and how that will impact users.