Artificial Intelligence

This AI judges you based on the biases we've instilled it with

A visitor of the Art Cologne Fair 2017 is reflected in the four-by-four concave mirror object made of aluminium, glass and mirrors in 1971 by German artist Adolf Luther on sale for 168,000 Euros (estimated 197,850 US dollars) in Cologne, Germany November 22, 2017.  REUTERS/Wolfgang Rattay

AI can be instilled with lots of the inherent biases of humans. Image: REUTERS/Wolfgang Rattay

Kristin Houser
Our Impact
What's the World Economic Forum doing to accelerate action on Artificial Intelligence?
The Big Picture
Explore and monitor how Artificial Intelligence is affecting economies, industries and global issues
A hand holding a looking glass by a lake
Crowdsource Innovation
Get involved with our crowdsourced digital platform to deliver impact at scale
Stay up to date:

Artificial Intelligence

First impressions

When we see someone for the first time, we make internal snap judgements about them. We can’t help it, we’re just judgmental like that. After looking at the person for just a few seconds, we might note their gender, race, and age or decide whether or not we think they’re attractive, trustworthy, or kind.

After actually getting to know the person, we might find out that our initial perception of them was wrong. No big deal, right?

Well, it’s a very big deal when you consider how our assumptions could shape how the artificial intelligence (AI) of the future make increasingly important decisions.

In an effort to illustrate this issue to the public, researchers from the University of Melbourne created Biometric Mirror.

Public perception

Biometric Mirror is an AI that analyzes a person’s face and then displays 14 characteristics about them, including their age, race, and perceived level of attractiveness.

Image: University of Melbourne

To teach the system to do this, the Melbourne researchers started by asking human volunteers to judge thousands of photos for the same characteristics. This became the dataset Biometric Mirror referenced when analyzing new faces. Because the information these volunteers provided was subjective, so was Biometric Mirror’s output. If most of the human respondents thought people with beards seemed less trustworthy, that would influence how the Biometric Mirror judged people with beards.


The ethics of AI

To use Biometric Mirror, a person just has to stand in front of the system for a few seconds. It quickly scans their face and then lists their perceived characteristics on a screen. The AI then asks the person to think about how they’d feel if it shared that information with others. How would they feel if they didn’t get a job because the AI ranked them as having a low level of trustworthiness? Or if law enforcement officials decided to target them because they ranked highly for aggression?

“Our study aims to provoke challenging questions about the boundaries of AI. It shows users how easy it is to implement AI that discriminates in unethical or problematic ways which could have societal consequences,” lead researcher Niels Wouters said in a press release. “By encouraging debate on privacy and mass-surveillance, we hope to contribute to a better understanding of the ethics behind AI.”

Eliminating bias

A system as biased as Biometric Mirror could have major consequences as AI becomes more widely used and makes increasingly important decisions. And this isn’t just some future possibility, either; we’re already seeing examples show up in today’s systems. While researchers work on figuring out ways to ensure future systems don’t contain those same flaws, it’s important that the public consider the potential impact of biased AI on society. Biometric Mirror could help them do just that.

Have you read?
Don't miss any update on this topic

Create a free account and access your personalized content collection with our latest publications and analyses.

Sign up for free

License and Republishing

World Economic Forum articles may be republished in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License, and in accordance with our Terms of Use.

The views expressed in this article are those of the author alone and not the World Economic Forum.

Related topics:
Artificial IntelligenceEmerging Technologies
World Economic Forum logo
Global Agenda

The Agenda Weekly

A weekly update of the most important issues driving the global agenda

Subscribe today

You can unsubscribe at any time using the link in our emails. For more details, review our privacy policy.

Causal AI: the revolution uncovering the ‘why’ of decision-making

Darko Matovski

April 11, 2024

About Us



Partners & Members

  • Join Us

Language Editions

Privacy Policy & Terms of Service

© 2024 World Economic Forum