For years, tech companies have relied on something called the Fitzpatrick scale to classify skin tones for their computer vision algorithms. Originally designed for dermatologists in the 1970s, the system comprises only six skin tones, a possible contributor to AI’s well-documented failures in identifying people of color. Now Google is beginning to incorporate a 10-skin tone standard across its products, called the Monk Skin Tone (MST) scale, from Google Search images to Google Photos and beyond. The development has the potential to reveal bias in datasets used to train AI in everything from health care to content moderation.
Google first signaled plans to go beyond the Fitzpatrick scale last year; internally, the project dates back to a summer 2020 effort to make AI “work better for people of color,” according to a Twitter thread from Xango Eyeé, a responsible AI product manager at the company. At today’s Google I/O conference, the company detailed how wide an impact the new system could have across its many products. Google will also open source MST, meaning it could replace Fitzpatrick as the industry standard for evaluating the fairness of cameras and computer vision systems.
“Think anywhere there are images of people’s faces being used where we need to test the algorithm for fairness,” says Eyeé.
The Monk Skin Tone scale is named after Ellis Monk, a Harvard University sociologist who has spent decades researching colorism’s impact on the lives of Black people in the United States. Monk created the scale in 2019 and worked with Google engineers and researchers to incorporate it into the company’s product development.
“The reality is that life chances, opportunities, all these things are very much tied to your phenotypical makeup,” Monk said in prepared remarks in a video shown at I/O. “We can weed out these biases in our technology from a really early stage and make sure the technology we have works equally well across all skin tones. I think this is a huge step forward.”
An initial analysis by Monk and Google research scientists last year involving more than 3,000 participants found that people felt better represented by MST than Fitzpatrick. The scale achieved representation results on par with skin tone scales involving more than 40 shades, like that used by Rihanna’s makeup company Fenty Beauty. Google is continuing work to validate the Monk Skin Tone in places like Brazil, India, Mexico, and Nigeria, according to a source familiar with the matter. Further details are expected soon in an academic research article.
The company will now expand its use of MST. Google Images will offer an option to sort makeup-related search results by skin tone based on the scale, and filters for people with more melanin are coming to Google Photos later this month. Should Google adopt the 10 skin tone scale across its product lines, it can have implications for fairly evaluating algorithms used in Google search results, Pixel smartphones, YouTube classification algorithms, Waymo self-driving cars, and more.
