Mirror, mirror on the wall – who’s the fairest of them all? A new patent coming out soon covers AI technology built to answer this age-old question. The patent will cover a system for providing an assessment of facial attractiveness using machine learning. US Patent Application Publication No. 2019/0272659 titled “System for Beauty, Cosmetic and Fashion Analysis” was just approved in March. One of the inventors is a professor at Northeastern University in MA, and the founder of Giaram, acquired by Shiseido in 2017. The concept of facial symmetry as the most reliable predictor of perceived beauty is well-known, but this invention utilizes an autoencoder-based framework to extract attractiveness-aware features in order to make an assessment of facial beauty. Beauty is said to be in the eye of the beholder, and therefore subjective; however, this invention transforms a subjective assessment into an objective one. The issue fee was paid on June 9, so the patent should publish shortly.
Northeastern University’s application of facial recognition to quantify beauty is not the first attempt to harness AI to turn a qualitative opinion of beauty into a quantitative decision. An AI judged beauty contest was held in 2016 where developers were invited to submit their AI software created to do an analysis of submitted images. China’s SenseTime reportedly also has developed software which ranks facial beauty.
With the capacity of neural networks and deep learning to rank beauty, many applications are feasible. Such a program could be used to grant access or special favors only to better looking people; powering a virtual bouncer that could exclude less attractive people from venues such as parties, events and gyms. It could also be used to guarantee a venue full of more attractive people by granting discounts in admission fees, hotel stays or cost of goods and services to those that the software judges to be more beautiful.
Further considering the idea of beauty being in the eye of the beholder, another invention uses deep learning to construct and apply a highly personalized face perception model. While Northeastern’s patent relies upon a universal measure of attractiveness (facial symmetry); U.S. Patent Publication No. 2019/0080012 titled “Method and System for Providing a Highly Personalized Recommendation Engine” recognizes that the ability to define a face as either desirable or undesirable is highly personal. For each individual, the perception of beauty is generated by a combination of many tangible and intangible factors. Tangible factors can include shape and size of particular features and symmetry. Intangible factors include a person’s prior encounters, emotional experiences, media influences and peer pressure. All these tangible and intangible factors led to a highly individualized decision mechanism being subconsciously applied. Using deep learning, a program is first trained to understand what an individual finds attractive by presenting images for the person to rank. Once the neural network has analyzed the choices, a personalized face perception model is created. Then, the model is used to analyze new images and predict desirability to the person, without the individual having to personally screen and rank the images. Several of the patent application’s claims to a computer-implemented method of constructing and using a personalized face perception model for a unique individual have already been allowed, so a patent could issue to the California inventors before the end of the year.
The obvious use of such a program is for a dating app. Singles can have potential dates chosen that are most suited to them, at least in terms of physical appeal, though it may be prudent to keep the saying “beauty is only skin deep” in mind, as AI-assisted over-reliance on superficial features ignores all the other elements essential to find the right significant other.
This technology may also be useful for directed marketing. Once a person’s preferences are known, it will be possible to show them ads including the types of people that have the features they’ve indicated to be more desirable or pleasing for product purchases (skin care, clothing etc). The knowledge that the program obtains about personal taste in looks could also be used to present targeted images in political messages or requests for charitable donations; or to design a virtual assistant or chatbot that a person would enjoy interacting with. While the movement to ban or pause use of facial recognition technology for surveillance grows, development will definitely continue at least for entertainment purposes. Because engaging AI for seemingly innocuous tasks like beauty ranking has so much potential to yield valuable user data, the rate of technological advances and more refined algorithms for facial recognition can only be expected to increase, even if privacy and bias concerns prevent security- related applications for now.
Beauty and the Law 