Smoke and Mirrors: Digitized Beauty’s Latest Inventions

Harnessing the power of artificial intelligence, innovation for the beauty industry is accelerating at warp speed. Here’s a brief description of a few of the latest developments related to using machine learning for cosmetics from U.S. patent applications published May 28.

Computer-Assisted Makeup Application

If you’d like to have the same makeup look that Jennifer Aniston rocked at the Emmy’s, TCMS Transparent Beauty of Texas’s invention can grant your wish. Building on the conventional mirror apps that are already available for your phone, algorithms have been created to allow you to choose Jennifer’s look, or other specific styles like an eighties look or a Goth look. Your phone will direct precisely where to apply makeup to achieve that look by the program. For those who don’t feel confident enough to put the makeup on themselves, a device or applicator to automatically apply a selected look is also discussed in U.S. Patent Application Publication No. 2020/0167983. The image below is from the patent application and shows another possible use of the invention – guidance to hide blemishes. You’ll be shown where they are, and after you’ve applied a concealer, they system checks your work and lets you know if you need to apply more makeup.

The TCMS Transparent Beauty application also discusses the interesting concept of a “looks ecosystem”, which is defined as a social internet-based environment that allows users to personalize their appearances. A looks ecosystem can establish an interactive relationship among and between users, style professionals and cosmetic producers of the exchange of styles, images, products, software applications and advice. The looks ecosystem can be used for group makeovers too.

Computer-Assisted Virtual Makeup

Maybe you want to look great, but not put in the time and effort involved to actually apply your makeup. Of course, there’s an app for that, and patent protection is being sought for it. U.S. Patent Application Publication No. 2020/0169674 is related to Shiseido’s Tele Beauty app (for putting virtual makeup on your image in a video call). The publication describes an information processing apparatus that allows you to be seen as though you are wearing makeup, even if you are not, taking the next logical steps from services like Zoom’s Touch Up My Appearance feature. “Phygital” fashion shows necessitated by social distancing rules could benefit from this technology, which could replace a makeup artist’s close contact for long periods of time with a catwalk model. The details of the patent application include other novel aspects. In addition to assisting you with making a choice about what makeup look to apply; Shiseido’s information processing apparatus can potentially also choose a makeup look for you, by mining user information about you such as your mental state, or your clothing and cosmetics purchase history from your phone or available online.

Digital Makeup Removal from an Image

In addition to helping you apply makeup; or applying makeup digitally, AI can also be used to subtract makeup from a facial image. Would you like to know what that supermodel or movie star looks like IRL? Northeastern University has a patent (U.S. Patent No. 10,339,685) that describes a way to analyze an image in order to detect the presence of makeup and remove it digitally from the image. It can be used to determine what type of makeup is used in a photo – everyday makeup, fashion makeup, television makeup, theatrical makeup etc. Besides discovering the truth about what a person really looks like, this system can analyze a photo for you; and provide the information to assist you to create the same look for yourself. It could also be used to assess and improve the skills of a makeup artist. As a natural extension of the system’s capabilities, it could find use for security, in uncovering the actual face of a person to be identified if makeup is used as a disguise to defeat facial recognition.

Smoke and mirrors effects courtesy of AI can bring new convenience and ease to the enhancement of beauty, but may also further blur the distinctions between reality and artifice for video and images.