Throughout the 2020 pandemic, the challenges and tragedies we experienced all round the world were highlighted through news outlets, social media, and even within our own personal circles. The effects of the pandemic will most certainly leave their mark on each of us, but it’s important to note that even the most trying of times can serve as opportunities for growth. As Plato famously said more than 2,200 years ago, “necessity is the mother of invention” and as we move through the endemic phase of Covid-19, the advancements made in the face of the adversities we faced over the course of the past few years are coming into focus. One of the most significant impacts is the degree to which we are more digitally connected. This shift has ripple effects that can be most easily seen in the continuation of virtual work environments, expansion of touchless POS in physical retail environments, and of course, online retail expansion.

In the height of the pandemic, the CDC expressed a need for facial recognition software that would enable consumers the ability to correctly size N95 and KN95 masks based on their individual features. While the utility of this need may have waned, the uses for Augmented Reality in our day to day lives are continuously expanding. Recognizing the need for viable AR tools across federal and commercial applications, Mobomo designers developed a prototype application as part of Mobomo Labs, a resource provided to Mobomo’s engineers with reinvested funds dedicated to R&D. Within a matter of weeks, the team had a user-friendly app that quickly and accurately captures the user’s unique facial topography, translates the results into a recommendation for a N95/KN95 mask that will more effectively control the spread of particulates and pathogens, and even connects the user with product options that match their results via Amazon.

3 basic features:

• Use your device's camera to measure your face
• Get a precise mask size according to the NIOSH table along with every detail of your face measurements.
• Get purchase options based on your results.

Aproach

Using native iOS development and selecting ARKit’s Face Geometry (a 3D mesh model that conforms to the shape of a scanned face) as their foundational tool. Due to the fact that ARKit doesn’t provide facial recognition points other than the eye’s pupil points by default and approximately 32 reference points are required to align with the 10 NIOSH PCA (the standards by which respirator masks are sized), the Mobomo team had to create and assign reference points that would form part of the mathematical equations used to map a 3D representation of the user’s exact facial features. That doesn’t seem too difficult until you realize that the 10 NIOSH PCA was designed when AR (Augmented Reality) was just an idea, and those 32 reference points were measured in real time and space.

• 
• 
• 
• 

Creating an AR tool that captures the specific features of each user’s face required that the Mobomo team define enough facial reference points within the 3D mesh model to produce accurate measurements. Leaning once more on a feature within ARKit, the team was able to get real-world positioning of a particular point on a person’s facial geometry (in xyz coordinates) to determine the distance between two reference points. The prototype ended up having 1220 facial reference points—each of which was associated with a particular facial feature and assigned a numerical value. For example, the vertex labeled “7” is a point on the tip of the nose, whereas “1047” is the lowermost point of the chin.

The complete tech stack used to create FitMask includes:

• Language: Swift 5+
• Library used for face scanning: ARKit
• Supported iOS versions: iOS 14+, iPadOS 14+
• Supported devices: iPhone X and later and iPad Pro models with the A12X Bionic chip (devices with the true-depth front-facing camera). No Android support.

Fit Mask - Technical hurdles

Exploring the opportunities with emerging technologies presents unique challenges and navigating those challenges is really the underlying purpose of Mobomo Labs. The hurdles the team navigated included:

• Lack of measurement standards: Face geometry vertices are currently not documented or standardized, therefore, the team had to develop their own. At some point in the future, Apple may change the position of the vertices within iOS/ARKit versions (though it has been two years since any updates have been made).
• Measurement accuracy: Creating perfectly accurate measurements using this approach cannot be guaranteed because of various factors, including the distance of the phone from the user’s face, facial hair, and the afore mentioned lack of measurement standards (if the vertices change in the future). In order to increase consistency in results, the team restricted the distance between the phone and the user’s fact to a known range, and programming feature detection to begin only when the phone and user’s face are parallel (made possible by calculating the Euler angle of the face geometry and setting a range). These restrictions we introduced immensely reduced variations in measurements.
• Limitations of the 10 NIOSH PCA: Head breadth is not one of the measurements in the 10 NIOSH PCA, and therefore cannot be calculated using this approach. ARKit provides only face geometry, hence parts of the head other than the face are outside the parameters of the measurement capabilities.

• Differences in Mask Style: Each mask has a particular root/reference point, and there is little to no control over how the strings/straps of each mask perform. Customization within the app based on exact product configuration may be necessary.

Ultimately, Mobomo’s design team solved some critical challenges to developing commercially viable AR tools while simultaneously creating a user-friendly app that takes the guess work (and effort) out of whether the masks you buy will get used or end up in a drawer – saving time, money, frustration – all while maximizing the utility of masking. Try it for yourself! FitMask can be found in the Apple App store

TRY IT NOW!