Virtual Try On of clothes online has been the dream of eComm forever, when I worked at Warby loads of customer had asked for something similar, but (with total respect) the Product team at Warby were super disciplined and determined not to do it unless it could be done well.
For ages we saw versions that were rubbish but when FaceID came out for iPhone it gave us objective scale for the glasses and a ‘facemesh’ that we could track in 3D space relative to the user’s face. Using both of those we could then track the glasses relative to the face and was like 10x improvement compared to the the existing systems.
Once we got it working for one pair, it was ‘just’ a case of getting 3D model of nearly all the glasses Warby offered, getting those to device, getting that rendered in a scene, covering up any weird edges with a bit of shader magic and so so much more.
Brain Bernberg , Micheal Rakwoski, and Dee Goldberg did the real work on this project but was really fun
I love a stupid picture so this was the moment we launched it:
Our version was done on iOS but Warby do one now on the website, which I think show how far browsers have come along in recent years
https://patents.justia.com/patent/11783557
