Vulcan, Inc. were interested in the potential applications of merging mixed reality and computer vision.
LOOOK developed a HoloLens application to take the devices' video camera output, run it through an image classifier, and place mixed-reality markers with object names in the real world.
Object recognition was done with a state of the art convolutional network with a pre-trained model using the VOC image set. All computation was done on the HoloLens with recognition times of ~2.5s. Offloading computation to a server (or local desktop) would enable close to real-time computation.