Projects

Telepresence

Video conferencing is currently the best commercially available telepresence technology, but it is far from the gold standard. Augmented reality (AR) systems, with their ability to superimpose virtual objects in real space and to allow spatial coherence, provide an opportunity for digital experiences that feel much closer to in-person interactions. 

Our effort to improve and understand AR telepresence is supported by two awards from National Science Foundation. If personal AR systems are to provide successful remote communication, it is critical that they integrate with their user’s lifestyle and workflow as unobtrusively as possible. Hence we are creating a telepresence system that utilizes a singular AR headset and RGB-D camera (a camera that records depth in addition to purely image) per person – in essence, a streamlined AR conferencing platform that doesn’t require a powerful computer.

The development of a high-quality and viable telepresence system will open the doors to countless new avenues of research. Just as virtual reality (VR) is currently used by scientists to study social psychology in highly-controlled settings, AR will enable research of real face-to-face interactions in new ways, in particular those involving integration of physical space into virtual meetings. 

For more information about these projects, contact Hanseul Jun ([email protected]), who has led the efforts on creating this technology and providing free access to the code.

A description of an early version of our system, which sends visual information from an RGB-D camera to an AR headset in real-time (i.e., high frame rate and low latency) over wireless networks with appropriate rendering and compression techniques is available here.

A new depth stream compression algorithm with allows for high performance is available here.

The source code for both systems is available here.

This work is supported by the National Science Foundation. You can access each NSF grant here and here.