Publication Date


Document Type


Committee Members

Yong Pei (Advisor), Mateen Rizki (Advisor), Paul Bender (Committee Member)

Degree Name

Master of Science (MS)


Augmented Reality (AR) places virtual contents along with real objects to enhance user perception in the physical environment. It is a technology that has shown great potential in many industries such as medicine, education and transportation. AR devices, such as Microsoft HoloLens, come with multi-modal interaction capabilities, such as voice, gesture and gaze, to support intuitive and reliable human-computer interactions. However, HoloLens gesture support is limited to simple gestures like tap, bloom and hold, it stops short to support meaningful interaction through full hand gesture and motion tracking, which may be critical in field operations of various industry practices in noisy environments, such as maintenance, repair, surgical operation and training. This thesis research intends to augment the HoloLens with fully-fledged hand gesture and motion tracking capabilities by interconnecting it with a Leap motion controller through the Internet, resulting in a functional prototype AR device capable of hand gesture/motion controlled operation. This prototype also enables the use of AR as a training device for skills that require hand operations, such as in surgery, repair, maintenance and etc. In this thesis, we present our system design and prototype implementation and testing, and also present a use case to illustrate its usage and value in real world working environments.

Page Count


Department or Program

Department of Computer Science and Engineering

Year Degree Awarded


Creative Commons License

Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.