Publication Date
2016
Document Type
Thesis
Committee Members
Yong Pei, Ph.D. (Advisor); Mateen Rizki, Ph.D. (Committee Member); Paul Bender, Ph.D. (Committee Member)
Degree Name
Master of Science (MS)
Abstract
In this thesis, we are primarily interested in exploring the HoloLens technologies for medical practices. Particularly, we will address the limitation of HoloLens’ capability in human body sensing, recognition and tracking. We will then introduce and demonstrate the use of Kinect to augment HoloLens’ sensory and processing capabilities in order to produce time and space-synchronized immersive environment with both virtual body and real body of the same patient for supporting distributed medical collaborations. Specifically, we are looking at a distributed solution in which we are collecting the patient body data from Kinect, followed by body recognition and position/motion tracking processing at a server; Then we will display and align the virtual body object, composed based on the Kinect-obtained patient body or body part position information, with the real patient body in the doctor’s augmented reality environment using HoloLens. We have implemented a prototype system to illustrate the distributed solution. Our experimental results using the prototype system demonstrate successful data flow and workflows. This results in effective time and space-synchronizations between the virtual and real bodies in the augmented reality environment. Through the prototype system, we can track movement, e.g., an arm, in all directions by the Kinect and the coordinates will be sent to HoloLens through which we will augment the real body with a virtual meshed body composed from Kinect-collected data. We have demonstrated that the solution and prototype can effectively address related limitations of HoloLens, and successfully present in real-time the movement of a recognized subject in the form of both virtual and real world human body, and aligned with each other. We believe this work has helped lay a solid foundation for future works to build complete virtual human atlas into the AR views, which may potentially assist surgeons with context-aware guidance by displaying him/her the instructions and feeding him with only the closely associated information through the course of operation. This distributed collaborative solution is of particular advantage for doctors to master faster and more safely advanced and complicated surgical techniques, like using advanced radiation therapy system for cancer treatments by eliminating the physical barriers such as distance, space, resource.
Page Count
54
Department or Program
Department of Computer Science and Engineering
Year Degree Awarded
2016
Copyright
Copyright 2016, all rights reserved. My ETD will be available under the "Fair Use" terms of copyright law.
ORCID ID
0000-0002-8185-8142
