HAIBIN WU was born in Harbin, China, in 1977.He received the B.S. and M.S. degrees from the Harbin Institute of Technology, Harbin, China, in 2000 and 2002, respectively, and the Ph. D. degree in measuring and testing technologies and instruments from the Harbin University of Science and Technology, Harbin, in 2008.From 2009 to 2012, he held a post doctor with the Key Laboratory of Underwater Robot, Harbin Engineering University. From2014 to 2015, he was a Visiting Scholar with the Robot Perception and Action Laboratory, University of South Florida. He serves as the Vice Chairman of the Electromagnetic Measurement Information Processing Instrument Branch of the China Instrumentation Society, the Vice Chairman of the Heilongjiang Instrumentation Society, and the Director of the Heilongjiang Key Laboratory of Laser Spectroscopy Technology and Applications. His main research areas include machine vision, visual detection and image processing, and medical virtual reality. He has led multiple projects such as the National Natural Science Foundation of China, published over 140 papers on journals and international conferences, and published three academic works.
Application of Deep Learning and Implicit Representation in 3D Reconstruction and Augmented Reality Display of MIS Images
The most significant work in implementing the AR framework in MIS is to provide the global 3D visualization of the entire surgical scene. The researchers have introduced visual SLAM into the MIS scene to achieve endoscopic tracking and 3D modelling of the luminal scene, thus establishing a solid foundation for augmented reality visualization in surgical navigation systems. This report is oriented towards the demand for augmented reality display of 3D reconstruction of MIS images, based on a comprehensive analysis of existing deep learning and scene representation technologies. It delves into the core issues encountered in the practical process of 3D reconstruction of MIS images under traditional SLAM systems. The research content covers the extraction of a new type of internal cavity feature system based on deep learning, the application of monocular depth estimation in internal cavity scenes, the construction of a VD-SLAM system framework, the introduction and optimization of implicit representation technology, and the development of interactive projection display functions. The overarching objective of these studies is to address the challenges posed by unknown environments and scales, limited annotation data for endoscopic reconstruction, and suboptimal reconstruction outcomes. By progressively attaining detailed reconstruction of endoscopic scenes and augmented reality visualization, the research provides medical professionals with feedback on the depth of the endoscope and facilitates informed decision-making. Finally, this report conducts a comprehensive discussion and forward-looking analysis of the future prospects and challenges of augmented reality surgical navigation systems.
