ľ¹Ï¸£ÀûÓ°ÊÓ

Recent years have witnessed great advancement in visual artificial intelligence (AI) research based on deep learning. In this thesis we investigate algorithms to design a high-quality simulator for mobile robots. We aim to narrow the gap between simulation and reality, generate infinitely many photo-realistic color-and-depth image pairs from arbitrary locations and allow  transferring algorithms that are developed and tested in simulation to physical platforms  without domain constraints.   

To achieve our goals, we have designed a view synthesis module used for our simulator to synthesize free-viewpoint photo-realistic  color-and-depth image pairs. Our approach combines depth refinement, adaptive view selection and layered 3D warping to lower the rendering complexity and improve the quality of  synthesized images.

Based on our simulator, we built a 3D dataset for benchmarking 6D object pose estimation which pays an important role in robotic grasping and manipulation research. Our dataset is freely distributed to research groups worldwide by the Shape Retrieval Challenge (SHREC) benchmark on 6D pose estimation. 

We conducted a variety of experiments to investigate the performance of different  pose estimation approaches proposed from our benchmark using different evaluation metrics. Apart from that, we propose a novel approach to further improve the performance of 6D object pose estimation by effectively computing hidden representations from color and depth images, and then fusing them properly with a graph attention network which fully exploits the relationship between visual and geometric features.