A framework of teleoperated and stereo vision guided mobile manipulation for industrial automation

Smart and flexible manufacturing requests the adoption of industrial mobile manipulators in factory. The goal of autonomous mobile manipulation is the execution of complex manipulation tasks in unstructured and dynamic environments. It is significant that a mobile manipulator is able to detect and grasp the object in a fast and accurate manner. In this research, we developed a stereo vision system providing qualified point cloud data of the object. A modified and improved iterative closest point algorithm is applied to recognize the targeted object greatly avoiding the local minimum in template matching. Moreover, a stereo vision guided teleoperation control algorithm using virtual fixtures technology is adopted to enhance robot teaching ability. Combining these two functions, the mobile manipulator is able to learn semi-autonomously and work autonomously. The key components and the system performance are then tested and proved in both simulation and experiments.