Robotics and sensing technology Team

Principal Investigator: Professor Wei-Yen Wang

—

Summary

In this project, we develop three kinds of robotic systems with artificial intelligence (AI). First, we design a mechanism to draw Chinese calligraphy and automatically modify the motion of the robotic arm by the feedback image information. Moreover, we propose a learning theory to train the robotic arm to draw Chinese calligraphy in our self-built virtual machine. Second, this project has presented a method to develop systematic learning for robots from human demonstrations. The proposed robotic arm system is capable of active learning and inference. Last, we design and build a two-wheel self-balancing robot with human identification and following.

Keywords

Learning from Demonstration (LFD), Deep Learning, Robotic Calligraphy System, 6-Axis Articulated Robotic Arm, Two-Wheel Self-Balancing Robot.

Innovations

• We combine YOLOv2 and with an improved network structure to complete recognition and segmentation of Chinese calligraphy words.
• We propose a learning theory to train the robot to write in Chinese calligraphy in the simulation environment.
• We design a mechanism to draw Chinese calligraphy and automatically modify the motion of the robotic arm by the feedback image information.
• This project has presented a method to develop systematic learning for robots from human demonstrations.
• Regarding the construction of the database, we developed a novel generator. Then, we utilize a CNN-based object recognition model to distinguish between two types (i.e., user and non-user) of data.

Benefits

• A novel robotic calligraphy system is developed and can automatically modify the motion of the robotic arm for drawing Chinese calligraphy.
• A virtual calligraphy system is also built for the robot to perform try-outs. It learns how to write in Chinese calligraphy according to our novel reward and punishment rules.
• Our robotic arm system can not only be taught by humans through demonstrations, but it can also adjust its behavior by itself.
• We apply deep learning to human recognition and the development of the follow-up system.
• In the future, we do not only focus on vision-based tasks, but also consider the other human senses, such as hearing and touch.
• This project will implement the proposed intelligent algorithms at the integrated circuits level to increase the performance and flexibility, and it can promote robots application to more other domain.