NOVICE: Neural network Operated VIsion guided Control of Equipment

A robotic system using simple visual processing and controlled by neural networks is described. The robot performs docking and target reaching without prior geometric calibration of its components. All effects of control signals on the robot are learned by the controller through visual observation during a training period, and refined during actual operation. Minor changes in the system's configuration result in a brief period of degraded performance while the controller adapts to the new mappings.

It is shown that a neural network-based controller can perform rapidly and accurately, taking into account the non-linearities of various mapping functions. Such a controller is easy to train, tolerate of imprecise equipment configurations, and insensitive to camera perturbations following training. This method features real-time adaptivity to changes in mappings, and is simpler than traditional control techniques, which require the solution of the inverse perspective projection and inverse kinematics of the system.

Various operations including approaching, centering, paralling, reaching and adjusting are performed by the robot as it navigates towards the target. The robot attempts to grasp targets that are sufficiently close, or approach them while avoiding collisions with obstacles.

Publications

Video

A video clip demonstrates the training procedure and the robot in action.