Knowledge-based 3-D vision systems

Purpose of the project

The aim of the project was to develop a knowledge-based 3D vision system. The question was how model knowledge can be used to assist 3D vision in typical applications. To give an answer to this question, we investigated ways to apply knowledge as well as flexible control schemes to the 3D vision problem in the frame of two specific vision environments: a development platform for 3D vision from range data on one hand and an experimental vision system for robotics and automated assembly on the other.

Key results

Results have been achieved on academic and industrial levels

	3D vision of arbitrary shaped objects. Our investigations led to a new approach for 3D vision from range data that is based on object modeling and geometric shape matching. It gets rid of the usual limitations on object structure and thus offers the possibility to recognize 3D objects of arbitrary shapes.
	Development platform for 3D vision from range data. This platform represents a powerful tool for experimental 3D vision from range data.
	Demonstrator for virtual reality (VR) robotics. This demonstrator shows the interaction of real and virtual worlds in robotics and demonstrates the function of knowledge-based 2D vision.
	Demonstrator for 3D vision in automated assembly. This demonstrator shows the automated assembly of scotch tape dispensers. It stresses the extensive use of knowledge for conducting and improving the assembly task. 3D vision allows flexible and robust part handling while VR task planning allows flexibility and reprogrammability of the assembly task. This demonstrator thus shows the feasibility and potential of knowledge-based vision in VR assisted tasks.

Methods and tools used

We developed software methods for object modeling and recognition such as methods for range data acquisition and preprocessing, geometric object matching, for reconstructing models from multiple views, as well as many control schemes for recognition. We also developed hardware tools like a special robot gripper. Tools used and adapted to the project include software environments like KBVision, Labo3D, Explore, Sense8, 3D-MBA, VRRE, range scanners like ABW and Biris, a 5-axis robot, a Matrox image processor as well as Sun, PC and SG computers.

Validation of results

Scientific work of the project gave rise to 11 international publications.

There was a strong interaction with other members of the SPP-IF's Vision module: prof. Thierry Pun, CUI, University of Geneva and Dr. Frank Ade, IfK, ETH Zurich. Meetings were held regularly for exchange of scientific and technical matters.

The demonstration for VR robotics performed together with NASA has been presented publicly. Television, radio and newspapers participated massively at this world premiere and since then, follow continuously the new progress.

The VR environment dedicated to robotics has been presented by IMT-EPFL as invited guest at several exhibitions (Computer'94, Industrial Handling 95, Ineltec'95).

The Intelligent Mechanism Group (IMG/NASA Ames - CA USA) uses several results of this project to perform high level control of robotized applications.

The demonstrator for 3D vision in automated assembly is now fully operational. It shows the potential of knowledge-based 3D vision when used in the frame of automated assembly controlled by VR.

Further perspectives

Knowledge-based vision thus appears as a significant means to feedback information from the real to the virtual world. As such, it has a large potential of development in future VR controlled tasks.

Applications are expected in assembly, inspection, teleoperation.

Recent international activity in teleoperation shows for instance that our approach could be extended to support multiple teleoperated experiments.

The goal of this project is to develop and validate the knowledge-based recognition approach both in a new development environment called Labo3D and in an experimental vision system for the inspection of industrial parts in assembly. In the following, we give a short description of the two systems as well as the three main modules which compose them.

References

Heinz Hügli, Christian Schütz, Dimitrios Semitekos, "Geometric matching for free-form 3D object recognition", Proc. ACCV'95, Vol III, pp 819-823, 5-8 December 1995
 

Christian Schütz & Heinz Hügli, "Free-form 3D object recognition", ISPRS, Proc. Third Conference on Optical 3-D Measurement Techniques, Vienna, Oct. 1995
 

Christian Schutz, Heinz Hugli, "Towards the recognition of 3D free-form objects", Proc. SPIE Int. Symposium on Intelligent Robots and Advanced Manufacturing, Intelligent Robots and Computer Vision XIV, Philadelphia, Oct. 1995
 

P. Gingins & H. Hügli, "Model-based 3D object recognition by a hybrid hypothesis generation and verification approach", Intelligent Robots and Computer Vision XIII: Algorithms and Computer Vision, Proc. SPIE, Vol. 2353, Boston, Oct. 1994
 

H. Hügli & P. Gingins, "Vision by range and intensity for model-based object recognition", Proc. 2nd Japan-France Congress on Mechatronics, Takamatsu, Japan, 1-3 Nov. 1994
 

Natonek E. & Baur C. "Recognition of 3D objects on complex background using model based vision and ranges images". Proc. IEEE Southwest Symp. on Image Analysis and Interpretation. Dallas, Apr. 21-22, 1994
 

Natonek, E., Zimmermann, Th., Flückiger, L.," Model based vision as feedback for virtual reality robotics environments", Proc. IEEE Virtual Reality Annual Symposium, North Carolina, 1995
 

Ch. Schütz & H. Hügli, "Recognition of 3-D objects with a closest point matching algorithm", "From Pixels to Sequences", Proc. conf. ISPRS, Zurich, Vol. 30, March 1995
 

H. Hügli, Ch.Schütz, Ch. Baur, E. Natonek, "Knowledge-based 3-D Vision Systems", Proc. Priority Programme Informatics Research, Information Conference Module 2, Yverdon, Dec 15-16, 1994