New Research


Cybernetic Body and Senses Augmentation

- Smart projection enriches QOL -

Prof. Kosuke Sato, Assoc. Prof. Daisuke Iwai, Assis. Prof. Haruka Matsukura, Assis. Prof. Parinya Pu

Basically a projector is a device that projects video contents onto a flat screen. "Projection mapping" is a technique that allows the projector to freely change its shape and appearance by projecting various CG images on real buildings or 3D objects. While the our research group have been studying three-dimensional image engineering related to these projections for many years, we consider our own body to be augmented freely, from the viewpoint of enriching our quality of life and welfare; ex. to overcome the limitations of body size and to improve the function of the body.


Various sensors measure our hand, finger and arm movements. Virtual hands, fingers and arms are synthesized with real time CG so as to follow the movement, and they are projected onto our living space by a projector. Then,"rubber hand illusion" occurs that confuses the fake body with his/her own body, and the user perceives the virtual hand as his/her hand, that is "my hand". We call this projected "my hand", "Extended Hand" and are developing various application systems.


For example, it is a typing interface for users with hand tremor. First, the three-dimensional position of the fingertip is tracked, and the character key that the user intends to type is predicted. As a result, even if another character key is pressed, by converting it as the intended key and inputting it to the computer, the user no longer has to worry about frequent type mistakes due to tremor It was. In addition, instead of the real hand with tremor, we are also studying welfare augmented-human interface, which the computer reproduces and projects the tremor-free hand movement desired by the user. For elderly people and wheelchair users who can not extend their hands, we are developing a smart electric wheelchair equipped with a small projector and a pan-tilt mirror on an electric wheelchair. It is a new welfare mobility environment that makes their hand extend and facilitates finger pointing communication. In our laboratory, we are studying not only hands but also the future cybernetic systems that expand various body senses such as temperature sense, olfaction sense, touch feeling etc.

Sato Lab. HP

Extended Hand HP

◎About this site
Go back to page top