INTRODUCTION

At present time the most popular pointing devices are the computer mouse and the sensor pad, the latter is mostly used in notebooks. Both pointing devices have substantial drawbacks. For example, a computer mouse has large size, needs flat surface to operate, period movement from the keyboard to the mouse and back is necessary, intense use of the index finger (or another one) for clicking may lead to RSI (Repetitive Strain Injuries), significant load on hand is a problem because of periodical repositioning of the mouse, additional element (e.g. wheel) is necessary for scrolling what destroys uniformity of pointing operations and adds mechanical load on the pointing finger. In case of the sensor pad one of the problems is the intense use of the pointing finger, including clicking on the Pad’s surface (“surface clicking”) what leads to quick finger fatigue, the sensor pad also needs significant real estate for placing in a notebook, significant movement from keyboard to the sensor pad remains, operation is not as precise as that of the mouse, left surface clicking can not be undone, differing from mechanical clicking, when after pressing the button the clicking can be canceled via moving the pointer beyond the button on the screen, separate area for scrolling is confusing for many customers, so unintended pointing operation or unintended scrolling can take place as a result.

Besides, these pointing devices don’t allow comfortable inputting of other information, e.g. text, using three-dimensional movements of the user’s hand.

This work describes such input device that is free from the mentioned above drawbacks of both mouse and touchpad and that allows other input operations, e.g. text input, via registration of three-dimensional movements of the user’s hand. The device also should be equally comfortable for both left- and right-handed people with minimal risk of the RSI (Repetitive Strain Injury) at the same time.


DESCRIPTION
 

Fig. 1 shows general view of the basic embodiment of the input device constructed in accordance with presented concept.

Fig. 1
 

This pointing device consists of a two-dimensional Photo-sensor, a Lens for projecting image of the user’s hand on the Photosensor, at least one Source of Light to illuminate the hand, and a Hand Presence Sensor to detect touch or pressure of the hand’s wrist during work. The Source of Light is preferably an infrared Light Emitting Diode (LED) to avoid disturbing light beams and thus to make work of the input device more comfortable for the user.

The Photosensor can be an industrial white & black video/photo matrix with e.g. 640x480 pixels resolution. The resolution influences sensitivity of the pointing device in a large extent and should be chosen depending on the desired sensitivity of the input device.

The Hand Presence Sensor can be e.g. a capacitive one to detect touch/approach of the wrist of the hand or the resistive one to detect pressure of the hand on its surface.

The pointing device can comprise more than one Source of Light to provide more uniform illumination of all the lower surface of the hand in wide range of its positions above the Lens.

Angle of view "alpha" of the Lens is such that tips of all fingers of the hand and at least part of its palm are projected on the surface of the Photo-sensor in all the range of working positions of the hand under condition that wrist of the hand touches the Hand Presence Sensor.

Angle of illumination "beta" of the Source of Light is not less than the angle of view "alpha" of the Lens. Light efficacy of the Source of Light and sensitivity of the Photo-sensor are such that the Source of Light provides sufficient level of illumination of the hand to effectively register its image on the surface of the Photo-sensor in wide range of ambient light intensity and in all range of working positions of the hand under condition that its wrist touches the Hand Presence Sensor.

Depth of field of the Lens is such that image of the tips of the fingers of the hand is sharp in the working range of the finger’s incline – from horizontal to about 45 degrees in vertical plane.

The Photo-sensor, the Source of Light, and the Hand Presence Sensor electrically interact with an Electronic Module which serves as (a) controller of the Photo-Sensor, of the Source of Light, and of the Hand Presence Sensor and (b) processing unit to process video data supplied by the Photo-Sensor.

The Electronic Module interacts with other devices, such as a personal computer to deliver information about position of the hand, and its fingers in particular, in relation to the Lens in real time with purpose of following calculation of position of the pointer on the screen of the display of the personal computer or for other purposes. Such calculation can be done in the Electronic Module itself as well.

The input device can be done as a separate module connecting to a PC using cord or radio waves similarly to connection of other pointing devices, e.g. mice or sensor pads. It can also be built into the PC’s or notebook’s keyboard as shown on Fig. 2.
 

Fig. 2

OPERATION
 

Essence of operation of the presented input device lies in processing of image of the hand (Fig. 1) formed by the Lens on the surface of the Photo-sensor. Approximate derived image of the hand in negative form (brighter regions are shown in black) is shown on Fig. 5 below. Perspective distortion of the image is corrected. Electronic module analyzes movement of the fingers of the hand mostly gathering information about movements of the fingers’ tips, movement of the index finger prevails. This means that in whole movement of the hand a movement of the tip of the index finger has the biggest weight though movement of other fingers is also considered. It is important to note that during operation changes in the fingers’ size and brightness are also considered what allows deriving three-dimensional (i.e. in both horizontal and vertical planes) movement of the hand.

Fig. 3 A and B explain “vertical” movement of the hand in relation to lens that leads to vertical movement of the pointer on the computer screen.

Fig. 3
 

Fig. 4 A and B explain “horizontal” movement of the hand in relation to the Lens that leads to horizontal movement of the pointer on the computer screen.

Fig. 4
 

Arbitrary movement of the pointer on the computer screen is a result of an arbitrary combination of simultaneous “vertical” and “horizontal” movements of the hand while it activates (touches) the Hand Presence Sensor (fingers touch nothing at this time). The movement is being registered only when the hand touches the Hand Presence Sensor. Release of the Hand Presence Sensor is used to (a) stop the pointing operation or (b) reposition the hand to continue the pointing operation. The hand’s movements are ignored if the Hand Presence Sensor is not activated (touched) by the hand.

Clicking, analogous to left and right clicking of a computer mouse, is provided by vertical movement of the index finger from its neutral position as shown on Fig. 5. Left clicking is provided by substantial moving of the index finger downward while Right clicking is provided by substantial moving of the index finger upward from the neutral position.
 

Fig. 5

Fig. 5 also shows increase of the size of the index finger of the hand during left clicking, brightness of the tip of the index finger also increases, at the same time once can notice decrease of the size of the index finger of the hand during right clicking, brightness of the tip of the index finger also decreases.

In order to process images of the hand, like those shown on Fig. 3 - 5, any of the image-processing methods well-known to those skilled in the art can be used.

Present input device can also use hand gestures to switch to different regimes. For example, change of position of thumb from its neutral position (see Fig. 6, left image) to position shown on the middle image of Fig. 6 can activate scrolling regime when moving of the hand results in vertical/horizontal scrolling of image on the computer monitor.

 

Fig. 6

Similarly, change of position of thumb from its neutral position to position shown on the right image of Fig. 6 can activate “toggle regime”, when “vertical” movements of the hand result in toggle between different open applications while “horizontal” movements of the hand 3 result in toggle between open documents within same application.



THE UKAZKA AS TEXT INPUT DEVICE

The input device can also be located in a PDA (Personal Digital Assistant) to provide pointing operations as well as comfortable text input. The latter can be realized as described in second embodiment of the input device shown on Fig. 7.

Fig. 7

In this case two identical input devices, each of them comprising the Photo-sensor, the Source of Light, and Hand Presence Sensor are located on the sides of the PDA’s housing as shown on Fig. 7. Angle of view "alpha" of each of the Lenses is such that tips of all fingers of the corresponding hand and at least part of its palm can be registered by the corresponding input device in all range of working positions of the hand under condition that part of the hand touches the Hand Presence Sensor as shown on Fig. 7.

Considering pointing operations, work of the present input device in its second embodiment (Fig. 7) is similar to its operation in the basic embodiment (Fig. 1). At the same time, ability of the input device to register three-dimensional movements of the hand allows typing in the air, imitating touch-typing over imagined QWERTY keyboard. Both Hand Presence Sensors should be touched simultaneously to type, at the same time the hands should take “typing position” similar to that during touch-typing on usual keyboard. Unusual for typing hand movements and gestures are ignored. The typing takes place in the air without touching anything, it is absolutely similar to typing over imagined QWERTY keyboard, some hints can be shown on the PDA’s screen to ease process of typing and learning. Any of the hands can be used for pointing operations any time, it is enough to touch the corresponding Hand Presence Sensor, while another one should be released, to start pointing operations by the touching hand. By other words, touch of any single Hand Presence Sensor activates pointing operations, while simultaneous touch of both Hand Presence Sensors activates typing regime.