Canesta was founded in April 1999 to develop “patent-pending user interface technology that renders a new paradigm for the interaction between humans and a broad spectrum of electronic devices including cell phones, PDAs, and Internet appliances.” Canesta is developing 3D-imaging sensor technology. Its mission is “to sight enable machines and digital devices with the world’s first electronic perception technology.”

In August 2000, Canesta closed first-round funding of $3.3M led by TechFund Capital and including Thales Corporate Ventures (formerly Thomson-CSF Ventures) and JP Morgan Partners (formerly Chase Capital Partners). In June 2001, Canesta received $17 million in a second round of financing led by Carlyle Venture Partners and including Apax Partners (formerly Patricof & Co. Ventures) and previous investors JP Morgan Partners and Thales. Canesta has roughly 40 employees.

Canesta has patented a method for forming electronic images of nearby objects in three dimensions. Unlike the sensors in digital still and video cameras that see the world in flat images, Canesta’s electronic perception technology can additionally compute the distance from the sensor for every pixel in the image, in real time. This allows Canesta-enabled devices to perceive and react to nearby objects or individuals. Because the sensors provide over fifty 3D frames per second, applications can track subtle movements and positions of nearby objects in real time. The company has filed or has been granted in excess of 45 patents.

When built into electronic devices, the technology can follow fingers – as in a keyboard application – or recognize faces – as in a security application. Moreover, it can assess volume or location of container contents for shipping applications.

Electronic Perception Technology can provide the ability to enable a rich set of user input through finger movements, pen movements, gestures, or body motion without the need for any kind of form factor. Electronic perception technology could enable game players to interact with the game in “real life” using the natural actions of his or her hands and body. Because electronic perception technology collects a rich set of 3D information about a field of view, security systems could gather and process a larger and more robust set of data about a given situation.

Inside the automobile, electronic perception technology could “see” the driver so that the driver does not have to “see” the controls. For instance, the driver could control the radio or air conditioning by using hand gestures, keeping his eyes on the road. The car could also inventory the driver’s physical profile to notify the airbag system if a large adult or small child is in the seat, enabling a proper level of inflation in the case of an accident.

Canesta’s electronic perception technology has two principal components: 3D electronic perception sensor chips and image processing software embedded in these chips.

Fundamentally, Canesta’s image sensor chip works in a manner similar to radar, where the distance to remote objects is calculated by measuring the time it takes an electronic burst of radio waves (or in Canesta’s case a burst of sub-visual light) to make the round trip from a transmitter to the object and back. The chips, which are not fooled by ambient light, either then time the duration it takes the pulse to reflect back to each pixel, using high speed on-chip timers, or simply count the number of returning photons, an indirect measure of the distance.

The result is an array of “distances” that provides a mathematically accurate, dynamic “relief” map of the surfaces being imaged. The image and distance information is then handed off to an on-chip processor running Canesta’s imaging software that further refines the 3D representation before sending it off chip to the OEM application.

Since Canesta’s software starts with a 3D “contour-map” view of the world, provided “for free” by the hardware, it has a substantial advantage over classical image processing software that must construct 3D representations using complex mathematics, and using images from multiple cameras or points of view. This reduction in complexity makes it possible to embed the application-independent portion of the processing software directly into the chips so they may be used in low-cost electronic devices. In addition, it accounts for the ability of the technology to compute 3D image maps at more than 50 frames per second.

The Canesta Keyboard, the first application for Canesta’s electronic perception technology, is a projection keyboard capable of being integrated by OEMs into portable electronic devices. The OEM device uses a tiny laser “pattern projector,” also developed by Canesta, to project the image of a full-sized keyboard onto a flat surface between the device and the user. The user can then type on this image and Canesta’s electronic perception technology will resolve the user’s finger movements into ordinary serial keystroke data.

Small size and low power consumption allows manufacturers to integrate Canesta Keyboard into their devices without significantly increasing the size or power consumption of the device. The large keys and familiar QWERTY layout of the Canesta Keyboard allows for input speeds in excess of 50 words-per-minute with error rates similar to that of a standard physical keyboard.

The Canesta Keyboard Perception Chipset includes all the modules needed for a projected keyboard product including the Canesta Keyboard Sensor Module, the Canesta Keyboard Light Source, and the Canesta Keyboard Pattern Projector.

The Canesta Keyboard Pattern Projector presents the image of the Canesta Keyboard. The Projector features a wide-angle lens so that a large pattern can be projected from the relatively low elevations associated with mobile devices. When activated, the Pattern Projector displays a standard QWERTY keyboard layout onto a flat surface. The Pattern Projector is independent of the electronic perception technology that drives the Canesta Keyboard.

The Canesta Keyboard Light Source houses the IR laser diode and associated optics into a single package. The IR Light Source operates by emitting a beam of infrared light, which is designed to overlap the area on which the Canesta Keyboard Pattern Projector displays the keyboard layout so that the user’s fingers are illuminated by the infrared light beam.

The Canesta Keyboard Sensor Module serves as the eyes of the Canesta Keyboard Perception Chipset and features Canesta’s electronic perception technology. It includes an integrated lens that performs all necessary filtering and focusing functions. This module operates by locating the user’s fingers in 3D space and tracking the intended keystrokes or mouse actions without assistance from the device host processor. Keystroke information can then be outputted to the device via an RS232 or USB interface.

Canesta has design wins in the PDA and cell phone arena and has shipped sample quantities of the Canesta Keyboard Perception Chipset to its customers, who are currently building products that incorporate this technology. Canesta expects its projection keyboard to be available from Canesta customers by the end of 2003.

NEC is evaluating Canesta’s electronic perception technology. Initially, NEC will focus on using the Canesta Keyboard Perception Chipset in mobile and wireless devices. At the WPC Expo in Tokyo, Japan, NEC exhibited a prototype projection keyboard using the Canesta Keyboard Perception Chipset. The keyboard was connected to an NEC Tablet PC.

Advanced Input Devices (www. advanced-input.com), a subsidiary of Esterline Technologies, will bring Canesta’s technology to a variety of vertical markets. AID has obtained rights to develop input systems for its own markets, as well as resell Canesta electronic perception components. Over the last 25 years AID has developed over 2000 input systems for more than 1000 companies. AID expects its first electronic perception technology based products to be in the market in early 2004.

AID believes there are a variety of opportunities for customized input devices that can be generated in effect “out of thin air” in any configuration and for any environment. These would include industrial or environmentally hazardous venues where physical input devices are obtrusive, a source of contamination, require a high degree of customization, or need to be reconfigurable from user to user or application to application.

For example, in a medical environment, a Canesta projection keyboard could be used in place of physical keyboards that collect biological contaminants and require constant attention to maintain sterility. The keyboard would be projected on an easily cleaned, benign surface such as stainless steel. In weight- or space-constrained military and aerospace environments, input devices could be projected onto any convenient location, without weight, and requiring no space when not in use.

Ongoing plans for the Canesta Keyboard Perception Chipset include reductions in size, power and cost of the components. Other plans include a series of generalized 3D sensing modules that have applications in many markets from automobile airbag deployment systems, to security intruder detection systems, gesture processing and control, and even consumer robotics.

Nazim Kareemi, Founder, President and CEO (formerly Executive VP of PenWare/MobiNetix, which is now a division of Symbol)

Abbas Rafii, Ph.D., Founder & Exec. VP (previously founder and VP of Software Engineering and Chief Architect at PenWare/MobiNetix)

Cyrus Bamji, Ph.D., Founder & CTO (previously an architect at Cadence)

Joep van Beurden, SVP of sales and marketing (most recently managing director and VP of business unity connectivity for Philips Components)

Shiraz Shivji, VP of Special Engineering Projects (formerly a consultant for Kaveri Networks and VP of Engineering at Advancel Logic)

James Goldberger, VP of Business Development (formerly a VP with Geoworks, a mobile device OS and wireless service provider)

Mike Van Meter, VP of Operations (previously VP of Client Services at Crossvue and Director of Operations for Crossvue’s parent company, Mobinetix/PenWare)

James Spare, VP of Product Marketing (previously business development director for the Microsoft TV Platform Division and COO of INBOXTV)

Patrick O’Connor, VP of Hardware Engineering (previously Platform Director for Parthus’ InfoStream ARM 920T-based SoC IP platform)

Sakuya Morimoto, director of business development in Asia and representative director (formerly with McKinsey & Co.)