|
For more press information contact:
|
CANESTA, INC.
|
Computer or "machine" vision is one of the oldest and most widely-researched fields in the post-industrial era, with roots going back to the dawn of television. But today, as much an art as a science, practical applications have been constrained to either extremely narrowly defined problems – such as automatically focusing cameras, or the machine assembly of pre-specified components – or to government-sponsored applications, such as mapping or spy satellites, where vast sums of money can be thrown at extremely complex and subtle technology.
The key challenge for most applications is to separate out specific objects in a scene from the background, or from one another, given only the 2-dimensional or "flat" images provided by today's camera technologies.
Both digital still cameras and video cameras utilize a common image sensing methodology in which light reflecting from objects in a three dimensional scene is projected by a lens onto a tiny, flat, semiconductor chip that contains hundreds of thousands to millions of sensing pixels arranged in a rectangular array. Each of the pixels converts light of certain frequencies – or colors – to electronic charges, which are then "read" by adjacent electronic circuitry, and each of the pixels represents a specific, unique feature in the scene. But the only information that such devices provide is the brightness and color of the light illuminating any individual pixel at the time it was read, and the relative location of each pixel.
Perceiving specific objects or other identifiable features in a scene, given only a 2-dimensional array of a million or so tiny colored dots, is a profoundly difficult problem. Thousands of mathematical algorithms have been researched in the last 60 years that attempt such identification, but many depend upon either a complex physical setup, such as multiple cameras and controlled light sources, or upon a foreknowledge of all of the elements expected in a scene. And virtually all require substantial amounts of computing power.
Canesta has made a significant breakthrough in this regard by its invention of a low cost electronic perception technology. The technology includes new types of chip-based image sensors, similar in size, complexity and cost to commodity-priced video camera chips, that are uniquely able to resolve the three-dimensional features of a scene relevant to specific applications. That is, in addition to the brightness of a specific color of light reflecting from nearby objects being captured by each pixel, the distance from the illuminated object to each pixel is also determined. Since each pixel in the sensor is illuminated by a different feature in the scene being viewed, the result is a true, three-dimensional representation. In addition, the sensors can operate at over 30 frames per second, making possible real-time applications, such as a projection keyboard, automotive sensors that can judge accurately the size, shape, and position of the passenger, security systems which recognize people by a three-dimensional view of their face, and many others.
The resulting geometric information acquired continuously from the scene is so rich that, depending upon the application, very often much of the computation can be performed in real time by comparatively lightweight processors embedded right into the sensor chip itself. This is in contrast to contemporary systems that can take from several seconds to several minutes to compute a 3-D map of a single frame. For example, in Canesta's recently-announced Integrated Canesta Keyboard , a user's finger motions typing on a "projected" keyboard are resolved into actual keystrokes, and then into serial data, for use by an OEM device such as a cell phone, smartphone, or PDA.
The availability of technology of this power in such a tiny, low-cost format means that a wide variety of machines and electronic products will be able to perceive and react to nearby objects or individuals in real time through the medium of sight. The applications are endless.
Electronic perception technology is technology that permits machines, consumer and electronic devices, or virtually any other class of modern product to perceive and react to objects and individuals in the nearby environment in real time, particularly through the medium of "sight," utilizing low-cost, high-performance, embedded sensors and software.
What sets electronic perception technology apart from classical "computer vision" applications, for example, is that for the first time, actionable information can be developed in real time by observation of the nearby environment utilizing an ultra-low-cost sensor technology that is a size comparable to that found in nature. And as portable...
Electronic perception technology uses a four-layer model that is mirrored in the actual implementation of the technology:
These are actions that happen automatically in animals and humans, but are virtually non-existent in machines, except in specialized and usually expensive applications, supported typically with extraordinary amounts of computing power.
See means the formation of an image in some medium where it is available for further processing. In animals, sight occurs in the retina of the eye, as light excites the receptors located there. In electronics, particularly video cameras, sight occurs in light-sensitive pixels arrayed on the surface of a semiconductor device. Each pixel is a light sensitive diode that when stimulated by light, accumulates an electrical charge.
Perceive means to analyze or process an image such that specific features or objects can be ascertained. In human terms, an aboriginal culture might see letters painted on a wall, but not perceive them as letters, having no acquaintance with written language. By contrast, another culture might perceive them as letters, but be unable to identify them (for example, a westerner confronted with Chinese characters).
Identify means to recognize an object or feature such that an action or reaction is possible.
React means to actually take that action.
The goal of electronic perception technology is to make it possible for devices or applications of any complexity, from "lightweight" appliances, PDAs, cell phones, or games, to heavyweight vehicle control, airport security, or national security-class applications, to be able to perceive objects and features in the nearby environment such that identification and action are practical and possible.
Canesta has taken a leadership role in defining and implementing practical electronic perception technology with the development of low-cost, semiconductor-based image sensor chip technology and powerful embedded image processing software. Referencing the four-layer model, Canesta's technology – depending upon the specific application – provides actionable perceptions or identifications that permit third-party applications embedding Canesta's technology to react.
Electronic perception technology actually has two principal components. First are electronic perception sensor chips, and second, proprietary image processing software embedded in such chips. The following discussion relates to one class of such chips, those described in a U.S. patent recently granted Canesta.
Most people understand that light takes a finite time to travel between two points – that photons of light from two different stars, for example, may have started their journeys years, or even millennia apart. Since light travels essentially at a constant speed (299,792.458 kilometers per second, or about 11.8 inches in a nanosecond – a billionth of a second), if you know the time, you can calculate the distance.
The light illuminating each individual pixel in an image sensor comes from a different feature in the scene being viewed. Canesta recognized that if you could determine the amount of time that light takes to reach each pixel, you then could calculate with certainty the exact distance to that feature. In other words, you could develop a three-dimensional "relief" map of the surfaces in the scene. In three dimensions, objects previously indistinguishable from the background, for example, metaphorically "pop" out. For a broad class of applications, this proves extremely helpful in reducing the mathematical and physical complexity that has plagued computer vision applications from the start.
In the recently-granted U.S. patent (#6,323,942) entitled "CMOS-Compatible Three-Dimensional Image Sensor IC," Canesta describes several of its inventions for "timing" the travel time of light to a unique, new class of low-cost sensor chips.
Fundamentally, the chips work in a manner similar to RADAR (RAdio Detection And Ranging), where the distance to remote objects is calculated by measuring the time it takes an electronic burst of radio waves to make the round trip from a transmitting antenna to a reflective object (like a metal airplane) and back. In the case of these chips, however, a burst of unobtrusive light is transmitted instead.
The chips, which are not fooled by ambient light, either then time the duration it takes the pulse to reflect back to each pixel, using high speed timers, in one method, or simply count the number of returning photons within a precise time interval – an indirect measure of the distance, in another. In either case, the result is an array of "distances" – updated as often as 50 times a second or more – that provides a mathematically-accurate, dynamic "relief" map of the surfaces being imaged. The image and distance information is then handed off to an on-chip processor running Canesta's proprietary imaging software that further refines the 3-D representation before sending it off chip to the OEM application.
The second component of Canesta's electronic perception technology is a robust body of new, "industrial grade" software designed for real-world applications.
Since Canesta's software starts with a three-dimensional view of the world, provided "for free" by the hardware, it has a substantial advantage over classical image processing software that struggles to construct three-dimensional representations using complex mathematics, and using images from multiple cameras or points of view. This significant reduction in complexity makes it possible to embed the application-independent portion of the processing software directly into the chips themselves so they may be used in the most modestly-priced, and even pocket-sized, electronic devices. In addition, it accounts for the remarkable ability of the technology to compute 3-dimensional image maps at over 50 frames per second; remarkable compared to existing technology that can take from several seconds to several minutes to generate a 3-dimensional representation of a single, static frame.
Finally, with an expectation of its use not only in mission critical applications such as medical instrumentation, automotive, or security, but in the notoriously unforgiving consumer products arena, Canesta's software features tolerant, self calibrating algorithms, and is built using a layered software model that features compact code, for ease of embedding in modest applications.
Although the foregoing discussion has focused on two specific electronic perception chip designs, Canesta, with over 20 hardware and software patents filed, and with more on the way, has substantial research and development initiatives underway that will result in future technology disclosures, product announcements, and strategic alliances well beyond what is discussed here.
Applications for sight-enabled devices, those made possible by current or future low-cost electronic perception technology, fall broadly into three categories.
Certain of these applications are immediately possible using Canesta's first generation electronic perception technology; others will become practical with technology under development today, or future embodiments of electronic perception technology. However, ultimately, the applications are limited only by one's imagination. If you sight-enable an electronic device or machine, it suddenly can react to and interact with the world in an entirely new way. And if the material cost of adding this capability to virtually any product is just a few dollars, then designers will, literally, let their imaginations run wild.
Recently, Canesta has announced the Integrated Canesta Keyboard™, the world's first fully-integrated projection keyboard capable of being integrated by OEMs into smartphones, cell phones, PDAs, or other mobile or wireless devices. When equipped with the Integrated Canesta Keyboard, the OEM device uses a tiny laser "pattern projector" – also developed by Canesta – to project the image of a full-sized keyboard onto a convenient flat surface between the device and the user, such as a tabletop or briefcase. The user "types" on this image. Canesta's electronic perception technology then resolves the user's finger movements in real time into ordinary serial keystroke data that is easily utilized by the wireless or mobile device. The Integrated Canesta Keyboard is made possible by the first application-specific chipset embodying electronic perception technology, called the Canesta Keyboard Perception Chipset™. The chipset contains a tiny 3-D sensor module, an equally tiny keyboard pattern projector, and a small infrared light source, and can be integrated by OEMs into their products with very little cost, power, size or weight impact. Canesta also offers an OEM development toolkit that aids OEMs in prototyping projection keyboard designs and layouts. The Canesta Keyboard Perception Chipset may be ordered with a standard QWERTY keyboard, or any custom Roman or non-roman keyboard that the OEM has designed with the development kit.
The Integrated Canesta Keyboard is an important, new application that resolves the "missing link" with mobile and wireless devices - the ability to do "true" data input. Although in common use, input solutions such as thumb keyboards or handwriting recognition are profoundly limited in their ability to support typing-intensive applications. An integrated projection keyboard means that the mobile or wireless device can now support applications that would ordinarily only be practical with a full-sized, mechanical keyboard. This is good news for OEMs that wish to differentiate their products with important, new mobility applications, and good news for service providers, that now can offer value added services to their subscribers, including "leave your notebook PC at home."
Canesta's electronic perception technology will be delivered in the form of application-specific sensor chips, embedded software, and application program interfaces. The first such application, the revolutionary fully-integrated projection keyboard described above, was announced September 18, 2002, and the chipset that makes it possible, the Canesta Keyboard Perception Chipset, is available from Canesta in sample quantities.
Canesta is the inventor of a revolutionary, low-cost electronic perception technology that enables ordinary electronic devices to perceive and react to nearby objects or individuals in real time.
When sight-enabled with Canesta's unique electronic perception chips and software that sense the environment as 3-dimensional moving images, consumer, automotive, industrial, and medical products will gain functionality and ease of use not possible in an era when electronics were blind.
Canesta was founded in April 1999, and is located in San Jose, CA. The company has filed or has been granted in excess of 30 patents. Investment to date exceeds $20 million, by Carlyle Venture Partners, Apax Partners (formerly Patricof & Co Ventures, Inc.), JP Morgan Partners (formerly Chase Capital Partners), TechFund Capital, and Thales Corporate Ventures (formerly Thomson-CSF Ventures.) Canesta has over 40 employees.
-30-