SixthSense - wearable gestural interface

We've evolved over millions of years to sense the world around us. When we encounter something, someone or some place, we use our five natural senses to perceive information about it; that information helps us make decisions and chose the right actions to take. But arguably the most useful information that can help us make the right decision is not naturally perceivable with our five senses, namely the data, information and knowledge that mankind has accumulated about everything and which is increasingly all available online. Although the miniaturization of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no link between our digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen. 

SixthSense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. ‘SixthSense’ frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer.

The SixthSense prototype is comprised of a pocket projector, a mirror and a camera. 
The hardware components are coupled in a pendant like mobile wearable device.
Both the projector and the camera are connected to the mobile computing device in the user’s pocket. 
The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user's hand gestures and physical objects using computer-vision based techniques.

The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision techniques. 
The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces.
The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.

   

The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. 
  • The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by Multi-Touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements. 
  • The drawing application lets the user draw on any surface by tracking the fingertip movements of the user’s index finger. 
  • SixthSense also recognizes user’s freehand gestures (postures). For example, the SixthSense system implements a gestural camera that takes photos of the scene the user is looking at by detecting the ‘framing’ gesture. The user can stop by any surface or wall and flick through the photos he/she has taken. 
  • SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognizes those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing an ‘@’ symbol lets the user check his mail. 
  • The SixthSense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. 
  • The gesture of drawing a circle on the user’s wrist projects an analog watch.
Pranav Mistry is the genius behind Sixth Sense, a wearable device that enables new interactions between the real world and the world of data.
Pranav Mistry is a PhD student in the Fluid Interfaces Group at MIT's Media Lab. Before his studies at MIT, he worked with Microsoft as a UX researcher. Mistry is passionate about integrating the digital informational experience with our real-world interactions.

Some previous projects from Mistry's work at MIT includes intelligent sticky notes, Quickies, that can be searched and can send reminders; a pen that draws in 3D; and TaPuMa, a tangible public map that can act as Google of physical world. His research interests also include Gestural and Tangible Interaction, Ubiquitous Computing, AI, Machine Vision, Collective Intelligence and Robotics.

Posted via email from Edward Anil's Blog

Comments

Post a Comment

Popular posts from this blog

Innovation - The Japanese Soap Factory Story

A Japanese soap factory had a problem: they sometimes shipped empty boxes, without the soap inside. This was due to the way the production line was set up, and people with experience in designing production lines will tell you how difficult it is to have everything happen with timings so precise that every single unit coming out of it is perfect 100% of the time. Customers who come all the way to the supermarket would end up buying someone else’s product. Understanding how important that was, the CEO of the soap factory got the top people in the company together and they decided to start a new project, in which they would hire an external engineering company to solve their empty boxes problem, as their engineering department was already too stretched to take on any extra effort. The project followed the usual process: budget and project sponsor allocated, RFP, third-parties selected, and six months (and $8 million) later they had a fantastic solution — on time, on budget, high quality ...
Read more

OOPS Concept explained by Steve Jobs

Would you explain, in simple terms, exactly what object-oriented software is?
Objects are like people. They’re living, breathing things that have knowledge inside them about how to do things and have memory inside them so they can remember things. And rather than interacting with them at a very low level, you interact with them at a very high level of abstraction, like we’re doing right here. Here’s an example: If I’m your laundry object, you can give me your dirty clothes and send me a message that says, “Can you get my clothes laundered, please.” I happen to know where the best laundry place in San Francisco is. And I speak English, and I have dollars in my pockets. So I go out and hail a taxicab and tell the driver to take me to this place in San Francisco. I go get your clothes laundered, I jump back in the cab, I get back here. I give you your clean clothes and say, “Here are your clean clothes.” You have no idea how I did that. You have no knowledge of the laundry place.  May...
Read more

The truth behind whitening lotions and creams

Let's face it -- the average Indian is obsessed with a fair complexion. Most dusky Indian beauties would give an arm and a leg to acquire pasty white skin, which we refer to as 'gora-chitta'. It's quite sad to see fresh, smooth-skinned (albeit dark) people ruin the texture and glow of their complexions by using every possible cream, lotion, soap, serum, face pack and what-have-you to become fair. The only ones benefiting from this craze are the fairness product companies who are laughing all the way to the bank. Don't get me wrong -- there are chemicals which work and there are ways to get rid of unwanted pigmentation, but you need to have the correct knowledge of what's good for you and what's not. Check out the ingredients on your fairness product package and read on to get the low-down on what works and what doesn't: 1.Hydroquinone: This is one of the most effective whitening compounds known today. However concentrations used matter immensely and in...
Read more