Thursday, May 7, 2009

sixth sense


'SixthSense' is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information.

We've evolved over millions of years to sense the world around us. When we encounter something, someone or some place, we use our five natural senses to perceive information about it; that information helps us make decisions and chose the right actions to take. But arguably the most useful information that can help us make the right decision is not naturally perceivable with our five senses, namely the data, information and knowledge that mankind has accumulated about everything and which is increasingly all available online. Although the miniaturization of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no link between our digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen. SixthSense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. ‘SixthSense’ frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer.

The SixthSense prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user's hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.

The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by Multi-Touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements. The drawing application lets the user draw on any surface by tracking the fingertip movements of the user’s index finger. SixthSense also recognizes user’s freehand gestures (postures). For example, the SixthSense system implements a gestural camera that takes photos of the scene the user is looking at by detecting the ‘framing’ gesture. The user can stop by any surface or wall and flick through the photos he/she has taken. SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognizes those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing an ‘@’ symbol lets the user check his mail. The SixthSense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the user’s wrist projects an analog watch.

The current prototype system costs approximate $350 to build.


PICTURES









. . . some more pictures are coming soon.


VIDEOS







. . . more videos are coming soon, too.

INVENT


A conceptual gap exists between the representations that people use in their minds when thinking about a problem and the representations that Computers will accept when they are programmed. For most people learning programming, this gap is as wide as the Grand Canyon . invent is my research project under the guidance of Prof. Ravi Poovaiah initiated with the vision to help solve the same. I have taken children as my users. The initial goal of invent is to design an intuitive programming environment for children. The key ideas are to use representations in the computer that are analogous to the real world objects being represented or letting children create them and to allow those representations to be directly manipulated in the process of programming. The child can create objects, give them properties and attributes & use those objects to create their world creating challenges & solutions to them. invent is an attempt to make programming more like thinking.
In brief, invent='Imagine...Explore...&...Learn'






DATAG2.02

With the aim of providing a more intuitive and physical interaction with systems and machines, the project DATAG2.02 explores a new mode of human-machine interaction. DATAG2.02 consist an artificial gadget kept in the real world. The user will use this glove like gadget to interact with objects and perform the desired action in the virtual world. The user's actions are mapped by the gadget and the virtual world experiences the changes done by user as well. The overall goal of the project was to explore the novel way to interact with virtual worlds and systems with inaccessibility, like a robo on moon or a crane hand lifting huge rocks. I won award at India level Open Hardware Contest for DATAG2.02. It is also awarded at Ingenium2002 for its innovative perspective.


MARBO


My research project with Microsoft Research - MARBO, is the concept of a communication device for children helping them share their feelings, events and learning with fun. After a deep study & analysis of children's world, we came up with the product called MARBO. With the initial goal of making the social circle larger, building trust among children, making learning interesting & fun for them, MARBO has been extended to a level that, it is ‘the' device for children. MARBO is basically inspired by the metaphor of the marbles, a traditionally popular toy in India . MARBO is a device through which children can communicate with each other to freely express their feelings, as well as have fun by exchanging of ideas, emotions and having collaborative and interactive learning. With its spherical screen and detachable marbles (digital memories), MARBO provides a new mode of interaction. For its wonderful unique design concept and social aspects, MARBO is awarded 1st prize in Design Contest at Innovation Fair.

projectCHILD

projectCHILD explores the very basic but interesting cognitive aspects of human mind, about learning and remembering that 2+2=4 or such similar concepts. The core initiative to the project is the thought that, there are some tasks which we human perform faster that these computers and robots. To make them do the same with the same efficiency we can try making them learn as we have learnt. Through projectCHILD I tried to teach computers of my lab that 2+2=4, but by the way a child is taught by his teacher, mom or friends(sometimes 2+2=5 or 2+2=?). These children (machines) at the end of the project learnt that 2+2=4 as a child learn the same.
SunFlower

interest in exploring nature led me to wonderful idea of having artificial sunflowers. With simple op-amp 741 circuitry & photo sensors I implemented a sunflower which could track the sun. Though later the concept was very efficiently used in solar panels at my institute as the panels could get the most efficient angle each time, at the core, there was my interest in designing electronic interactive experiences.


Sandesh

In the search of work, to earn their bread lots of people in India come to metros from their distant villages. Sandesh addresses the need of communication of such people. It is an attempt to connect their parents & relatives to them where no reliable or efficient modes of communication are available. Besides the illiteracy in reading and writing in standard languages like Hindi and English, inaccessibility of communication services like phone and mails creates the major problem of other end(village) connectivity. To help solve it Sandesh suggests a new system using existing PSTN network and simple interaction methods. Sandesh contains a message receiving unit at rural end and kiosks at metro end with visual aids. It uses of print/sound based medium to convey messages.

Ghost in the machine

It is by logic we solve, but it is by intuition we create." Questioning the nature of creativity and experimenting with my desktop computer to create pieces of art ended with the discovery of a ghost in my machine. From studying drawings done by children to the robots playing chess and painting on canvas it is the result of an experimental formative approach. It is an attempt to answer some of the questions arose during my journey of enjoying children drawing mountains, sun and river and those happy moments of programming my desktop machine. With a vision of questioning the nature of creativity the idea was to generate pieces of so called art or creativity in collaboration with machines. "Creative machines?" "Can machines be creative?" My thoughts and attempts to explore the same, to seek in the field with a positive approach of making machines creative, which can generate pieces of so called "art", are also documented as a paper titled "ghost in the machine'. I think Life is "Trying things to see if they work'.

RoadRunner 2.01

RoadRunner was one of my attempts to make computer games when I was in second year of computer engineering. The interesting feature of the project is that the game gives user a 3D game experience, though it is made by 2D graphics techniques. Study of human mind's perspective about 3D and algorithms to generate 3D look from simple 2D graphics primitives of C are at the core of the project.



VET

New technologies have created new opportunities to look forward the vision of 'Technology to Masses' from different perspectives. Not technology, technology in 'meaningful manner', only will help solve the current problems. Lack of proper scheduling, decision making support, planning and resource management seems the driving force for the inefficiency in veterinary services provided by Amul in Gujarat . The inefficiency in existing scheduling system ends in inefficient use of system resources and improper handling of emergencies. With the proposal of a revised system based on decentralization of visit request registering and a map-based interface of VET for scheduling, the project aims to solve the problems to a level. With the help of a story based design approach throughout the project and evaluations at various stages helped define the efficacy of the suggested solution.

Sthiti

I think Sthiti was my first project which help me broaden my perspective to look at technology. Now, in this age when we have enough of things to explore, it is not important that what we can do. Important is what we should do. Sthiti was an attempt to help people in finding things, people, places along with basic information they need. It is a system design of having a local positioning and information system in the small regions like university campus, hospital, shopping mall, multiplex theatre, etc. A small handheld device or user's mobile phone can be used as a medium to provide this location specific information or to help them finding their way. The project gave me an opportunity not only to explore networking, mobile technologies but also user interaction design.

Akshar

Data entry methods for Indian languages like Hindi, Gujarati and in general are not keyboard friendly and their entry using QWERTY keyboards is cumbersome and complicated, it involves the use of multiple shift modes. The same applies to the input methods on mobile phone and other interactive digital kiosk like systems. There is a great need for practical input mechanism/scheme specifically for various Indian languages. With the vision of ‘Innovate for India’, Akshar is basically an attempt to have ‘the’ input mechanism for inputting Indic scripts in Digital devices like Mobile phones, Kiosk booths, WLLs, Interactive TVs, VEDa or Personal Computers. It is observed that most of the scripts in India have their base in Brahmi script. Akshar uses unique features of grouping of 5 consonants in such script. Study has shown that this new mechanism is far better than any other input methods available around today. The reason for that is, without any consideration of features of Indic scripts, these methods are taken as they were from Latin based script mechanisms