A project, still in the development stage, and known as Kinsight uses a depth camera-based system to retrieve lost objects in and around the house, like wallets and keys. Using Microsoft’s Kinect sensors, this project has been shown to work when used in real-life by two computer science researchers at a conference in China.
One of the researchers describes its use, in saying, “Imagine if we had a system that could keep account of all the objects that we interact with in our daily lives. By keeping track of the locations of the objects, we could build a smart search engine for our home that could answer queries like – where are my eye glasses, or my TV-remote, or my wallet?”. The researchers also understood that building a computer program to track several objects in real-time would be processor-intensive, and chose a ‘common sense’ approach instead.

This system tracks humans first and then locates objects whose locations might have changed in their immediate surroundings. It is based on the principle that it is only when humans move objects that their position changes.

But that’s not all – this project also involves the use of an algorithm that helped the computer learn the appearance of almost 48 objects, and where they were most likely to be found – amounting to almost 80 locations around a house.

The limitations of this project are mainly due to the Kinect sensors, which allows the system to see object at a distance of no more than 11 feet. Also, errors were possible if the objects were too small, placed far away, and transparent or located too close to each other. Some of the objects used in the demonstration were knives, forks, keys and quite singularly, even a Rubix cube.