Researchers at America’s Cornell University have programmed a robot to predict what you’re about to do and help you achieve it. Using a Microsoft Kinect sensor, the robot watches your body movements, then accesses a video database of around120 household activities — ranging from putting food in the microwave, eating, brushing teeth to pouring a drink — to predict what your actions will be a few seconds into the future. It can then make a decision about what you’re likely to do next, and what it can do to assist you in that task.
Understanding when to pour a beer or offer assistance opening a refrigerator door can be difficult for a robot because of the many variables it encounters while assessing the situation. However, in tests, the robot made correct predictions 82 percent of the time when looking one second into the future, 71 percent correct for three seconds and 57 percent correct for 10 seconds.
“Even though humans are predictable, they are only predictable part of the time,” says Ashutosh Saxena, Cornell professor of computer science and co-author of the study. “The future would be to figure out how the robot plans its action. Right now we are almost hard-coding the responses, but there should be a way for the robot to learn how to respond.”
"We extract the general principles of how people behave," he explains. "Drinking coffee is a big activity, but there are several parts to it." The robot builds a "vocabulary" of such small parts that it can put together in various ways to recognize a variety of big activities”.
Having better telepresence robotics, creating robots that work better alongside humans in factories, offices, or hospitals and can aid the elderly or disabled in their own homes are obvious applications for the new technology which is available under an open source license.
Professor Saxena will join Cornell graduate student Hema Koppula as they present their research at the International Conference of Machine Learning, June 18-21 in Atlanta, and the Robotics: Science and Systems conference June 24-28 in Berlin, Germany.