Learning from Interactive Dialog

Date: August 2009

JJ Lee and AL Thomaz (2009).
Goal-Oriented Actions in Task Learning with Sequential Constraints. Unpublished

Rather than preprogramming a large set of skills, robots will need to be able to learn from everyday people. People can teach robots how to fold a towel or clean dishes by showing them demonstrations and giving them instructional feedback.

I developed a learning mechanism that allows everyday people to teach a humanoid robot through interactive dialog. The implementation is capable of learning a simple task representation that tries to generalize a task by learning the underlying goal. Through multiple demonstrations provided by a human teacher, Simon, a humanoid robot, learns how to assembly a block house. It learns that the sequence of the objects’ shape is relevant to completing the task but not its color. Simon learns this underlying goal and can apply this knowledge to many different configurations of the task.