How Do You Make a Robot Smarter? Program It to Know What It Doesn’t Know

Princeton University and Google researchers developed a technique using large language models (LLMs) to teach robots to realize when they do not know something and to request further instructions. The system involves setting an uncertainty threshold that will prompt the robot to ask for assistance based on the degree of success sought by the user. For instance, the researchers asked a robot to «place the bowl in the microwave,» leaving it to choose between metal and plastic bowls. Four actions, each assigned a probability, were generated by the robot’s LLM-based planner, and the robot asked which bowl to place in the microwave. Said Princeton’s Anirudha Majumdar, «Using the technique of conformal prediction, which quantifies the language model’s uncertainty in a more rigorous way than prior methods, allows us to get to a higher level of success.»

More info here: Princeton University School of Engineering and Applied Science