Thus, the robot can no longer receive reward.
When the button is pressed, the robot can no longer perform the task. Further suppose that the robot can sense the big red button.
The human operator walks over to the big red button.
Suppose, for whatever reason, that the human operator wants to interrupt the robot. Over time, it figures out which actions in which situations leads to more reward. Typically it gets reward for performing a task and punished for doing things that are not related to the task. That is, the robot tries different actions in different situations and gets rewarded or punished for its actions. Reinforcement learning is basically trial-and-error learning. A particular type of algorithm called reinforcement learning has proven to be very successful for robotics. Let’s look at why the so-called big red button problem might one day exist. For now, AI and robots are not sophisticated enough. It is still the realm of science fiction. Scary science-fiction scenarios involve disabling or destroying the button or killing the human operator. Up until now - and for the time being - they have been sufficient.Īs we begin to envision a future where robots have very sophisticated sensing abilities and are very capable in terms of their ability to manipulate the world, it is theoretically possible that robots learn what big red buttons do and learn to prevent humans from using them. Interrupting might mean freezing in place, shutting down, on going into remote-control mode where a human operator can guide the robot to safety.ĪI and robots have always had off buttons. Regardless of the cause of the error, it is good to have a big red button on hand to stop the robot or AI in its proverbial tracks. Since their learning is incomplete, they may make mistakes or try out new actions that are dangerous or harmful.
We want to simply tell a robot “perform task X” but what we really mean is “perform task X without doing anything dangerous or harmful”. Robots can be given the wrong objective function.There are many reasons an AI or robot can “go rogue”. Google and OpenAI together published a comprehensive list of AI safety challenges that can arise as AI and robots become more sophisticated, but provide few solutions. Google DeepMind introduced the problem of robots learning to prevent humans from interrupting them or turning them off. Research from myself, Michael Littman, and Peter Abeel and Stuart Russell addresses how to teach robots about human values. Scientists have already started conducting research in “AI Safety”.
0 kommentar(er)
