Ken Goldberg: 4 lessons from robots about being human

I know this is going to sound strange, but I think robots can inspire us to be better humans. See, I grew up in Bethlehem, Pennsylvania, the home of Bethlehem Steel. My father was an engineer, and when I was growing up, he would teach me how things worked. We would build projects together, like model rockets and slot cars. Here's the go-kart that we built together. That's me behind the wheel, with my sister and my best friend at the time, and one day, he came home, when I was about 10 years old, and at the dinner table, he announced that for our next project, we were going to build a robot.

A robot. Now, I was thrilled about this, because at school, there was a bully named Kevin, and he was picking on me because I was the only Jewish kid in class. So I couldn't wait to get started to work on this so I could introduce Kevin to my robot. (Laughter) (Robot noises)

But that wasn't the kind of robot my dad had in mind. See, he owned a chromium plating company, and they had to move heavy steel parts between tanks of chemicals, and so he needed an industrial robot like this that could basically do the heavy lifting.

But my dad didn't get the kind of robot he wanted, either. He and I worked on it for several years, but it was the 1970s, and the technology that was available to amateurs just wasn't there yet. So Dad continued to do this kind of work by hand, and a few years later, he was diagnosed with cancer.

You see, what the robot we were trying to build was telling him was not about doing the heavy lifting. It was a warning about his exposure to the toxic chemicals. He didn't recognize that at the time, and he contracted leukemia, and he died at the age of 45. I was devastated by this, and I never forgot the robot that he and I tried to build. When I was in college, I decided to study engineering, like him. And I went to Carnegie Mellon, and I earned my PhD in robotics. I've been studying robots ever since.

So what I'd like to tell you about are four robot projects and how they've inspired me to be a better human. By 1993, I was a young professor at USC, and I was just building up my own robotics lab, and this was the year that the World Wide Web came out. And I remember my students were the ones who told me about it, and we would -- we were just amazed. We started playing with this, and that afternoon, we realized that we could use this new, universal interface to allow anyone in the world to operate the robot in our lab.

So, rather than have it fight or do industrial work, we decided to build a planter, put the robot into the center of it, and we called it the Telegarden. And we had put a camera in the gripper of the hand of the robot, and we wrote some special scripts and software so that anyone in the world could come in and by clicking on the screen they could move the robot around and visit the garden. But we also allowed, set up some other software that lets you participate and help us water the garden remotely, and if you water it a few times, we'd give you your own seed to plant.

Now, this was a project, an engineering project, and we published some papers on the design, the system design of it, but we also thought of it as an art installation. It was invited, after the first year, by the Ars Electronica Museum in Austria to have it installed in their lobby, and I'm happy to say it remained online there, 24 hours a day, for almost nine years. That robot was operated by more people than any other robot in history.

Now, one day, I got a call out of the blue from a student, who asked a very simple but profound question. He said, "Is the robot real?" Now, everyone else had assumed it was, and we knew it was because we were working with it. But I knew what he meant, because it would be possible to take a bunch of pictures of flowers in a garden and then, basically, index them in a computer system such that it would appear that there was a real robot when there wasn't. And the more I thought about it, I couldn't think of a good answer for how he could tell the difference.

This was right about the time that I was offered a position here at Berkeley, and when I got here, I looked up Hubert Dreyfus, who's a world-renowned professor of philosophy, and I talked with him about this, and he said, "This is one of the oldest and most central problems in philosophy. It goes back to the Skeptics, and up through Descartes. It's the issue of epistemology, the study of how do we know that something is true."

So he and I started working together, and we coined a new term: telepistemology, the study of knowledge at a distance. We invited leading artists, engineers, and philosophers to write essays about this, and the results, the results are collected in this book from MIT Press. So thanks to this student who questioned what everyone else had assumed to be true, this project taught me an important lesson about life, which is to always question assumptions.

Now, the second project I'll tell you about grew out of the Telegarden. As it was operating, my students and I were very interested in how people were interacting with each other and what they were doing with the garden. So we started thinking, what if the robot could leave the garden and go out into some other interesting environment? Like, for example, what if it could go to a dinner party at the White House? (Laughter) So, because we were interested more in the system design and the user interface than in the hardware, we decided that, rather than have a robot replace the human to go to the party, we'd have a human replace the robot. We called it the Tele-Actor.

We got a human, someone who's very outgoing and gregarious, and she was outfitted with a helmet with various equipment, cameras and microphones, and then a backpack with wireless Internet connection, and the idea was that she could go into a remote and interesting environment, and then over the Internet, people could experience what she was experiencing, so they could see what she was seeing, but then, more importantly, they could participate by interacting with each other and coming up with ideas about what she should do next and where she should go, and then conveying those to the Tele-Actor. So we got a chance to take the Tele-Actor to the Webby Awards in San Francisco, and that year, Sam Donaldson was the host. Just before the curtain went up, I had about 30 seconds to explain to Mr. Donaldson what we were gonna do, and I said, "The Tele-Actor is going to be joining you on stage, and this is a new experimental project, and people are watching her on their screens, and she's got -- there's cameras involved and there's microphones and she's got an earbud in her ear, and people over the network are giving her advice about what to do next." And he said, "Wait a second, that's what I do." (Laughter)

So he loved the concept, and when the Tele-Actor walked onstage, she walked right up to him, and she gave him a big kiss right on the lips. (Laughter) We were totally surprised. We had no idea that would happen. And he was great. He just gave her a big hug in return, and it worked out great. But that night, as we were packing up, I asked the Tele-Actor, how did the Tele-Directors decide that they would give a kiss to Sam Donaldson? And she said they hadn't. She said, when she was just about to walk on stage, the Tele-Directors were still trying to agree on what to do, and so she just walked on stage and did what felt most natural. (Laughter) So, the success of the Tele-Actor that night was due to the fact that she was a wonderful actor. She knew when to trust her instincts, and so that project taught me another lesson about life, which is that, when in doubt, improvise. (Laughter)

Now, the third project grew out of my experience when my father was in the hospital. He was undergoing a treatment, chemotherapy treatments, and there's a related treatment called brachytherapy, where tiny, radioactive seeds are placed into the body to treat cancerous tumors. And the way it's done, as you can see here, is that surgeons insert needles into the body to deliver the seeds, and all this, all these needles are inserted in parallel, so it's very common that some of the needles penetrate sensitive organs, and as a result, the needles damage these organs, cause damage which leads to trauma and side effects. So my students and I wondered, what if we could modify the system so that the needles could come in at different angles?

So we simulated this, and we developed some optimization algorithms and we simulated this, and we were able to show that we are able to avoid the delicate organs and yet still achieve the coverage of the tumors with the radiation.

So now, we're working with doctors at UCSF and engineers at Johns Hopkins and we're building a robot that has a number of, it's a specialized design with different joints that can allow the needles to come in at an infinite variety of angles, and as you can see here, they can avoid delicate organs and still reach the targets they're aiming for. So, by questioning this assumption that all the needles have to be parallel, this project also taught me an important lesson: When in doubt -- When your path is blocked, pivot.

And the last project also has to do with medical robotics. And this is something that's grown out of a system called the da Vinci surgical robot, and this is a commercially available device. It's being used in over 2,000 hospitals around the world, and the idea is it allows the surgeon to operate comfortably in his own coordinate frame, but many of the subtasks in surgery are very routine and tedious, like suturing, and currently, all of these are performed under the specific and immediate control of the surgeon, so the surgeon becomes fatigued over time. And we've been wondering, what if we could program the robot to perform some of these subtasks, and thereby free the surgeons to focus on the more complicated parts of the surgery, and also cut down on the time that the surgery would take if we could get the robot to do them a little bit faster?

Now, it's hard to program a robot to do delicate things like this, but it turns out my colleague, Pieter Abbeel, who's here at Berkeley, has develeloped a new set of techniques for teaching robots from example. So he's gotten robots to fly helicopters, do incredibly interesting, beautiful acrobatics, by watching human experts fly them. So we got one of these robots. We started working with Pieter and his students, and we asked a surgeon to perform a task, and what we do is we, with the robot, so what we're doing is asking the robot, the surgeon to perform the task, and we record the motions of the robot.

So here's an example. I'll use a figure eight, tracing out a figure eight as an example. So here's what it looks like when the robot, this is what the robot's path looks like, those three examples. Now, those are much better than what a novice like I could do, but they're still jerky and imprecise.

So we record all these examples, the data, and then we go through a sequence of steps. First, we used a technique called dynamic time warping from speech recognition, and this allows us to temporally align all of the examples, and then we apply Kalman filtering, a technique from control theory, that allows us to statistically analyze all the noise and extract the desired trajectory that underlies them. Now, so what we're doing is that we take those human demonstrations, they're all noisy and imperfect, and we extract from them an inferred task trajectory and control sequence for the robot. We then execute that on the robot, we observe what happens, then we adjust the controls using a sequence of techniques called iterative learning. Then what we do is, we increase the velocity a little bit. We observe the results, adjust the controls again, and observe what happens. And we go through this several rounds. And here's the result. That's the inferred task trajectory, and here's the robot moving at the speed of the human. Here's four times the speed of the human. Here's seven times. And here's the robot operating at 10 times the speed of the human. So we're able to get a robot to perform a delicate task, like a surgical subtask, at 10 times the speed of a human. So this project also, because of its involved practicing and learning, doing something over and over again, this project also has a lesson, which is, if you want to do something well, there's no substitute for practice, practice, practice.

So these are four of the lessons that I've learned from robots over the years, and robotics, the field of robotics has gotten much better over time. Nowadays, high school students can build robots like the industrial robot my dad and I tried to build. And now, I have a daughter, named Odessa. She's eight years old, and she likes robots, too. Maybe it runs in the family. (Laughter) I wish she could meet my dad. And now I get to teach her how things work, and we get to build projects together, and I wonder what kind of lessons that she'll learn from them.

Robots are the most human of our machines. They can't solve all of the world's problems, but I think they have something important to teach us. I invite all of you to think about the innovations that you're interested in, the machines that you wish for, and think about what they might be telling you, because I have a hunch that many of our technological innovations, the devices we dream about, can inspire us to be better humans.

Thank you. (Applause)