Click the links that follow for more news items about awards won by Brown CS alums
In the current day and age, self-driving cars are perhaps one of the most promising technologies on the horizon. With the potential to dramatically improve traffic planning, fuel efficiency, and productivity, it’s easy to understand the excitement surrounding this innovation. One of the largest challenges facing the field, however, is safety. Brown CS alums Jacob Beck ‘18 and Zoe Papakipos ‘18 have eagerly taken up this challenge. Currently working at Microsoft Research and Facebook AI Research respectively, the duo’s research on the training of autonomous driving software has recently been recognized in New Scientist, one of the leading publications recognizing those at the cutting edge of technological progress.
What exactly does the research entail? “Autonomous vehicle research is about making computers drive cars,” remarks Jake, “and it’s important because computers have a fast reaction time, infinite attention, and are consistent. The focus of our work is to do this in a scalable way by teaching the computers from real world examples.” Currently, one common way to teach autonomous vehicles to drive is to train the AI off hours of footage of human driving that is deemed to depict perfect behavior. The problem with this approach, however, is that it restricts the software to only one side of the spectrum. It learns all about good driving, but it doesn’t get exposure to poor driving and the dangerous states it brings about. The car would have no idea what to do in a “bad state”, hindering its response in the real world.
“One alternative to this approach is a hybrid version of reinforcement learning – having a human driver try actions, and telling the computer how good or bad each action was,” says Zoe. The team recorded a human driving well, but also showed examples of swerving and erratic driving, and a backseat driver labelled the driving as positive or negative. The software then used this information to drive the car itself, and performed over 1.5 times better than the behavioral cloning method with perfect driving behavior. The research clearly showed a marked improvement over prior approaches, but this new approach also came with its own set of challenges.
“It was really difficult giving good feedback to the car,” explains Jake, “it’s really hard to know the exact angle for the steering wheel without turning the car yourself.” Eventually, the pair overcame this problem by having a human labeller turn the wheel to show approximately how much the car should change its action. For example, if the car should be turning a little bit more to the left, the human labeller turns the wheel slightly to the left. If the car is turning sharply left off the road, the labeller turns the wheel considerably further to the right.
Having made such an impact so early on their careers, what do they think made it possible for them to achieve so much so quickly? “It all really began in Professor Michael Littman’s autonomous driving lab at Brown,” explains Zoe, “and we were really interested in the applications of machine learning.” “It’s pretty amazing how willing to help professors are at Brown,” adds Jake, “and my work in the lab really prepared me well.” Although the pair have made much progress in the field, there is clearly much work to be done. “It feels really great, being recognized for a project that we were both so interested in,” laughs Zoe, “but we definitely need to put more work on it to be conference ready. The core ideas are there.” There’s no doubt that the duo’s research may prove to be vitally important to this exciting field, and the possibilities are endless.
For more information, click the link that follows to contact Brown CS Communications Outreach Specialist Jesse Polhemus.