This is the second in a series of four posts. The first post can be found here.
I have been working on autonomous vehicles and driver assistance systems for 23 years. I started to collect background material during my Ph.D. work from 1998 to 2001. Ten years ago, I started lecturing ME302B at Stanford University and collected even more material that I then compiled into a presentation. I was often asked to write this down instead of just talking about it.
Comments, additions, input for missing pieces, and suggestions are welcome. Contact firstname.lastname@example.org. Thanks
Research on self-driving or robotic vehicles started already in the late 1950s. Jameson Wetmore provides a detailed summary of the History and Motivations Behind 60 Years of Automated Highway Systems in America here and here. GM and the Radio Corporation of America (RCA) jointly began to conduct tests of automated highway prototypes with radio control for velocity and steering in 1958. As reported in 1960, magnets embedded in the vehicles tracked a steel cable buried in the road, while control towers managed overall traffic flow. The start of the new revolution seemed imminent, so much so that the New York Times predicted in 1960 that “full use [is] seen 15 years away”. Image: Radio Corporation of America (RCA).
Originally built around 1960, the Stanford Cart was designed to study the feasibility of controlling a lunar explorer from Earth. The cart was further developed by a series of graduate students over the next two decades, which culminated with Hans Moravec, who outfitted the cart with stereo-vision, enabling it to autonomously navigate a chair-filled room —in only five hours—in 1979. Image: Stanford University.
Around the same time, Shakey was developed by Nils Nilsson at Stanford Research Institute's Artificial Intelligence Center. It was the first mobile robot with the ability to sense its environment, via a TV camera "eye" and whisker-like feelers. Shakey also planned a course of action to complete a task. This project was succeeded by Carnegie Mellon University’s (CMU) Rover.
The first automated street vehicle with environment perception was developed at the Mechanical Engineering Lab at Tsukuba University in 1977. The group continued to develop automated vehicles for 20 years and in this 1994 paper reported a vehicle built by Toyota, that was able to follow lane markings for up to 50 m at speeds of up to 50 km/h with a single camera. Image: Tsubuka University.
Developed in the 1980s, the DARPA Autonomous Land Vehicle (ALV) was a 12 feet tall, eight-wheeled robot with multiple sensors, tasked to go from point A to point B without human intervention in the hills outside of Denver in 1985.
In the 1980s, Prof. Ernst Dickmanns and his research group at Bundeswehr University Munich built the world's first actual robotic street vehicles, using computer vision and probabilistic algorithms to drive 20 km on an empty highway with velocities of up to 96 km/h. His research continued in the European publicly-funded EUREKA-project Prometheus (PROgraMme for a European Traffic of Highest Efficiency and Unprecedented Safety), which ran from 1987 to 1994. It included many European participants and received a significant amount of public funding. Image Ernst D. Dickmanns.
Bosch showed in 1993 navigation systems and camera-based lane detection developed within Prometheus and found that “a meaningful combination of a global digital map and local sensor information is the key to targeted autonomous driving.”
Prometheus' final demo took place on the Paris Autoroute 1, where Prof. Dickmanns’ VAmP vehicle drove more than 1000 km at speeds of up to 130 km/h, automatically passing slower cars in the left lane and tracking up to twelve other cars simultaneously. In 1995, the team drove 1758 km from Munich to Denmark and back, both longitudinal and lateral guidance was done by computer vision with the velocities up to 175 km/h, while the longest stretch driven autonomously was 158 km. Prof. Dickmanns provides his account on the technology development in the field from the 1950s to present at a conference in 2011.
Also in 1995, the CMU research vehicle Navlab 5 drove from Pittsburgh, PA, to San Diego, CA, with automated steering but manual longitudinal control in the “No Hands Across America” drive. This 1998 History Channel video summarizes the CMU Navlab activities in the 1990s. Image: CMU.
The ARGO Project was started in 1996 by Alberto Broggi at the University of Parma, Italy. The MilleMiglia in Automatico tour demonstrated the state of a project driving 2000 km in automated mode in 1998. The team published a summary in this video ten years later.
The U.S. Department of Transportation’s National Automated Highway System Research Program (NAHS) culminated in 1997 in a technology demonstration of automated cars, buses, and trucks on I-15 in San Diego, California. Lateral lane keeping was achieved using computer vision and magnets embedded in the road. Longitudinal distance keeping was realized using radar or lidar sensors. Image: California Partners for Advanced Transportation Technology.
The German research project "Autonomes Fahren" included Volkswagen, Bosch, the University of Braunschweig, and others, and ran from 1997 to 2000. It aimed at automating endurance testing of production vehicles at the Volkswagen's proving grounds. The vehicles were equipped with several lidars, cameras, radar, GPS, a digital map of the course.
To drive without a test driver on a closed test course in mixed traffic with other vehicles, the vehicles were equipped with electromechanical driving robots to actuate the three pedals, the gear shifter, and the steering wheel.
Images: Jan Becker.
The DARPA Grand Challenge was a series of competitions for autonomous vehicles, funded by the Defense Advanced Research Projects Agency (DARPA). The 2001 US National Defense Authorization Act stated that "it shall be the goal of the Armed Forces to achieve the fielding of unmanned remotely controlled technology such that... by 2015, one-third of the operational ground combat vehicles of the Armed Forces are unmanned." Given the aggressive timeline in the directive, DARPA determined that organizing a prize authority event would be the quickest and most cost-effective approach to stimulate innovation and expand the research community in autonomous ground vehicle technologies. The first event took place in 2004. Only 15 teams qualified for the final competition. The unmanned self-driving vehicles attempted the route of a 142-mile course in the Mojave Desert, Nevada, through dirt roads, flats, and mountain passes. Image: DARPA.
Teams were only given a set of GPS waypoints 24 hours before the event. The CMU vehicle succeeded furthest, just completing 7.32 miles or less than 5% of the full distance. Nevertheless, as Wired pointed out correctly, the first of the three Grand Challenges created the self-driving developer community as we know it today.
DARPA repeated the event at the 2005 DARPA Grand Challenge. Five teams completed the 132-mile course, and all but one of the 23 finalists in the 2005 race surpassed the 7.32-mile distance completed by the best vehicle in the 2004 race.
The robot car Stanley, developed by Sebastian Thrun and his team at Stanford University, arrived at the finish line first, completing the course in 6 hours and 54 minutes. Sebastian later provided his account on the work at Stanford on Stanley in this article and in this talk. The two Carnegie Mellon University vehicles Sandstorm and Highlander came in just a few minutes after Stanley. Twelve years later, the team coincidentally found that a filter that goes in between the engine control module and the fuel injectors, one of only two electronic parts in the engine on a 1986 Hummer, was broken and caused the engine to lose power.
For the 2007 DARPA Urban Challenge, DARPA moved the competition to an urban area course at the site of the now-closed George Air Force Base in Victorville, California. Teams had to complete an urban course, negotiating four-way intersections, blocked roads or parking lots, in mixed traffic with robotic as well as human-driven vehicles. Over 90 teams applied to the Urban Challenge, 53 teams received site visits, 36 were invited to the final event, and just eleven made it into the final race. DARPA felt that since there would be real humans near the robots in the traffic cars as well as DARPA officials scoring robot performance. The event saw the first autonomous vehicle traffic jam as well as the first—though minor—collision of two self-driving vehicles. Carnegie Mellon’s vehicle Boss won the 60-mile race, followed by Stanford’s Junior.
Left: Stanford's Junior meets Virginia Tech's Victor Tango at the 2007 DARPA Urban Challenge. Right: CMU's Boss crossing the finish line. Images: DARPA.
A few more—mostly academic—activities took place after the Urban Challenge, among them the Stadtpilot project at the TU Braunschweig addressing urban driving including lane changes, parking, and merging into moving traffic; the Vislab Intercontinental Autonomous Challenge in 2010 where a convoy with four vehicles without a driver, mostly following a lead vehicle, over the 13,000 km from Parma, Italy to Shanghai, China; Stanford drove up the Pikes Peak without a driver and built a self-driving race-car in 2012; the Grand Cooperative Driving Challenge was a competition in Europe in 2011 and 2016 which vehicles had to follow a platoon leader, react to oscillations of the lead vehicle, collaboratively start at traffic lights, and merge platoons.
The Urban Challenge though, marked the transition from academic research to industrial development—which we will cover in the next part of this series. The mysterious 20-year hurdle was finally overcome.