Inside ‘Project Rodeo,’ the Tesla effort pushing the limits of self-driving technology

None of the test drivers who spoke with BI said they had been involved in a crash.

Five who worked for the company in 2024 said they narrowly avoided collisions, including almost hitting a group of pedestrians. One former critical-intervention driver in Texas told BI that they sometimes ventured into their city’s bar district late at night to see how Tesla’s FSD software reacted to drunk patrons spilling out after last call. The former driver in San Francisco recalled riding around Stanford University during training, testing how close FSD would allow the vehicle to get to people at crosswalks before they had to take over. And a third critical-intervention driver said they allowed the car to speed through yellow lights and drive 35 mph under the speed limit on an expressway to avoid disengaging the system.

A former Autopilot engineer told BI that while testing is done on open roads, Tesla runs hundreds of simulations and sometimes tests difficult scenarios on a closed course before rolling out new software to test drivers’ vehicles.

Tesla did not respond to a detailed list of questions about Project Rodeo and its self-driving technology.

The test drivers’ experiences highlight the balancing act Tesla and other automakers face as they prepare their self-driving software for widespread consumer use.

Experts say public testing is crucial and can help identify safety issues before the technology hits the market. Missy Cummings, a former safety advisor for the National Highway Traffic Safety Administration, said that while practices vary, she believes many autonomous-vehicle companies likely employ tactics similar to Tesla’s.

“In theory, these drivers have gone through training, and eventually these cars do need to be able to operate in the public domain,” Cummings said. She added that clearly marked vehicles could help the public better identify test drivers.

Safety experts say fragmented and limited autonomous-vehicle regulations, coupled with self-reporting by automakers, create a complex environment where companies balance public safety with getting their products ready for commercial use.

“There are very few rules around autonomous testing and a lot of dependency on self-reporting,” said Mark Rosekind, a former NHTSA administrator and chief safety innovation officer for Zoox, an Amazon-owned autonomous-taxi firm. “If companies aren’t reporting, it’s hard to know what’s going on.”

In the past decade, authorities have investigated several automakers, including Tesla, Waymo, and Cruise, over crashes involving self-driving or driver-assist software.

Two years later, test drivers were asked to train the system near pedestrians, test drivers said. Five recalled a bug with FSD that made vehicles brake too early at crosswalks. To improve its performance, they were instructed to interact with pedestrians as frequently as possible.

Sometimes, the drivers said, the software would slam on the brakes when no one was at the crosswalk; other times, it wouldn’t stop at all. According to Tesla employees and internal documentation, FSD’s performance depended heavily on its software version, and the versions appeared to operate at different levels of caution.

The former San Francisco driver said that as they drove around Stanford University, their trainer, another test operator with more experience on the team, chastised them for braking too early. They recalled that at one point they came within three feet of hitting a bicyclist at a roundabout.

“I vividly remember this guy jumping off his bike. He was terrified,” the driver told BI. “The car lunged at him, and all I could do was stomp on the brakes.” They said the trainer was pleased by the incident. “He told me, ‘That was perfect.’ That was exactly what they wanted me to do.”

The driver added that “it felt like the goal was almost to simulate a hit-or-miss accident and then prevent it at the last second.”

The former Autopilot engineer said it was better for training to see whether the software could correct itself. They also said that not intervening when the car acted abnormally — including veering into another lane or doing something that confuses another driver — was important for training. Human motorists don’t always drive rationally, they explained, and the software needs to know how to respond. It’s also easier to parse the data if there are fewer driver interventions, they said.

“At the critical juncture where it’s about to make the key decision,” they said, it’s helpful to see whether the software makes the right or wrong call. “We want the data to know what led the car to that decision,” the engineer said. “If you keep intervening too early, we don’t really get to the exact moment where we’re like, OK, we understand what happened.”

A ‘Wild West’ with little regulation

Tesla is one of many automakers attempting to make autonomous vehicles a reality. Waymo, backed by Alphabet, launched the first driverless taxi service in Phoenix in 2020.

“In many ways, it’s like the Wild West out there,” said Cummings, the former NHTSA safety advisor. “There is very little regulation around training or informing the public about testing.”

The stakes of data collection on public roads are high. In 2018, a self-driving Uber with a person behind the wheel struck and killed a pedestrian in Arizona. Cruise paused testing after one of its driverless vehicles hit a pedestrian in October 2023. Another vehicle had already hit the pedestrian when the Cruise car struck her, dragging her 20 feet before stopping. It resumed testing with safety drivers in some cities in May.

Two former Waymo employees said that they had a team similar to Tesla’s critical-intervention team but that Waymo’s version of critical-intervention testing was limited to closed tracks with dummies. Two former Cruise employees said that they had mapping teams and teams that tested on closed courses and public roads but that, unlike at Tesla, those teams were instructed to take over as soon as the software went off track, and they typically tested with at least two people in the car.

A Waymo spokesperson said the company’s safety framework included rigorous testing in controlled environments and on public roads. A Cruise spokesperson said the company’s vehicles were designed as fully autonomous systems and were therefore fundamentally different from Tesla’s driver-assistance technologies.

Philip Koopman, an autonomous-driving expert at Carnegie Mellon University, said that Tesla’s critical-intervention approach, as described to him by BI, was “irresponsible” and that the company should be playing out all “critical scenarios” on a closed course.

“By allowing the software to continue misbehaving to the point a test driver needs to avoid a crash, Tesla would be imposing a risk on other road users who have not agreed to serve as test subjects,” Koopman said.

Alex Roy, a general partner at NIVC and a former director of operations at the autonomous-driving startup Argo AI, said companies should be correcting the software as soon as it strays from the course, particularly on public roads.

“You should play those mistakes out in a simulation, not on an open road,” Roy said. “Real-world testing is necessary, but real-world mistakes are not.”

“If you have a parent that’s holding the bike the entire time, it never gets to learn.”
A former Tesla Autopilot engineer

The former Tesla engineer said they doubted that computer simulations are sophisticated enough to replicate the data generated by real-world driving. The former engineer said that, to help the software improve, it was best for test drivers to avoid intervening whenever possible.

“If you have a parent that’s holding the bike the entire time, it never gets to learn,” the engineer said.

Test drivers on Project Rodeo felt this keenly.

“You’re pretty much running on adrenaline the entire eight-hour shift,” one former test driver in the Southwest said. “There’s this feeling that you’re on the edge of something going seriously wrong.”

Do you work for Tesla or have a tip? Reach out to the reporter via a non-work email and device at gkay@businessinsider.com or 248-894-6012.