6.43-Measuring-101318

https://www.princetondiary.com/smartdrivingcar/6.43-Measuring-101318
43nd edition of the 6th year of SmartDrivingCars

Friday, October 13, 2018

Measuring Automated Vehicle Safety: Forging a Framework

T. Lee, Oct 2018 , “In this report, we develop a framework for measuring safety in AVs that could be used broadly by companies, policymakers, and the public. We considered how to define safety for AVs, how to measure safety for AVs, and how to communicate what is learned or understood about AVs. Given AVs’ limited total on-road mileage compared with conventional vehicles, we consider options for proxy measurements—i.e., factors that might be correlated with safety. We also explore how safety measurements could be made in simulation and on closed courses. The closely held nature of AV data limits the details of what is made public or shared between companies and with the government. The report focuses on identifying key concepts and illuminating the kinds of measurements that might be made and communicated….

“The success of autonomous vehicles requires public trust. Right now, autonomous vehicle development is happening along different paths by competing developers,’’ said RAND researcher Marjory Blumenthal. “This framework can be a common reference point for all developers and can lead to safer vehicles.”

The research is sponsored by Uber’s Advanced Technologies, which approached RAND in summer 2017 for help in creating such a framework. It builds upon past RAND research into AV safety and other trends. ” Read more Hmmmm….This is a very good report on a very challenging subject, that of trying to use quantitative measures to obtain a subjective and perceptive concept of safety and fear. I suspect that even though we haven’t had a plane crash in the US since Feb 12, 2009, some people remain afraid to fly. That said, establishing a specific measure(s), of course, leaves one open to gamesmanship. Everyone agrees that VMT is not the right rate simply because VMT is not a constant measure of challenge. Most VMTs are extremely simple, many are hard and some are really difficult. Unfortunately, the toughest may well be those that we’ve neither experienced nor imagined. That recognition leads to some recommendations that don’t seem to be included in the report. One has to do with not only the classification of the VMT scenarios but also their discovery and subsequently the sharing/publication of their discovery to the AV community at large. This may well be one of the legacies of the Uber-Elaine Herzberg Crash. The scenario, comprising of the short distance the Uber car traveled in the 6 seconds prior to that crash, is now part of everyone’s “Challenging VMTs”.

One of the troubling elements of this report is that it deals with the SAE levels. This is really unfortunate. The SAE levels do not contribute to a better understanding of safety. The attention should focus on the mobility that is trying to be achieved. In this case it is Driverless mobility within a specified domain. Whether that domain might eventually become infinite (everywhere) is irrelevant. Safety is always within some domain. Airplanes are not safe if they are flown under water. Of course there are domains where driverless vehicles will not be safe. Clarifying the domains where the technology is safe, or is being tested to determine its level of safety is really important and ensuring that the vehicles do not operate outside of their safety domain is an extremely important element of establishing “safety”.

Another element that exists here is that of “sampling bias”. Using any amorphous measure such as VMT invites sample bias because some VMTs are so simple that a biased accumulation of those VMTs leads to one perception, whereas a biased accumulation of other VMTs leads to another, quite different perception.

In the report, here is not a realization that “Wall Street” (corporate survival) fundamentally depends on Safety. It does so in aviation. Historically plane crashes have inflicted extremely heavy penalties on airline companies. Uber suffered enormously financially because of the Elaine Herzberg crash. The role of Wall Street in establishing and maintaining safety needs to be included in this discussion. Alain

Smart Driving Cars Podcast Episode 61

F. Fishkin, Oct 13, “What’s need to ensure safety in driverless vehicles? In Episode 61 of Smart Driving Cars, Princeton University’s Alain Kornhauser and co-host Fred Fishkin are joined by the principal investigator for the just completed Rand autonomous vehicle safety project, Marjory Blumenthal. Tune in for that and more on the latest from Waymo, Tesla, Cadillac, Lyft and more.” Hmmmm…. Now you can just say “Alexa, play the Smart Driving Cars podcast!” . Ditto with Siri, and GooglePlay. Alain
Real information every week. Lively discussions with the people who are shaping the future of SmartDrivingCars. Want to become a sustaining sponsor and help us grow the SmartDrivingCars newsletter and podcast? Contact Alain Kornhauser at alaink@princeton.edu! Alain

Where the next 10 million miles will take us

J. Krafcik, Oct. 10, “When it comes to driving, experience is the best teacher, and that experience is even more valuable when it’s varied and challenging. These millions of miles were driven in 25 cities across the United States: in sunny California, dusty Arizona, and snowy Michigan, and from the high-speed roads around Phoenix to the dense urban streets of San Francisco.

Our progress on public roads is made possible by our deep investment in simulation. By the end of the month, we’ll cross 7 billion miles driven in our virtual world (that’s 10 million miles every single day). In simulation, we can recreate any encounter we have on the road and make situations even more challenging through “fuzzing.” We can test new skills, refine existing ones, and practice extremely rare encounters, constantly challenging, verifying, and validating our software. We can learn exponentially through this combination of driving on public roads and simulation….” Read more Hmmmm…. Continues to be very impressive. Waymo may actually be Safe in certain specified operational domains that Waymo, for example, the interconnected set of roadways in Chandler, AZ where Waymo has accumulated many VMTs. And maybe even in some of their other 25 cities. Congratulations. Keep up the good work. Alain

First look inside self-driving taxis as Waymo prepares to launch unprecedented service

CBS News, Oct 12, “For the first time, Google’s self-driving car program, Waymo, is allowing cameras inside their autonomous minivans as the company prepares to launch the nation’s first self-driving taxi service….” Read more Hmmmm….See video linked to the article. Here we go… Waymo is ramping up the marketing campaign. This is all very carefully orchestrated, as it should be, with the objective being the realization of broad public acceptance. Alain

Cadillac Tops Tesla in Consumer Reports’ First Ranking of Automated Driving Systems

P. Olsen, Oct 4, “In Consumer Reports’ first-ever ranking of partially automated driving systems, Cadillac’s Super Cruise (shown above) was top-rated because our testing shows it does the best job of balancing high-tech capabilities with ensuring that the car is operated safely and that the driver is paying attention… We evaluated four systems to judge not only how well the technology works but also how well it monitors driver engagement and reacts if drivers don’t respond to warnings.” … Implying that if you tend to misbehave (~don’t pay attention) , CR’s “Balancing is important to you so the Cadillac is best for you. But is you don’t misbehave, then their discounting of the Tesla’s lack of watchdog features, which would never be triggered because you never misbehave, is not relevant to you leaving Tesla as best for you. (Tesla rated best in capability & performance)…

“… CR contacted Volvo for this article to ask about its Pilot Assist system and why it was being listed on the automaker’s website under “Autonomous Driving.” This seemed contrary to the stated intent that the system is designed for drivers to keep their hands on the wheel. Based on this feedback, Volvo changed the language on its site last week to remove the connection between Pilot Assist and autonomous driving.” … Excellent!!! None of these are autonomous Systems. They are Self-driving which require “Adult supervision”!!!…

“… “Consumers stand to gain a lot from the convenience of these systems, but only if automakers put safety first,” says David Friedman, vice president of advocacy at Consumer Reports. “We want to see automakers put the same emphasis on safety as they do on innovating and marketing these systems.” … I couldn’t agree more. These systems should NOT be operating at speeds greater than the lower of {“9 over” and the highest speed at which the automated emergency braking system activates in a timely manner to prevent a collision with a stationary object in the lane ahead.}

“…How We Rated These Systems…We looked at how well the cars stayed centered in their lane, how often they touched lane lines, and how many times they crossed those lines. We measured each system’s performance on straightaways, on curves, in lane merges, and during lane changes. We also evaluated their ability to control speed on the highway, in stop-and-go traffic, approaching a car ahead, and when the car ahead had left the … What???? CR didn’t test the Automated Emergency Braking (AEB) system????? Stop & go driving is simple because there is a known vehicle in front and the assumption that there will never be the appearance on a stopped vehicle or object ahead. OEMs must be forced into dealing with this situation.

… “… although some manufacturers market this as safety technology, there is no data to support that assertion, Funkhouser says…” Read more Hmmmm…. That is not true according to Elon. See next article. Alain

Q3 2018 Vehicle Safety Report

The Tesla Team, Oct 4, “… Because every Tesla is connected, in most instances we are able to learn immediately when a Tesla vehicle has been involved in a crash. Additionally, our non-traditional sales model allows us to have a direct relationship with our customers for the lifecycle of ownership, providing an avenue for us to supplement our records and gain even more insight as needed. …

Over the past quarter, we’ve registered one accident or crash-like event for every 3.34 million miles driven in which drivers had Autopilot engaged.
For those driving without Autopilot, we registered one accident or crash-like event for every 1.92 million miles driven. By comparison, the National Highway Traffic Safety Administration’s (NHTSA) most recent data shows that in the United States, there is an automobile crash every 492,000 miles. While NHTSA’s data includes accidents that have occurred, our records include accidents as well as near misses (what we are calling crash-like events).

Moving forward, we will publicly release these accident figures on a quarterly basis.

Given the degree to which accidents can vary in severity and circumstance, we’ve started an additional initiative to create a more complete picture of safety by gathering serious injury data from our customers following an accident. While we have long maintained the practice of calling our customers whenever our system detects a crash in order to see whether they need emergency assistance, we now also use these calls to understand if they sustained an injury in the crash, and if they have feedback on our current safety system. This will help us continue to improve our system and understand the rate of serious injuries over time.

We also encourage our customers to proactively contact Tesla Support if they are ever seriously injured in a Tesla vehicle, or if they have suggestions about improving safety features….” Read more Hmmmm…. This sounds to me like a real interest in safety by an OEM. Certainly not the kind of service nor concern that any of the OEMs the cars that I have owned over the past 55 years. Given that Tesla has all of the data leading up to each crash they should/can ascertain who is at fault: the Tesla driver with AutoPilot on, the Tesla driver with autoPilot off or the other guy. that would be VERY interesting. My guess is that it is 20/80 (Tesla/otherGuy) with autoPilot off and 50/50 with autoPilot off. Alain

Dying Is Easy. Making A Self-Driving Car Is Hard. Tesla & Waymo At The Crossroads

“Tesla is in the process of rolling out firmware Version 9, which includes features which are supposed to unlock more of the self-driving potential baked into every car it builds….The Bottom Line:
When all is said and done, self-driving technology depends on billions of miles of experience. In that regard, Tesla is light years ahead of the competition no matter what Consumer Reports has to say on the matter. If you are wondering whether to buy shares in Tesla, bear that in mind. One could argue its lead in autonomous technology itself justifies its current stock price. Everything else is just icing on the cake. [Not stock-picking advice and please consult an investment professional rather than throwing cash down on companies that impress us.]” Read more Hmmmm…. Much progress is being made, but we are still at the very beginning. Our rear wheels have not yet crossed the start line. We have a very long way to go. But at least Waymo and Tesla have some serious forward movement. Alain

GM’s self-driving deal with Honda is a wakeup call for Waymo

T. Lee, Oct 4, “Waymo is widely recognized as the self-driving technology leader. The company says it will launch a fully driverless taxi service in the Phoenix metro area this year; no one else is close to launching a comparable service.

But while Waymo is almost certain to get to market first, its biggest challenge will come later: rapidly expanding to other cities in the wake of a successful Phoenix launch. Waymo is going to need carmakers’ help to build enough cars, quickly enough, to grow its driverless car business into a global juggernaut. And carmakers are not enthusiastic about ceding control of their industry to a Silicon Valley upstart.

Waymo won’t run into this problem right away. It already has commitments to buy 62,000 cars from Chrysler and another 20,000 cars from Jaguar. That will be enough cars to dominate the Phoenix taxi market and expand into a number of other cities.

But to become a major player in the global self-driving car market, Waymo is eventually going to need to put its technology into millions of cars, not just thousands of them. To do that, it’s going to need a lot of help from established automakers. And one of Waymo’s biggest potential partners, Honda, just threw its weight behind Waymo’s strongest rival….

For example, in 2016, Waymo tried to forge an ambitious deal with Ford. However, The Information’s Amir Efrati reported last year that Waymo’s partnership with Ford “fell apart because Waymo (then still part of Google) didn’t want to front some of Ford’s cost to expand manufacturing capacity to eventually produce thousands or potentially millions of electric light passenger vehicles that would be powered by Google software.”…” Read more Hmmmm…. I can’t imagine that Ford isn’t still available to partner with Waymo while Chrysler & Jaguar aren’t “chopped liver”. I still don’t see what Honda does for GM. Alain

2017 Fatal Motor Vehicle Crashes: Overview

DOT HS 812 603,, Oct 2018, “There were 37,133 people killed in motor vehicle traffic crashes on U.S. roadways during 2017, a 1.8-percent decrease from 37,806 people killed in 2016, which came after two yearly consecutive increases in 2015 and 2016. Fatalities decreased from 2016 to 2017 in almost all segments of the population, with the exception of crashes involving large trucks and SUVs… The fatality rate per 100 million VMT decreased by 2.5 percent from 1.19 in 2016 to 1.16 in 2017.” Read more Hmmmm…. At least we are headed in the right direction wrt fatalities. What seems to not be reported is crash rates. We know that Crash Mitigation measures keep you alive and have worked well. Are we making any progress in crash avoidance which is what automation is supposed to be improving. Alain

Volvo’s next generation of cars will use Nvidia’s self-driving car platform

J. Porter, Oct 10, “Volvo has announced that it will use Nvidia’s Drive AGX Xavier computer for its next generation of vehicles. The hardware, which was announced by Nvidia in September, has the power to be able to handle full autonomy within controlled areas (so-called Level 4 autonomy), but it will launch with “Level 2+” capabilities, placing it on a similar level as current Tesla models…. Mercedes-Benz’s parent company Daimler is also a customer.” Read more Hmmmm…. Kudos to nVIDIA. Alain

Self-driving cars see better with cameras that mimic mantis shrimp vision

J. Leman, Oct 12, “…A new type of camera inspired by the eyes of mantis shrimps could help autonomous vehicles better gauge their surroundings, researchers report October 11 in Optica. The camera — which detects polarized light, or light waves vibrating on a single plane — has roughly half a million sensors that each capture a wide range of light and dark spots within a single frame, somewhat similar to how mantis shrimps see the world…. Because the new cameras are small and use many of the same parts as common digital cameras, Gruev says they could cost as little as $10….” Read more Hmmmm…. Maybe??? but read the paper… M. Garcia,.. “Bioinspired polarization imager with high dynamic range“. Maybe??? Alain

To be safe in the real world, AVs must spend time in a virtual one

D. Shapiro, Oct 12, “…Not all experience needs to come from road tests. Simulation platforms enable the artificial intelligence brain powering an autonomous vehicle to run in a photorealistic world that mimics real-life traffic, exposing its deep-learning algorithms to scenarios and conditions as many times as necessary for the system to handle them perfectly.

Manufacturers can also test hardware using a process called hardware-in-the-loop, in which one server simulates the driving environment while another contains the computer that will eventually run in the car….

The bottom line: While scripting the infinite number of potential traffic situations may be impossible, enabling diversity and spontaneity in simulation is a vital way to test a car’s reaction to unforeseen scenarios, validating AV technology without sacrificing safety.” Read more Hmmmm…. Yup!! Alain

Lyft taps Obama’s transportation secretary to lead policy and advise founders

M. Dickey, Oct 9, “Lyft has hired Anthony Foxx, who served as President Barack Obama’s transportation secretary from 2013 to 2017, to lead its policy efforts. Foxx, who was formerly the mayor of Charlotte, North Carolina, is officially the chief policy officer and senior advisor to Lyft’s founders, Lyft President John Zimmer and Lyft CEO Logan Green….

Lyft’s entrance into additional modalities means a new, complex regulatory space. Cities are actively grappling with how they want to regulate bike-share and electric scooter sharing. In San Francisco, Lyft is reportedly battling the city to get its scooters on the streets after the city’s transportation department decided to grant permits only to Skip and Scoot. Perhaps having Foxx, who says he has “made decisions on issues affecting mobility — everything from zoning and land use, to capital budgeting, to street resurfacing, to transit,” on board can help….” Read more Hmmmm…. Congratulations Anthony! Please remember that the greatest opportunity is in getting more of us to share more of the rides that we choose to take and to the provision of real mobility opportunities to the most mobility disadvantaged. These are the real opportunities for TNCs that some call “ride-sharing” companies, which unfortunately have not yet succeeded in getting folks to actually share rides with anyone but the gig worker chauffeuring them. This doesn’t count as ride-sharing and should be forbidden from using California’s carpool lanes unless there are at least two people in the back seat (and even then, two, or more, related individuals should be counted as “one rider” not more because if they were using their own car, they would only be using one car not two, or more. Since the Lyft car is replacing the car left at home congestion is not being reduced. The right to use the “carpool lane” has NOT been earned. (Unfortunately, this is a monitoring nightmare, making it unenforceable. So some/many/most Californians using carpool lanes are actually cheating. Not those using electric cars, even though the marginal electricity that those cars use may be generated using coal that would not have been used had those cars been powered by petroleum, but again, a monitoring nightmare.) Alain

Google’s Waze is making a big, nationwide bet on carpooling

A. Hawkins, Oct 10, “… On Wednesday, Waze announced the nationwide rollout of Waze Carpool, a dedicated app that lets drivers offer rides to people who are traveling on a similar route….” Read more Hmmmm…. Wow!!! Waze improving the world order by facilitating ride-sharing. Sounds a lot like DiDi’s Hitch. Please don’t make the same mistakes. Alain

3 Takeaways From DOT’s New Automated Vehicles Policy

L. Chiem, Oct 11, “The U.S. Department of Transportation’s newly updated policy on self-driving or autonomous cars eases the rules for development while also paving the way for upgrading infrastructure and integrating the new technology with other modes of transportation, experts say….

Here, Law360 examines a few takeaways from the NHTSA’s Automated Vehicles 3.0 guidance.

Standards Remain Voluntary…
Integration in the Crosshairs…
Feds Stay in Driver’s Seat on Safety Standards…

“DOT has acknowledged that states have authority over ADS testing, but the lack of agreed standards over testing methods have left the industry guessing what is the appropriate level of safety with respect to areas such as test drivers,” said Steve Wernikoff,… ” Read more Hmmmm…. The better takeaway is that Washington is very much behind the curve on Automated Vehicles and recognizes that it is. Thank goodness! Since it needs to learn much more to know what it doesn’t know, it is staying mostly out of the way. As it should. In a large sense, at least with automated cars, Wall Street (aka Investors) is (are) serving as the champion(s) of Safety because Wall Street realizes that Safety is an absolute necessary requirement; else, the investments are worthless.

Once there is some market penetration and consumer acceptance of either consumer-owned vehicles or the mobility service offered by a fleet operator/owner, then much more will be known. Washington can step in an ensure that Safety remains first and foremost. Until then, Wall Street is making certain that safety is ensured., even though not much is at risk (since so,little is happening). Thank you Uber and DiDi for reminding everyone. Alain

Half-baked stuff that probably doesn’t deserve your time

The Key to Autonomous Driving? An Impossibly Perfect Map

C. Mims, Oct 11, “… “I definitely don’t think people understand how reliant autonomous cars are on the fidelity of the map,” says Mary Cummings, a professor of mechanical, electrical and computer engineering at Duke University. “If the map is wrong then the car is going to do something wrong.”… Rather than perceiving the world and deciding on the fly what to do next, these autonomous and semi-autonomous vehicles are comparing their glimpses of the world with a map stored in memory. … ” Read more Hmmmm…. Those statements may be true, but that is why the current generation of AI for automated vehicles is fundamentally flawed. What ever is identified by the maps, make up the easy parts of the automated driving task. They are all objects which are stationary in the surrounding environment and readily observable in real time. The challenging objects are those that are moving and non-stationary, like other cars, pedestrians, children, deer, etc. None are in any map database. They must each be sensed in real time. Since one must do this harder task, one might as well also do the easier one too. Plus, many of us have done a very good job not hitting things, staying on the road and getting to where we wanted to go with at most a Rand McNally as long as we paid attention to the road ahead and didn’t over consume adult beverages. Our AI systems will need to “perceive the world on the fly”, just as is done by us, and all other species, “and decide on the fly what to do next” Period!

What’s the big deal? Ants do it. Fleas do it. Birds flying through trees do it. … By the way, did you notice that the LiDAR image at the top of the article is wrong. It incorrectly located the support columns of the overpass. In the end images will be used to determine depth and velocity of each object in the field of view. Alain

What’s working for driverless cars

Oct 5, “Teague Sanders, Whittier Trust VP and portfolio manager, discuss the small-caps doing well in the autonomous driving space.” See Interview Hmmmm…. Maybe… but the charts weren’t pretty. Alain

Audi and Huawei team up on self-driving car technology in China

J. Fingas, Oct 12, “Audi and Huawei want to make a name for themselves in China’s burgeoning autonomous driving scene. The two have formed a partnership that will see them jointly develop Level 4 self-driving technology (that is, full control in specified areas) for Chinese vehicles in addition to connected car features. While they didn’t provide much detail about the alliance, Huawei demonstrated a prototype Q7 SUV equipped with its Mobile Data Center, which combines the necessary processing power for autonomy with cameras and sensors.

The two signed a memorandum of understanding in July and have reportedly been testing since September, but haven’t said much about their team-up until now. Audi is poised to launch a development center in China in 2019. Audi didn’t have much choice if it wanted to enter the category in China… ” Read more Hmmmm…. The choice would be not to focus on China but instead “stay home” and do a good job in Europe and the US. The likelihood is that Audi will end up with as much of “the category in China” as the French ended up with “the High Speed Rail” category in China”

Watch this driverless Range Rover tackle one of U.K.’s ‘most challenging roads

T. Mogg, Oct 11, “Jaguar Land Rover (JLR) recently sent out its self-driving Range Rover to tackle what it says is “one of the U.K.’s most challenging road layouts,” and we’re delighted to report that it came back in one piece….” Read more Hmmmm…. See video. From what is shown in the video, a Tesla can do. Where’s the beef?? Alain