11.50 - Merry xmas
50th edition of the 11th year of SmartDrivingCars eLetter
Even though we've had a very challenging year, we much to be very thankful for. The glass is half-full. Merry Xmas! Enjoy!J
Waymo significantly outperforms comparable human benchmarks over 7+ million miles of rider-only driving
Waymo Team, Dec. 20, "Safety leads everything we do at Waymo. This year alone, Waymo has served over 700,000 ride-hailing trips with public riders and no human driver. We couldn't have hit that milestone without putting safety front and center, and we are working hard to improve the measurement, transparency, and performance of our fleet.
Our comprehensive research — across more than twenty safety papers that we've published over the years to enhance transparency — shows that the Waymo Driver performs safely across a range of evaluations. Building on that work, we've published two new papers today: one that compares the Waymo Driver's crash rates to human drivers' over our 7+ million rider-only miles from Phoenix, San Francisco, and Los Angeles; and another that develops clear human crash benchmarks to enable such comparisons.
Our new research found that Waymo Driver performance led to a significant reduction in the rates of police-reported and injury-causing crashes compared to human drivers in the cities where we operate.
"These reports represent a good-faith effort by Waymo to evaluate how the safety of its autonomous driving system compares with the safety of human driving. The results are encouraging and represent one step in our evolving understanding of autonomous driving safety," said David Zuby, chief research officer of The Insurance Institute for Highway Safety (IIHS) after reviewing the papers…." Read More
Hmmmm…. The quote from David Zuby is indeed substantive and reflects my interpretation of the two papers cited above, as well as previous papers and posts that Waymo has made on this topic.
Waymo deserves enormous kudos for their approach to Safety. I firmly agree with the first highlighted phrase (Since no one else has gotten to where they are, the statement is true by default. That's a recognition, not a dig.) as well as the highlighted transparency. We have at least one data point that suggests that even the slightest lack of "transparency" can easily negate the virtues of a focused safety mindset.
Safety is tough! It is a qualitative perception of perfection whose quantitative measures can only be achieved asymptotically, if at all. Consequently, any quantitative assessment is doomed to be a comparative to benchmarks rather than absolute, wherein the challenge. All benchmarks come with their own set of caveats and adjustments so the comparison ends up being "apples v apples" as opposed to "apples v oranges," which is never easy, and may not even end up as being as good as having the comparison not even being "the same fruit."
Having the comparison be on the same basis is particularly challenging in this safety situation, where the comparison is made between two different scenarios: comparing the safety of being "given a ride" by Waymo to the safety of "giving oneself a ride" by conventional automobiles.
At least one of the big challenges of having enough data to satisfy "data scientists" of "statistical significance" exists on the "given a ride" side. Also, these data are essentially pristine because Waymo archives its continuous oversight of the operation of its driverless cars and its commitment to transparency.
Unfortunately, such data quality does not exist on the "giving oneself a ride" side requiring significant adjustment and modification so as to enable a substantive side-by-side comparison of safety measures. I know from the mishaps that I've been personally involved in my lifetime, only a small percentage were ever recorded by any means other than human memory of those involved. Only one or at most two rose to the level of documentation in a police report and only a few more were reported to my insurance company. Certainly the time(s) that I ran over a fire hose or didn't pull over instantly for an oncoming emergency vehicle never went viral.
Anyway, it is very nice that Waymo has unveiled what has been obvious for some time: they have passed the proof-of-concept aspect of its "rider-only" process of giving rides by actually "giving rides safely" throughout the Operational Design Domains that it has carefully delineated.
One recommendation as to how Waymo could further enhance its safety credibility would be to compare its performance data with a "giving oneself a ride" data set that is much more comparable in quality to its "giving a ride" data set so as to forego all of the adjustments it needs to make to the existing public data set on the safety of today's systems by which we give ourselves rides.
It turns out that Tesla seems to have compiled data from its vehicles that I suspect rivals, in quality and depth, the data that Waymo has archived. If those two entities could collaborate and subsequently reveal their safety reality, it would do both of them and the public at large an enormously clearer understanding of not only how well each of their technology works, but also in the net contribution, plus or minus, that today's human driver plays in delivering safety to the mobility of rides whether they are given by a human with technology assistance, or given by technology with minimal monitoring and incidental assistance by a human. Such comparison could readily be conducted, because both Tesla and Waymo have archived vast amounts of substantive, machine-readable data on enabling a more straightforward assessment of the safety of "giving of oneself a ride" (Tesla) and the "giving of a ride" (Waymo).
Even without comparison with Tesla, Waymo has successfully proven that its "giving a ride" is achieved more safely than what we do collectively in "giving ourselves a ride." With this achievement in hand, Waymo should now focus on building upon that foundation, while realizing that even for Alphabet, financial constraints exist and sufficiently large contributions from the public sector can't be expected to be greater than rounding errors. Consequently, Waymo is going to need to earn its societal value in order to justify its continued existence.
Becoming even safer can earn additional societal value; however, the magnitude of that gain is unlikely to be sufficient as viewed from Alphabet or by any potential acquirer. Improved safety is simply insufficiently valuable to those needing a ride to achieve financial viability and demonstrate a "proof-of-market." What Waymo needs to do is to quickly focus on building its more fundamental attributes of flexibility in time and space of giving safe rides, thus delivering an extremely high level of service, safely and affordably.
Once one can give a high level of service (LoS) one needs to realize that other modes (drive oneself, be driven by Limo/taxi/Uber/Lyft) also give high quality LoS that is safe and not cheap. For those for whom affordability is not important, Waymo is an "also ran" in the competition to welcome them as a loyal customer any time soon. Efforts to survive on just an LoS comparison is unlikely to be successful.
However, focusing the high-quality LoS on customers that will find the affordability aspects as substantive are likely to be disruptive. At prices they are willing to pay, high quality service is unavailable. Consequently, high-quality LoS will be disruptive to those where affordability matters most. Plus affordability builds on itself. Affordability builds scale which leads to even higher affordability where Moore's Law technology is doing the safe driving.
For a more straightforward reporting on Waymo's papers, see Andrew Hawkings: "Waymo has 7.1 million driverless miles — how does its driving compare to humans?". Alain is repeated below…