November 8. 2018

Uber has released its Voluntary Safety Self-Assessment (VSSA).  The United States DOT introduced VSSAs in their federal automated vehicle policy 2.0, A Vision for Safety, and in 3.0 Preparing for the Future of Transportation, USDOT emphasized the option for entities to publish documents that demonstrate the safety aspects of automated driving systems to the public.

Five other AV companies have previously released VSSA’s, but Uber’s VSSA release included two major elements that the othersdo not. The document, entitled A Principled Approach to Safety, discusses performance measurement (though Ford does mention performance measurement once in their VSSA), and Uber released it in parallel with an external review of the safety culture at Uber.

The law firm LeClair Ryan conducted the independent review, authored by qualified lawyers with experience on the National Transportation Safety Board, at DOT and with the FAA. Similar to recent rhetoric from DOT, the report draws heavy parallels to safety programs in the aviation industry. It focuses on the importance of safety culture within a company and an industry, using aviation and energy examples, as well as examples in current vehicle manufacturing. The report provides multiple specific recommendations relating to:

• Establishing and implementing a safety management system
• Bolstering leadership, communication, and programming around safety culture
• Extending and enhancing safety training
• Evaluating current and implement future safety policies processes and procedures

Though tailored to observations of culture and procedures at Uber, many of these recommendations can be applied to other AV manufacturers and to the industry as a collective.

The Independent Review of Safety Culture isn’t the only external evaluation Uber recently sought. In 2017 (before the death of Elaine Herzberg in an Uber AV collision in Tempe, AZ), Uber Advanced Technologies Group approached Rand Corporation with support for research in regards to developing a framework for AV safety. The resulting report recommends safety measures for development, demonstration, and deployment of highly automated vehicles (SAE levels 4 and 5). Rand identifies “leading” and “lagging” measures, to assess variables correlated with safety, such as compliance with traffic laws, and actual safety outcomes, such as collisions.

Uber acknowledges in their VSSA that standard, measurable metrics are necessary to quantify the safety of AVs. They state that, “We anticipate that demonstrating safer performance than human drivers will require that we quantify safe driving with tractable, credible metrics.” They cite the Rand report, noting examples, but do not fully develop or explain their own performance measurement or assessment framework.

The closest any of the other submitted VSSAs get to acknowledging the need for performance measures is Ford’s mention that historically, the company tests performance, and Nuro’s identification of the need to measure success.  Nuro’s VSSA says, “We measure our success by how many people’s lives are substantially improved by our products,” but they also do not identify measurable metrics to define success, particularly in terms of safety.

As more companies submit VSSAs, public awareness of AV manufacturers’ safety considerations also increases. However, these documents are not all-encompassing assessments of AV safety, and the independent evaluations such as those commissioned by Uber, as well as academic and industry research on safety performance measures associated with automated technologies helps provide external perspective and accountability along the road to enhancing public safety.