Automated Driving System Safety: Miles for 95% Confidence in “Vision Zero” 2020-01-1205

Engineering reliability models from RAND, MobilEye, and Volvo concluded that billions of miles of on-road data were required to validate that the real-world fatality rate of an “Automated Driving System-equipped vehicle” (AV) fleet for an improvement over human-driven conventional vehicles (CV). RAND said 5 billion miles for 20%, MobileEye 30 billion for 99.9%, and Volvo 5 billion for 50% improvement. All these models used the Gaussian distribution, which is inaccurate for low crash numbers. The current study proposes a new epidemiologic method and criterion to validate real-world AV data with 95% confidence for zero to ten fatal crashes. The upper confidence limit (UL) of the AV fatal crash rate has to be lower than the CV fatal crash rate with 95% confidence. That criterion is met if the UL of the AV fatal crash incidence rate ratio estimate is below one. That UL was estimated using the mid-P exact method for calculating confidence limits for a dual Poisson process, using a one-tailed 95% confidence level. The required AV mileage was adjusted by trial and error until the UL was just below 1. The method was applied to the real-world fatal crash and mileage data from California (Waymo and GM Cruise), U.S. (Uber and Tesla), China (Tesla), and Sweden (Volvo) to calculate the AV miles required to validate a “Vision Zero” AV fatal crash objective with 95% confidence. For AV fleets in California, U.S., and China, the AV miles required are only in the millions. In Sweden, 1.9 billion AV miles are required because its CV fatal crash rates are already so low. In conclusion, epidemiologic analysis of real-world data finds that only millions of real-world AV data are typically required to validate AV safety with 95% confidence to a “Vision Zero” fatal crash criterion.