Access the full text.
Sign up today, get DeepDyve free for 14 days.
M. Fay, E. Feuer (1997)
Confidence intervals for directly standardized rates: a method based on the gamma distribution.Statistics in medicine, 16 7
H. Ng, G. Filardo, G. Zheng (2008)
Confidence interval estimating procedures for standardized incidence ratesComput. Stat. Data Anal., 52
Weizhen Wang (2013)
A note on bootstrap confidence intervals for proportionsStatistics & Probability Letters, 83
M. Swift (2010)
A simulation study comparing methods for calculating confidence intervals for directly standardized ratesComput. Stat. Data Anal., 54
Matthew Schwall, Tom Daniel, T. Victor, F. Favarò, Henning Hohnhold (2020)
Waymo Public Road Safety Performance DataArXiv, abs/2011.00038
T. Gneiting, A. Raftery (2007)
Strictly Proper Scoring Rules, Prediction, and EstimationJournal of the American Statistical Association, 102
W. Rolke, Angel Lopez, Janice Conrad (2004)
Limits and confidence intervals in the presence of nuisance parametersNuclear Instruments & Methods in Physics Research Section A-accelerators Spectrometers Detectors and Associated Equipment, 551
Y. Yatracos (1998)
A SMALL SAMPLE OPTIMALITY PROPERTY OF THE M.L.E., 60
W. Cochran (1963)
Sampling Techniques, 3rd Edition
N. Webb, Dan Smith, Christopher Ludwick, T. Victor, Q. Hommes, F. Favarò, George Ivanov, Tom Daniel (2020)
Waymo's Safety Methodologies and Safety Readiness DeterminationsArXiv, abs/2011.00054
A. Dempster, N. Laird, D. Rubin (1977)
Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper
M. Fay, Sungwook Kim (2017)
Confidence intervals for directly standardized rates using mid‐p gamma intervalsBiometrical Journal, 59
R. Tiwari, L. Clegg, Zhaohui Zou (2006)
Efficient interval estimation for age-adjusted cancer ratesStatistical Methods in Medical Research, 15
D. Venzon, S. Moolgavkar (1988)
A Method for Computing Profile-Likelihood- Based Confidence IntervalsApplied statistics, 37
Chin-Shan Chuang, T. Lai (1998)
Hybrid resampling methods for confidence intervals
R. Buehler (1957)
Confidence Intervals for the Product of Two Binomial ParametersJournal of the American Statistical Association, 52
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, P. Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, S. Ettinger, Maxim Krivokon, A. Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, Dragomir Anguelov (2019)
Scalability in Perception for Autonomous Driving: Waymo Open Dataset2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
E. Ziegel, E. Lehmann, G. Casella (1950)
Theory of point estimation
Ekim Yurtsever, Jacob Lambert, Alexander Carballo, K. Takeda (2019)
A Survey of Autonomous Driving: Common Practices and Emerging TechnologiesIEEE Access, 8
Sen Bodhisattva, Michael Woodroofe, M. Walker (2009)
ON THE UNIFIED METHOD WITH NUISANCE PARAMETERSStatistica Sinica, 19
P. Bickel, K. Doksum (2015)
Mathematical Statistics: Basic Ideas and Selected Topics, Volume I, Second Edition
M. Kenward (2007)
An Introduction to the Bootstrap
K. Perez (2014)
Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment
Autonomous vehicles are continually increasing their presence on public roads. However, before any new autonomous driving software can be approved, it must first undergo a rigorous assessment of driving quality. These quality evaluations typically focus on estimating the frequency of (undesirable) behavioral events. While rate estimation would be straight‐forward with complete data, in the autonomous driving setting this estimation is greatly complicated by the fact that detecting these events within large driving logs is a non‐trivial task that often involves human reviewers. In this article, we outline a streaming partial tiered event review configuration that ensures both high recall and high precision on the events of interest. In addition, the framework allows for valid streaming estimates at any phase of the data collection process, even when labels are incomplete, for which we develop the maximum likelihood estimate and show it is unbiased. Constructing honest and effective confidence intervals (CI) for these rate estimates, particularly for rare safety‐critical events, is a novel and challenging statistical problem due to the complexity of the data likelihood. We develop and compare several CI approximations, including a novel gamma CI method that approximates the exact but intractable distribution with a weighted sum of independent Poisson random variables. There is a clear trade‐off between statistical coverage and interval width across the different CI methods, and the extent of this trade‐off varies depending on the specific application settings (e.g., rare vs. common events). In particular, we argue that our proposed CI method is the best‐suited when estimating the rate of safety‐critical events where guaranteed coverage of the true parameter value is a prerequisite to safely launching a new ADS on public roads.
Applied Stochastic Models in Business and Industry – Wiley
Published: Sep 1, 2023
Keywords: autonomous vehicle; confidence interval; gamma method; rare events; stratified sampling; tiered review; weighted sum of independent Poisson
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.