Access the full text.
Sign up today, get DeepDyve free for 14 days.
Steve Brooks, A. Gelman, Galin Jones, X. Meng (2012)
Handbook of Markov Chain Monte Carlo: Hardcover: 619 pages Publisher: Chapman and Hall/CRC Press (first edition, May 2011) Language: English ISBN-10: 1420079417CHANCE, 25
Daniel Brown, S. Niekum (2017)
Efficient Probabilistic Performance Bounds for Inverse Reinforcement LearningArXiv, abs/1707.00724
Nikola Banovic, Antti Oulasvirta, P. Kristensson (2019)
Computational Modeling in Human-Computer InteractionExtended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems
Deepak Ramachandran, Eyal Amir (2007)
Bayesian Inverse Reinforcement Learning
Youngjae Chang, Akhil Mathur, Anton Isopoussu, Junehwa Song, F. Kawsar (2020)
A Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity RecognitionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4
Jonathan Goodman, J. Weare (2010)
Ensemble samplers with affine invariance, 5
Brian Ziebart (2012)
Probabilistic pointing target prediction via inverse optimal control
Shuochao Yao, Yiran Zhao, Huajie Shao, Chao Zhang, Aston Zhang, Shaohan Hu, Dongxin Liu, Shengzhong Liu, Lu Su, T. Abdelzaher (2018)
SenseGAN: Enabling Deep Learning for Internet of Things with a Semi-Supervised FrameworkProc. ACM Interact. Mob. Wearable Ubiquitous Technol., 2
K. Farrahi, D. Gática-Pérez (2012)
Extracting Mobile Behavioral Patterns with the Distant N-Gram Topic Model2012 16th International Symposium on Wearable Computers
Robert Wilson, A. Collins (2019)
Ten simple rules for the computational modeling of behavioral dataeLife, 8
J. Salsman, David Victorson, Seung Choi, A. Peterman, A. Heinemann, C. Nowinski, D. Cella (2013)
Development and validation of the positive affect and well-being scale for the neurology quality of life (Neuro-QOL) measurement systemQuality of Life Research, 22
A. Kratz, S. Murphy, T. Braley (2017)
Pain, Fatigue, and Cognitive Symptoms Are Temporally Associated Within but Not Across Days in Multiple Sclerosis.Archives of physical medicine and rehabilitation, 98 11
R. Ghanem, D. Higdon, H. Owhadi (2017)
Handbook of Uncertainty Quantification
S. Gershman, E. Horvitz, J. Tenenbaum (2015)
Computational rationality: A converging paradigm for intelligence in brains, minds, and machinesScience, 349
S. Maxwell, Ken Kelley, Joseph Rausch (2008)
Sample size planning for statistical power and accuracy in parameter estimation.Annual review of psychology, 59
D. Foreman-Mackey, D. Hogg, D. Lang, J. Goodman (2012)
emcee: The MCMC HammerPublications of the Astronomical Society of the Pacific, 125
A. Kratz, T. Braley, Emily Foxen-Craft, E. Scott, John Murphy, S. Murphy (2017)
How Do Pain, Fatigue, Depressive, and Cognitive Symptoms Relate to Well-Being and Social and Physical Functioning in the Daily Lives of Individuals With Multiple Sclerosis?Archives of physical medicine and rehabilitation, 98 11
Rebecca Adaimi, Edison Thomaz (2019)
Leveraging Active Learning and Conditional Mutual Information to Minimize Data Annotation in Human Activity RecognitionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3
Chelsea Finn, S. Levine, P. Abbeel (2016)
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
James Joyce (2011)
Kullback-Leibler Divergence
[ (2019)
Integrating activity recognition and nursing care records: The system, deployment, and a verification studyProceedings of the ACM on Interactive, 3
Jacob Cohen (1977)
CHAPTER 1 – The Concepts of Power Analysis
J. Bernardo, J. Berger, A. Dawid, A. Smith, Peter Muller (1999)
Simulation-Based Optimal Design
A. Kangasrääsiö, Samuel Kaski (2017)
Inverse reinforcement learning from summary dataMachine Learning, 107
R. Soukoreff, I. Mackenzie (2004)
Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts' law research in HCIInt. J. Hum. Comput. Stud., 61
T. Ziemssen, R. Kern, K. Thomas (2016)
Multiple sclerosis: clinical profiling and data collection as prerequisite for personalized medicine approachBMC Neurology, 16
[ (2017)
How do pain, fatigue, depressive, and cognitive symptoms relate to well-being and social and physical functioning in the daily lives of individuals with multiple sclerosis? Archives of Physical Medicine and Rehabilitation 98, 11 (2017), 2160–2166How do pain
J. Berger (1988)
Statistical Decision Theory and Bayesian Analysis
C. Adcock (1997)
Sample size determination : a reviewThe Statistician, 46
Xiuli Chen, S. Starke, Chris Baber, Andrew Howes (2017)
A Cognitive Model of How People Make Decisions Through Interaction with Visual DisplaysProceedings of the 2017 CHI Conference on Human Factors in Computing Systems
Sozo Inoue, Xincheng Pan (2016)
Supervised and Unsupervised Transfer Learning for Activity Recognition from Simple In-home SensorsProceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services
Nikola Banovic, Anqi Wang, Yanfeng Jin, Christie Chang, Julian Ramos, A. Dey, Jennifer Mankoff (2017)
Leveraging Human Routine Models to Detect and Generate Human BehaviorsProceedings of the 2017 CHI Conference on Human Factors in Computing Systems
Natthaphan Boonyanunta, P. Zeephongsekul (2004)
Predicting the Relationship Between the Size of Training Sample and the Predictive Power of Classifiers
Mahdi Imani, U. Braga-Neto (2019)
Control of Gene Regulatory Networks Using Bayesian Inverse Reinforcement LearningIEEE/ACM Transactions on Computational Biology and Bioinformatics, 16
Radford Neal (2011)
Handbook of Markov Chain Monte Carlo
D. Lindley (1997)
The choice of sample size, 46
Qiang Liu, Dilin Wang (2016)
Stein Variational Gradient Descent: A General Purpose Bayesian Inference AlgorithmArXiv, abs/1608.04471
Arun Venkatraman, M. Hebert, J. Bagnell (2015)
Improving Multi-Step Prediction of Learned Time Series Models
Richard Lewis, A. Howes, Satinder Singh (2014)
Computational Rationality: Linking Mechanism and Behavior Through Bounded Utility MaximizationTopics in cognitive science, 6 2
Iuliia Kotseruba, John Tsotsos (2018)
40 years of cognitive architectures: core cognitive abilities and practical applicationsArtificial Intelligence Review, 53
Brian Ziebart, Andrew Maas, A. Dey, J. Bagnell (2008)
Navigate like a cabbie: probabilistic reasoning from observed context-aware behavior
Edmon Begoli, Tanmoy Bhattacharya, D. Kusnezov (2019)
The need for uncertainty quantification in machine-assisted medical decision makingNature Machine Intelligence, 1
A. O’Hagan, C. Buck, A. Daneshkhah, J. Eiser, H. Paul, Garthwaite, D. Jenkinson, J. Oakley, T. Rakow (2006)
Uncertain Judgements: Eliciting Experts' Probabilities
W. Oberkampf, T. Trucano, C. Hirsch (2004)
Verification, Validation, and Predictive Capability in Computational Engineering and PhysicsApplied Mechanics Reviews, 57
Burr Settles (2009)
Active Learning Literature Survey
A. Chorin, M. Morzfeld, Xuemin Tu (2010)
Communications in Applied Mathematics and Computational Science
S. Hauser, J. Oksenberg (2006)
The Neurobiology of Multiple Sclerosis: Genes, Inflammation, and NeurodegenerationNeuron, 52
Radford Neal (2006)
Pattern Recognition and Machine LearningPattern Recognition and Machine Learning
Nan Li, S. Kambhampati, S. Yoon (2009)
Learning Probabilistic Hierarchical Task Networks to Capture User PreferencesArXiv, abs/1006.0274
A. Kangasrääsiö, Jussi Jokinen, Antti Oulasvirta, Andrew Howes, Samuel Kaski (2019)
Parameter Inference for Computational Cognitive Models with Approximate Bayesian ComputationCognitive Science, 43
M. Puterman (1994)
Markov Decision Processes: Discrete Stochastic Dynamic Programming
R. Melnik (2015)
Universality of Mathematical Models in Understanding Nature, Society, and Man‐Made World
[ (2018)
RDeepSense: Reliable deep mobile computing models with uncertainty estimationsProceedings of the ACM on Interactive, 1
A. Oulasvirta, Jussi Jokinen, Andrew Howes (2022)
Computational Rationality as a Theory of InteractionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems
Galin Jones, Qian Qin (2021)
Markov Chain Monte Carlo in Practice.Annual review of public health
P. Green (1996)
Markov chain Monte Carlo in Practice
[ (2017)
Handbook of uncertainty quantificationSpringer International Publishing
D. Glowacka, Tuukka Ruotsalo, Ksenia Konyushkova, Kumaripaba Athukorala, Samuel Kaski, Giulio Jacucci (2013)
Directing exploratory search: reinforcement learning from user interactions with keywords
Christoph Gebhardt, Antti Oulasvirta, Otmar Hilliges (2020)
Hierarchical Reinforcement Learning as a Model of Human Task InterleavingArXiv, abs/2001.02122
D. Scott (1992)
Multivariate Density Estimation, Theory, Practice and VisualizationThe Statistician, 43
AdaimiRebecca, ThomazEdison (2019)
Leveraging Active Learning and Conditional Mutual Information to Minimize Data Annotation in Human Activity Recognition
Sozo Inoue, P. Lago, Tahera Hossain, Tittaya Mairittha, Nattaya Mairittha (2019)
Integrating Activity Recognition and Nursing Care RecordsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3
Brian Ziebart, J. Bagnell, A. Dey (2010)
Modeling Interaction via the Principle of Maximum Causal Entropy
M. Magnússon (2000)
Discovering hidden time patterns in behavior: T-patterns and their detectionBehavior Research Methods, Instruments, & Computers, 32
D. Blei, A. Kucukelbir, Jon McAuliffe (2016)
Variational Inference: A Review for StatisticiansJournal of the American Statistical Association, 112
J. Helton, T. Trucano, M. Pilch (2006)
Ideas underlying quantification of margins and uncertainties(QMU): a white paper.
L. Crespo, S. Kenny, D. Giesy (2014)
The NASA Langley Multidisciplinary Uncertainty Quantification Challenge
A. Kratz, S. Murphy, T. Braley (2017)
Ecological Momentary Assessment of Pain, Fatigue, Depressive, and Cognitive Symptoms Reveals Significant Daily Variability in Multiple Sclerosis.Archives of physical medicine and rehabilitation, 98 11
R. Alejo, V. García, J. Pacheco-Sánchez (2015)
An Efficient Over-sampling Approach Based on Mean Square Error Back-propagation for Dealing with the Multi-class Imbalance ProblemNeural Processing Letters, 42
Rosa Figueroa, Q. Zeng-Treitler, S. Kandula, Long Ngo (2012)
Predicting sample size required for classification performanceBMC Medical Informatics and Decision Making, 12
Shuochao Yao, Yiran Zhao, Huajie Shao, Aston Zhang, Chao Zhang, Shen Li, T. Abdelzaher (2017)
RDeepSenseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1
Nikola Banovic, Tofi Buzali, Fanny Chevalier, Jennifer Mankoff, A. Dey (2016)
Modeling and Understanding Human Routine BehaviorProceedings of the 2016 CHI Conference on Human Factors in Computing Systems
U. Toussaint (2011)
Bayesian inference in physicsReviews of Modern Physics, 83
L. Rojas-Barahona, Christophe Cerisara (2014)
Bayesian Inverse Reinforcement Learning for Modeling Conversational Agents in a Virtual Environment
G. Parmigiani, Lurdes Inoue, H. Lopes (2009)
Decision Theory: Principles and Approaches
[ (2001)
CKennedy and Anthony O’Hagan, 63
(2001)
Statistical modeling: The two cultures
Yuchen Cui, S. Niekum (2017)
Active Learning from Critiques via Bayesian Inverse Reinforcement Learning
Gideon Mann, A. McCallum (2010)
Generalized Expectation Criteria for Semi-Supervised Learning with Weakly Labeled DataJ. Mach. Learn. Res., 11
M. Kennedy, A. O’Hagan (2001)
Bayesian calibration of computer modelsJournal of the Royal Statistical Society: Series B (Statistical Methodology), 63
R. Fiebrink, P. Cook, D. Trueman (2011)
Human model evaluation in interactive supervised learningProceedings of the SIGCHI Conference on Human Factors in Computing Systems
A. Ng, Stuart Russell (2000)
Pharmacokinetics of a novel formulation of ivermectin after administration to goats
[ (2003)
Probability Theory: The Logic of ScienceCambridge University Press.
R. Lenth (2001)
Some Practical Guidelines for Effective Sample Size DeterminationThe American Statistician, 55
T. Cover, Joy Thomas (2006)
Elements of information theory (2. ed.)
[ (2019)
Computational modeling in human-computer interactionProceedings of the Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems.Association for Computing Machinery, 2019
L. Kaelbling, M. Littman, A. Moore (1996)
Reinforcement Learning: A SurveyJ. Artif. Intell. Res., 4
K. Fukunaga, R. Hayes (1989)
Effects of Sample Size in Classifier DesignIEEE Trans. Pattern Anal. Mach. Intell., 11
Hoon Kim (2000)
Monte Carlo Statistical MethodsTechnometrics, 42
Katri Leino, Kashyap Todi, Antti Oulasvirta, M. Kurimo (2019)
Computer-supported form design using keystroke-level modeling with reinforcement learningProceedings of the 24th International Conference on Intelligent User Interfaces: Companion
Nikola Banovic, Jennifer Mankoff, A. Dey (2018)
Computational Model of Human Routine Behaviours
K. Chaloner, I. Verdinelli (1995)
Bayesian Experimental Design: A ReviewStatistical Science, 10
N. Eagle, A. Pentland (2009)
Eigenbehaviors: identifying structure in routineBehavioral Ecology and Sociobiology, 63
Huaiyu Zhu (1997)
On Information and Sufficiency
Christoph Gebhardt, Brian Hecox, B. Opheusden, Daniel Wigdor, James Hillis, Otmar Hilliges, Hrvoje Benko (2019)
Learning Cooperative Personalized Policies from Gaze DataProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology
A. Sadilek, John Krumm (2012)
Far Out: Predicting Long-Term Human MobilityProceedings of the AAAI Conference on Artificial Intelligence
Nikola Banovic, Tovi Grossman, G. Fitzmaurice (2013)
The effect of time-based cost of error in target-directed pointing tasksProceedings of the SIGCHI Conference on Human Factors in Computing Systems
Yunxiu Zeng, Kai Xu, Quanjun Yin, Long Qin, Yabing Zha, W. Yeoh (2018)
Inverse Reinforcement Learning Based Human Behavior Modeling for Goal Recognition in Dynamic Local Network Interdiction
[ (2000)
Bayesian TheoryJohn Wiley & Sons
Stéphane Ross, Geoffrey Gordon, J. Bagnell (2010)
No-Regret Reductions for Imitation Learning and Structured PredictionArXiv, abs/1011.0686
[ (2000)
Algorithms for inverse reinforcement learningProceedings of the 17th International Conference on Machine Learning.Morgan Kaufmann Publishers Inc.
S. Sarcar, Jussi Jokinen, Antti Oulasvirta, Zhenxin Wang, Chaklam Silpasuwanchai, Xiangshi Ren (2018)
Ability-Based Optimization of Touchscreen InteractionsIEEE Pervasive Computing, 17
(2006)
Data Analysis: A Bayesian Tutorial (2nd ed.)
Katri Leino, Antti Oulasvirta, M. Kurimo (2019)
RL-KLM: automating keystroke-level modeling with reinforcement learningProceedings of the 24th International Conference on Intelligent User Interfaces
Xuhai Xu, Prerna Chikersal, Afsaneh Doryab, Daniella Villalba, Janine Dutcher, Michael Tumminia, Tim Althoff, Sheldon Cohen, Kasey Creswell, J. Creswell, Jennifer Mankoff, A. Dey (2019)
Leveraging Routine Behavior and Contextually-Filtered Features for Depression Detection among College StudentsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3
N. Japkowicz, Shaju Stephen (2002)
The class imbalance problem: A systematic studyIntell. Data Anal., 6
E. Jaynes, T. Grandy, Ray Smith, Tom Loredo, M. Tribus, J. Skilling, G. Bretthorst (2005)
Probability theory: the logic of scienceThe Mathematical Intelligencer, 27
X. Huan, Y. Marzouk (2011)
Simulation-based optimal Bayesian experimental design for nonlinear systemsJ. Comput. Phys., 232
Computational models that formalize complex human behaviors enable study and understanding of such behaviors. However, collecting behavior data required to estimate the parameters of such models is often tedious and resource intensive. Thus, estimating dataset size as part of data collection planning (also known as Sample Size Determination) is important to reduce the time and effort of behavior data collection while maintaining an accurate estimate of model parameters. In this article, we present a sample size determination method based on Uncertainty Quantification (UQ) for a specific Inverse Reinforcement Learning (IRL) model of human behavior, in two cases: (1) pre-hoc experiment design—conducted in the planning stage before any data is collected, to guide the estimation of how many samples to collect; and (2) post-hoc dataset analysis—performed after data is collected, to decide if the existing dataset has sufficient samples and whether more data is needed. We validate our approach in experiments with a realistic model of behaviors of people with Multiple Sclerosis (MS) and illustrate how to pick a reasonable sample size target. Our work enables model designers to perform a deeper, principled investigation of the effects of dataset size on IRL model parameters.
ACM Transactions on Computer-Human Interaction (TOCHI) – Association for Computing Machinery
Published: Mar 7, 2023
Keywords: Sample size determination
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.