Interactive Q-learning

Kristin Linn

Department of Statistics

North Carolina State University

Engr Room 1602

4400 University Drive, Fairfax, VA 22030

Time: 10:30 A.M. - 11:30 P.M.

\Date: Wednesday, Jan 22, 2014 


Evidence-based rules for optimal treatment allocation are key components in the quest for efficient, effective health care delivery.  Q-learning, an approximate dynamic programming algorithm, is a popular method for estimating optimal sequential decision rules from data. Q-learning requires modeling nonsmooth, nonmonotone transformations of the data, complicating the search for adequately expressive, yet parsimonious, statistical models.  The default Q-learning working model is multiple linear regression, which is misspecified under most data-generating models.  We propose an alternative strategy for estimating optimal sequential decision rules for which the requisite statistical modeling does not depend on nonsmooth, nonmonotone transformed data and is thus amenable to established statistical approaches for exploratory data analysis, model building and validation. We derive the new method, Interactive Q-learning (IQ-learning), via an interchange in the order of certain steps in Q-learning.  IQ-learning performs favorably in simulated experiments, and an illustrative case study is provided using data from a sequentially randomized trial studying depression therapies.

Yunpeng Zhao, PhD

Assistant Professor
Department of Statistics
Volgenau School of Engineering
George Mason University
Engineering Building, Room 1719, MS 4A7
4400 University Drive
Fairfax, VA 22030-4444
Phone: 703-993-1674
Email: [log in to unmask]