*Interactive Q-learning*

*Kristin Linn*

*Department of Statistics*

*North Carolina State University*

*Engr Room 1602*

*4400 University Drive, Fairfax, VA 22030*

*Time: 10:30 A.M. - 11:30 P.M.*

*\Date: Wednesday, **Jan 22**, 201**4*


Evidence-based rules for optimal treatment allocation are key components in
the quest for efficient, effective health care delivery.  Q-learning, an
approximate dynamic programming algorithm, is a popular method for
estimating optimal sequential decision rules from data. Q-learning requires
modeling nonsmooth, nonmonotone transformations of the data, complicating
the search for adequately expressive, yet parsimonious, statistical models.
The default Q-learning working model is multiple linear regression, which
is misspecified under most data-generating models.  We propose an
alternative strategy for estimating optimal sequential decision rules for
which the requisite statistical modeling does not depend on nonsmooth,
nonmonotone transformed data and is thus amenable to established
statistical approaches for exploratory data analysis, model building and
validation. We derive the new method, Interactive Q-learning (IQ-learning),
via an interchange in the order of certain steps in Q-learning.  IQ-learning
performs favorably in simulated experiments, and an illustrative case study
is provided using data from a sequentially randomized trial studying
depression therapies.

Yunpeng Zhao, PhD

Assistant Professor
Department of Statistics
Volgenau School of Engineering <[log in to unmask]>
George Mason University
Engineering Building, Room 1719, MS 4A7
4400 University Drive
Fairfax, VA 22030-4444

Phone: 703-993-1674
Email: [log in to unmask]