Potential-based online policy iteration algorithms for Markov decision processes

doi:10.1109/TAC.2004.825647

CSpace > 系统科学研究所

	Potential-based online policy iteration algorithms for Markov decision processes
	Fang, HT; Cao, XR
	2004-04-01
发表期刊	IEEE TRANSACTIONS ON AUTOMATIC CONTROL
ISSN	0018-9286
卷号	49 期号:4 页码:493-505
摘要	Performance potentials play a crucial role in performance sensitivity analysis and policy iteration of Markov decision processes. The potentials can be estimated on a single sample path of a Markov process. In this paper, we propose two potential-based online policy iteration algorithms for performance optimization of Markov systems. The algorithms are based on online estimation of potentials and stochastic approximation. We prove that with these two algorithms the optimal. policy can be attained after it finite number of iterations. A simulation example,is given to illustrate the main ideas and the convergence rates of the algorithms.
关键词	Markov decision process potential recursive optimization
DOI	10.1109/TAC.2004.825647
语种	英语
WOS研究方向	Automation & Control Systems ; Engineering
WOS类目	Automation & Control Systems ; Engineering, Electrical & Electronic
WOS记录号	WOS:000220884800003
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://ir.amss.ac.cn/handle/2S8OKBNM/19742
专题	系统科学研究所
通讯作者	Fang, HT
作者单位	1.Chinese Acad Sci, Acad Math & Syst Sci, Lab Syst & Control, Beijing 100080, Peoples R China 2.Hong Kong Univ Sci & Technol, Kowloon, Hong Kong, Peoples R China
推荐引用方式 GB/T 7714	Fang, HT,Cao, XR. Potential-based online policy iteration algorithms for Markov decision processes[J]. IEEE TRANSACTIONS ON AUTOMATIC CONTROL,2004,49(4):493-505.
APA	Fang, HT,&Cao, XR.(2004).Potential-based online policy iteration algorithms for Markov decision processes.IEEE TRANSACTIONS ON AUTOMATIC CONTROL,49(4),493-505.
MLA	Fang, HT,et al."Potential-based online policy iteration algorithms for Markov decision processes".IEEE TRANSACTIONS ON AUTOMATIC CONTROL 49.4(2004):493-505.