A Stochastic Approximation Frame Algorithm with Adaptive Directions

	A Stochastic Approximation Frame Algorithm with Adaptive Directions
	Xu, Zi 2; Dai, Yu-Hong1
	2008-11-01
发表期刊	NUMERICAL MATHEMATICS-THEORY METHODS AND APPLICATIONS
ISSN	1004-8979
卷号	1 期号:4 页码:460-474
摘要	Stochastic approximation problem is to find some root or extremum of a nonlinear function for which only noisy measurements of the function are available. The classical algorithm for stochastic approximation problem is the Robbins-Monro (RM) algorithm, which uses the noisy evaluation of the negative gradient direction as the iterative direction. In order to accelerate the RM algorithm, this paper gives a frame algorithm using adaptive iterative directions. At each iteration, the new algorithm goes towards either the noisy evaluation of the negative gradient direction or some other directions under some switch criterions. Two feasible choices of the criterions are proposed and two corresponding frame algorithms are formed. Different choices of the directions under the same given switch criterion in the frame can also form different algorithms. We also proposed the simultanous perturbation difference forms for the two frame algorithms. The almost surely convergence of the new algorithms are all established. The numerical experiments show that the new algorithms are promising.
关键词	Stochastic approximation conjugate gradient adaptive directions
语种	英语
资助项目	NSF[10571171] ; NSF[40233029]
WOS研究方向	Mathematics
WOS类目	Mathematics, Applied ; Mathematics
WOS记录号	WOS:000207474100005
出版者	GLOBAL SCIENCE PRESS
引用统计
文献类型	期刊论文
条目标识符	http://ir.amss.ac.cn/handle/2S8OKBNM/6247
专题	计算数学与科学工程计算研究所
通讯作者	Dai, Yu-Hong
作者单位	1.Chinese Acad Sci, State Key Lab Sci & Engn Comp, Inst Computat Math & Sci Engn Comp, Acad Math & Syst Sci, Beijing 100190, Peoples R China 2.Shanghai Univ, Coll Sci, Dept Math, Shanghai 200444, Peoples R China
推荐引用方式 GB/T 7714	Xu, Zi,Dai, Yu-Hong. A Stochastic Approximation Frame Algorithm with Adaptive Directions[J]. NUMERICAL MATHEMATICS-THEORY METHODS AND APPLICATIONS,2008,1(4):460-474.
APA	Xu, Zi,&Dai, Yu-Hong.(2008).A Stochastic Approximation Frame Algorithm with Adaptive Directions.NUMERICAL MATHEMATICS-THEORY METHODS AND APPLICATIONS,1(4),460-474.
MLA	Xu, Zi,et al."A Stochastic Approximation Frame Algorithm with Adaptive Directions".NUMERICAL MATHEMATICS-THEORY METHODS AND APPLICATIONS 1.4(2008):460-474.