Gradient methods exploiting spectral properties

doi:10.1080/10556788.2020.1727476

CSpace

	Gradient methods exploiting spectral properties
	Huang, Yakui 1; Dai, Yu-Hong 2; Liu, Xin-Wei 1; Zhang, Hongchao 3
	2020-02-15
发表期刊	OPTIMIZATION METHODS & SOFTWARE
ISSN	1055-6788
页码	25
摘要	We propose a new stepsize for the gradient method. It is shown that this new stepsize will converge to the reciprocal of the largest eigenvalue of the Hessian, when Dai-Yang's asymptotic optimal gradient method (Computational Optimization and Applications, 2006, 33(1): 73-88) is applied for minimizing quadratic objective functions. Based on this spectral property, we develop a monotone gradient method that takes a certain number of steps using the asymptotically optimal stepsize by Dai and Yang, and then follows by some short steps associated with this new stepsize. By employing one step retard of the asymptotic optimal stepsize, a nonmonotone variant of this method is also proposed. Under mild conditions, R-linear convergence of the proposed methods is established for minimizing quadratic functions. In addition, by combining gradient projection techniques and adaptive nonmonotone line search, we further extend those methods for general bound constrained optimization. Two variants of gradient projection methods combining with the Barzilai-Borwein stepsizes are also proposed. Our numerical experiments on both quadratic and bound constrained optimization indicate that the new proposed strategies and methods are very effective.
关键词	Gradient methods spectral property Barizilai-Borwein method linear convergence quadratic optimization bound constrained optimization
DOI	10.1080/10556788.2020.1727476
收录类别	SCI
语种	英语
资助项目	National Natural Science Foundation of China[11701137] ; National Natural Science Foundation of China[11631013] ; National Natural Science Foundation of China[11671116] ; National 973 Program of China[2015CB856002] ; China Scholarship Council[201806705007] ; USA National Science Foundation[1522654] ; USA National Science Foundation[1819161]
WOS研究方向	Computer Science ; Operations Research & Management Science ; Mathematics
WOS类目	Computer Science, Software Engineering ; Operations Research & Management Science ; Mathematics, Applied
WOS记录号	WOS:000514512500001
出版者	TAYLOR & FRANCIS LTD
引用统计
文献类型	期刊论文
条目标识符	http://ir.amss.ac.cn/handle/2S8OKBNM/50835
专题	中国科学院数学与系统科学研究院
通讯作者	Dai, Yu-Hong
作者单位	1.Hebei Univ Technol, Inst Math, Tianjin, Peoples R China 2.Chinese Acad Sci, Acad Math & Syst Sci, LSEC, Beijing, Peoples R China 3.Louisiana State Univ, Dept Math, Baton Rouge, LA 70803 USA
推荐引用方式 GB/T 7714	Huang, Yakui,Dai, Yu-Hong,Liu, Xin-Wei,et al. Gradient methods exploiting spectral properties[J]. OPTIMIZATION METHODS & SOFTWARE,2020:25.
APA	Huang, Yakui,Dai, Yu-Hong,Liu, Xin-Wei,&Zhang, Hongchao.(2020).Gradient methods exploiting spectral properties.OPTIMIZATION METHODS & SOFTWARE,25.
MLA	Huang, Yakui,et al."Gradient methods exploiting spectral properties".OPTIMIZATION METHODS & SOFTWARE (2020):25.