Gradient methods exploiting spectral properties
Huang, Yakui1; Dai, Yu-Hong2; Liu, Xin-Wei1; Zhang, Hongchao3
AbstractWe propose a new stepsize for the gradient method. It is shown that this new stepsize will converge to the reciprocal of the largest eigenvalue of the Hessian, when Dai-Yang's asymptotic optimal gradient method (Computational Optimization and Applications, 2006, 33(1): 73-88) is applied for minimizing quadratic objective functions. Based on this spectral property, we develop a monotone gradient method that takes a certain number of steps using the asymptotically optimal stepsize by Dai and Yang, and then follows by some short steps associated with this new stepsize. By employing one step retard of the asymptotic optimal stepsize, a nonmonotone variant of this method is also proposed. Under mild conditions, R-linear convergence of the proposed methods is established for minimizing quadratic functions. In addition, by combining gradient projection techniques and adaptive nonmonotone line search, we further extend those methods for general bound constrained optimization. Two variants of gradient projection methods combining with the Barzilai-Borwein stepsizes are also proposed. Our numerical experiments on both quadratic and bound constrained optimization indicate that the new proposed strategies and methods are very effective.
KeywordGradient methods spectral property Barizilai-Borwein method linear convergence quadratic optimization bound constrained optimization
Indexed BySCI
Funding ProjectNational Natural Science Foundation of China[11701137] ; National Natural Science Foundation of China[11631013] ; National Natural Science Foundation of China[11671116] ; National 973 Program of China[2015CB856002] ; China Scholarship Council[201806705007] ; USA National Science Foundation[1522654] ; USA National Science Foundation[1819161]
WOS Research AreaComputer Science ; Operations Research & Management Science ; Mathematics
WOS SubjectComputer Science, Software Engineering ; Operations Research & Management Science ; Mathematics, Applied
WOS IDWOS:000514512500001
Citation statistics
Document Type期刊论文
Corresponding AuthorDai, Yu-Hong
Affiliation1.Hebei Univ Technol, Inst Math, Tianjin, Peoples R China
2.Chinese Acad Sci, Acad Math & Syst Sci, LSEC, Beijing, Peoples R China
3.Louisiana State Univ, Dept Math, Baton Rouge, LA 70803 USA
Recommended Citation
GB/T 7714
Huang, Yakui,Dai, Yu-Hong,Liu, Xin-Wei,et al. Gradient methods exploiting spectral properties[J]. OPTIMIZATION METHODS & SOFTWARE,2020:25.
APA Huang, Yakui,Dai, Yu-Hong,Liu, Xin-Wei,&Zhang, Hongchao.(2020).Gradient methods exploiting spectral properties.OPTIMIZATION METHODS & SOFTWARE,25.
MLA Huang, Yakui,et al."Gradient methods exploiting spectral properties".OPTIMIZATION METHODS & SOFTWARE (2020):25.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Huang, Yakui]'s Articles
[Dai, Yu-Hong]'s Articles
[Liu, Xin-Wei]'s Articles
Baidu academic
Similar articles in Baidu academic
[Huang, Yakui]'s Articles
[Dai, Yu-Hong]'s Articles
[Liu, Xin-Wei]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Huang, Yakui]'s Articles
[Dai, Yu-Hong]'s Articles
[Liu, Xin-Wei]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.