Stochastic proximal quasi-Newton methods for non-convex composite optimization

doi:10.1080/10556788.2018.1471141

CSpace

	Stochastic proximal quasi-Newton methods for non-convex composite optimization
	Wang, Xiaoyu 1,2; Wang, Xiao 2; Yuan, Ya-xiang 1
	2019-09-03
发表期刊	OPTIMIZATION METHODS & SOFTWARE
ISSN	1055-6788
卷号	34 期号:5 页码:922-948
摘要	In this paper, we propose a generic algorithmic framework for stochastic proximal quasi-Newton (SPQN) methods to solve non-convex composite optimization problems. Stochastic second-order information is explored to construct proximal subproblem. Under mild conditions we show the non-asympotic convergence of the proposed algorithm to stationary point of original problems and analyse its computational complexity. Besides, we extend the proximal form of Polyak-Lojasiewicz (PL) inequality to constrained settings and obtain the constrained proximal PL (CP-PL) inequality. Under CP-PL inequality linear convergence rate of the proposed algorithm is achieved. Moreover, we propose a modified self-scaling symmetric rank one incorporated in the framework for SPQN method, which is called stochastic symmetric rank one method. Finally, we report some numerical experiments to reveal the effectiveness of the proposed algorithm.
关键词	Non-convex composite optimization Polyak-Lojasiewicz (PL) inequality stochastic gradient stochastic variance reduction gradient symmetric rank one method rank one proximity operator complexity bound
DOI	10.1080/10556788.2018.1471141
语种	英语
资助项目	National Natural Science Foundation of China[11331012] ; National Natural Science Foundation of China[11301505] ; National Natural Science Foundation of China[11731013] ; National Natural Science Foundation of China[11688101]
WOS研究方向	Computer Science ; Operations Research & Management Science ; Mathematics
WOS类目	Computer Science, Software Engineering ; Operations Research & Management Science ; Mathematics, Applied
WOS记录号	WOS:000486079100002
出版者	TAYLOR & FRANCIS LTD
引用统计
文献类型	期刊论文
条目标识符	http://ir.amss.ac.cn/handle/2S8OKBNM/35632
专题	中国科学院数学与系统科学研究院
通讯作者	Wang, Xiao
作者单位	1.Chinese Acad Sci, LSEC, Inst Computat Math & Sci Engn Comp, AMSS, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Sch Math Sci, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Wang, Xiaoyu,Wang, Xiao,Yuan, Ya-xiang. Stochastic proximal quasi-Newton methods for non-convex composite optimization[J]. OPTIMIZATION METHODS & SOFTWARE,2019,34(5):922-948.
APA	Wang, Xiaoyu,Wang, Xiao,&Yuan, Ya-xiang.(2019).Stochastic proximal quasi-Newton methods for non-convex composite optimization.OPTIMIZATION METHODS & SOFTWARE,34(5),922-948.
MLA	Wang, Xiaoyu,et al."Stochastic proximal quasi-Newton methods for non-convex composite optimization".OPTIMIZATION METHODS & SOFTWARE 34.5(2019):922-948.