KMS Of Academy of mathematics and systems sciences, CAS
| A Minibatch Proximal Stochastic Recursive Gradient Algorithm Using a Trust-Region-Like Scheme and Barzilai-Borwein Stepsizes | |
| Yu, Tengteng1; Liu, Xin-Wei2; Dai, Yu-Hong3,4; Sun, Jie5,6 | |
| 2021-10-01 | |
| Source Publication | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
 |
| ISSN | 2162-237X |
| Volume | 32Issue:10Pages:4627-4638 |
| Abstract | We consider the problem of minimizing the sum of an average of a large number of smooth convex component functions and a possibly nonsmooth convex function that admits a simple proximal mapping. This class of problems arises frequently in machine learning, known as regularized empirical risk minimization (ERM). In this article, we propose mSRGTR-BB, a minibatch proximal stochastic recursive gradient algorithm, which employs a trust-region-like scheme to select stepsizes that are automatically computed by the Barzilai-Borwein method. We prove that mSRGTR-BB converges linearly in expectation for strongly and nonstrongly convex objective functions. With proper parameters, mSRGTR-BB enjoys a faster convergence rate than the state-of-the-art minibatch proximal variant of the semistochastic gradient method (mS2GD). Numerical experiments on standard data sets show that the performance of mSRGTR-BB is comparable to and sometimes even better than mS2GD with best-tuned stepsizes and is superior to some modern proximal stochastic gradient methods. |
| Keyword | Convergence Convex functions Risk management Gradient methods Learning systems Sun Barzilai-Borwein (BB) method empirical risk minimization (ERM) proximal method stochastic gradient trust-region |
| DOI | 10.1109/TNNLS.2020.3025383 |
| Indexed By | SCI |
| Language | 英语 |
| Funding Project | Chinese NSF[11671116] ; Chinese NSF[11701137] ; Chinese NSF[12071108] ; Chinese NSF[11631013] ; Chinese NSF[11991020] ; Chinese NSF[12021001] ; Major Research Plan of the NSFC[91630202] ; Beijing Academy of Artificial Intelligence (BAAI) |
| WOS Research Area | Computer Science ; Engineering |
| WOS Subject | Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic |
| WOS ID | WOS:000704111000031 |
| Publisher | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| Citation statistics | |
| Document Type | 期刊论文 |
| Identifier | http://ir.amss.ac.cn/handle/2S8OKBNM/59405 |
| Collection | 中国科学院数学与系统科学研究院 |
| Corresponding Author | Liu, Xin-Wei |
| Affiliation | 1.Hebei Univ Technol, Sch Artificial Intelligence, Tianjin 300401, Peoples R China 2.Hebei Univ Technol, Inst Math, Tianjin 300401, Peoples R China 3.Chinese Acad Sci, Acad Math & Syst Sci, State Key Lab Sci & Engn Comp LSEC, Beijing 100190, Peoples R China 4.Univ Chinese Acad Sci, Sch Math Sci, Beijing 100049, Peoples R China 5.Hebei Univ Technol, Inst Math, Tianjin 300401, Peoples R China 6.Natl Univ Singapore, Sch Business, Singapore 119245, Singapore |
| Recommended Citation GB/T 7714 | Yu, Tengteng,Liu, Xin-Wei,Dai, Yu-Hong,et al. A Minibatch Proximal Stochastic Recursive Gradient Algorithm Using a Trust-Region-Like Scheme and Barzilai-Borwein Stepsizes[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2021,32(10):4627-4638. |
| APA | Yu, Tengteng,Liu, Xin-Wei,Dai, Yu-Hong,&Sun, Jie.(2021).A Minibatch Proximal Stochastic Recursive Gradient Algorithm Using a Trust-Region-Like Scheme and Barzilai-Borwein Stepsizes.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,32(10),4627-4638. |
| MLA | Yu, Tengteng,et al."A Minibatch Proximal Stochastic Recursive Gradient Algorithm Using a Trust-Region-Like Scheme and Barzilai-Borwein Stepsizes".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 32.10(2021):4627-4638. |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment