KMS Of Academy of mathematics and systems sciences, CAS
| Stein variational gradient descent with local approximations | |
| Yan, Liang1; Zhou, Tao2 | |
| 2021-12-01 | |
| Source Publication | COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING
 |
| ISSN | 0045-7825 |
| Volume | 386Pages:20 |
| Abstract | Bayesian computation plays an important role in modern machine learning and statistics to reason about uncertainty. A key computational challenge in Bayesian inference is to develop efficient techniques to approximate, or draw samples from posterior distributions. Stein variational gradient decent (SVGD) has been shown to be a powerful approximate inference algorithm for this issue. However, the vanilla SVGD requires calculating the gradient of the target density and cannot be applied when the gradient is unavailable or too expensive to evaluate. In this paper we explore one way to address this challenge by the construction of a local surrogate for the target distribution in which the gradient can be obtained in a much more computationally feasible manner. More specifically, we approximate the forward model using a deep neural network (DNN) which is trained on a carefully chosen training set, which also determines the quality of the surrogate. To this end, we propose a general adaptation procedure to refine the local approximation online without destroying the convergence of the resulting SVGD. This significantly reduces the computational cost of SVGD and leads to a suite of algorithms that are straightforward to implement. The new algorithm is illustrated on a set of challenging Bayesian inverse problems, and numerical experiments demonstrate a clear improvement in performance and applicability of standard SVGD. (C) 2021 Elsevier B.V. All rights reserved. |
| Keyword | Stein variational gradient decent Bayesian inference Deep learning Local approximation |
| DOI | 10.1016/j.cma.2021.114087 |
| Indexed By | SCI |
| Language | 英语 |
| Funding Project | NSF of China[11822111] ; NSF of China[11688101] ; NSF of China[11731006] ; NSF of China[11771081] ; science challenge project, China[TZ2018001] ; Zhishan Young Scholar Program of SEU, China ; National Key R&D Program of China[2020YFA0712000] ; science challenge project[TZ2018001] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA25000404] ; youth innovation promotion association (CAS), China |
| WOS Research Area | Engineering ; Mathematics ; Mechanics |
| WOS Subject | Engineering, Multidisciplinary ; Mathematics, Interdisciplinary Applications ; Mechanics |
| WOS ID | WOS:000703985200009 |
| Publisher | ELSEVIER SCIENCE SA |
| Citation statistics | |
| Document Type | 期刊论文 |
| Identifier | http://ir.amss.ac.cn/handle/2S8OKBNM/59339 |
| Collection | 中国科学院数学与系统科学研究院 |
| Corresponding Author | Yan, Liang |
| Affiliation | 1.Southeast Univ, Sch Math, Nanjing 210096, Peoples R China 2.Chinese Acad Sci, LSEC, Inst Computat Math, Acad Math & Syst Sci, Beijing 100190, Peoples R China |
| Recommended Citation GB/T 7714 | Yan, Liang,Zhou, Tao. Stein variational gradient descent with local approximations[J]. COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING,2021,386:20. |
| APA | Yan, Liang,&Zhou, Tao.(2021).Stein variational gradient descent with local approximations.COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING,386,20. |
| MLA | Yan, Liang,et al."Stein variational gradient descent with local approximations".COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING 386(2021):20. |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment