Stein variational gradient descent with local approximations
Yan, Liang1; Zhou, Tao2
AbstractBayesian computation plays an important role in modern machine learning and statistics to reason about uncertainty. A key computational challenge in Bayesian inference is to develop efficient techniques to approximate, or draw samples from posterior distributions. Stein variational gradient decent (SVGD) has been shown to be a powerful approximate inference algorithm for this issue. However, the vanilla SVGD requires calculating the gradient of the target density and cannot be applied when the gradient is unavailable or too expensive to evaluate. In this paper we explore one way to address this challenge by the construction of a local surrogate for the target distribution in which the gradient can be obtained in a much more computationally feasible manner. More specifically, we approximate the forward model using a deep neural network (DNN) which is trained on a carefully chosen training set, which also determines the quality of the surrogate. To this end, we propose a general adaptation procedure to refine the local approximation online without destroying the convergence of the resulting SVGD. This significantly reduces the computational cost of SVGD and leads to a suite of algorithms that are straightforward to implement. The new algorithm is illustrated on a set of challenging Bayesian inverse problems, and numerical experiments demonstrate a clear improvement in performance and applicability of standard SVGD. (C) 2021 Elsevier B.V. All rights reserved.
KeywordStein variational gradient decent Bayesian inference Deep learning Local approximation
Indexed BySCI
Funding ProjectNSF of China[11822111] ; NSF of China[11688101] ; NSF of China[11731006] ; NSF of China[11771081] ; science challenge project, China[TZ2018001] ; Zhishan Young Scholar Program of SEU, China ; National Key R&D Program of China[2020YFA0712000] ; science challenge project[TZ2018001] ; Strategic Priority Research Program of Chinese Academy of Sciences[XDA25000404] ; youth innovation promotion association (CAS), China
WOS Research AreaEngineering ; Mathematics ; Mechanics
WOS SubjectEngineering, Multidisciplinary ; Mathematics, Interdisciplinary Applications ; Mechanics
WOS IDWOS:000703985200009
Citation statistics
Document Type期刊论文
Corresponding AuthorYan, Liang
Affiliation1.Southeast Univ, Sch Math, Nanjing 210096, Peoples R China
2.Chinese Acad Sci, LSEC, Inst Computat Math, Acad Math & Syst Sci, Beijing 100190, Peoples R China
Recommended Citation
GB/T 7714
Yan, Liang,Zhou, Tao. Stein variational gradient descent with local approximations[J]. COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING,2021,386:20.
APA Yan, Liang,&Zhou, Tao.(2021).Stein variational gradient descent with local approximations.COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING,386,20.
MLA Yan, Liang,et al."Stein variational gradient descent with local approximations".COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING 386(2021):20.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yan, Liang]'s Articles
[Zhou, Tao]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yan, Liang]'s Articles
[Zhou, Tao]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yan, Liang]'s Articles
[Zhou, Tao]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.