Interpreting the Basis Path Set in Neural Networks

doi:10.1007/s11424-020-0112-y

CSpace > 应用数学研究所

	Interpreting the Basis Path Set in Neural Networks
	Zhu Juanping 1; Meng Qi 2; Chen Wei 2; Ma Zhiming3
	2021-01-12
发表期刊	JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY
ISSN	1009-6124
页码	13
摘要	The G-SGD algorithm significantly outperforms the conventional SGD algorithm in ReLU neural networks by adopting the basis path set. However, how the inner mechanism of basis paths works remains mysterious, and the G-SGD algorithm that helps to find a basis path set is heuristic. This paper employs graph theory to investigate structure properties of basis paths in a more general and complicated neural network with unbalanced layers and edge-skipping. The hierarchical Algorithm HBPS is proposed to find a basis path set, by decomposing the complicated network into several independent and parallel substructures. The paper theoretically extends the study of basis paths and provides one methodology to find the basis path set in a more general neural network.
关键词	Basis path hierarchical algorithm independent path neural network substructure path
DOI	10.1007/s11424-020-0112-y
收录类别	SCI
语种	英语
资助项目	National Nature Science Foundation of China[11601471] ; Project for Innovation Team (Cultivation) of Yunnan Province[202005AE160006] ; Key Project of Yunnan Provincial Science and Technology Department and Yunnan University[2018FY001014]
WOS研究方向	Mathematics
WOS类目	Mathematics, Interdisciplinary Applications
WOS记录号	WOS:000608140200033
出版者	SPRINGER HEIDELBERG
引用统计
文献类型	期刊论文
条目标识符	http://ir.amss.ac.cn/handle/2S8OKBNM/58044
专题	应用数学研究所
通讯作者	Zhu Juanping
作者单位	1.Yunnan Univ, Dept Math, Kunming 650500, Yunnan, Peoples R China 2.Microsoft Res Asia, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Acad Math & Syst Sci, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Zhu Juanping,Meng Qi,Chen Wei,et al. Interpreting the Basis Path Set in Neural Networks[J]. JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,2021:13.
APA	Zhu Juanping,Meng Qi,Chen Wei,&Ma Zhiming.(2021).Interpreting the Basis Path Set in Neural Networks.JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,13.
MLA	Zhu Juanping,et al."Interpreting the Basis Path Set in Neural Networks".JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY (2021):13.