Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization

doi:10.1007/s11424-022-1326-y

CSpace

	Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization
	Yu Lijia ; Gao Xiao-Shan 1
	2022-09-15
发表期刊	JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY
ISSN	1009-6124
页码	26
摘要	In this paper, the L-2,L-infinity normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network (DNN) with Relu as activation functions. It is shown that the L-2,L-infinity normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions, which reduces over-fitting of the DNN. A global measure is proposed for the robustness of a classification DNN, which is the average radius of the maximal robust spheres with the training samples as centers. A lower bound for the robustness measure in terms of the L-2,L-infinity norm is given. Finally, an upper bound for the Rademacher complexity of DNNs with L-2,L-infinity normalization is given. An algorithm is given to train DNNs with the L-2,L-infinity normalization and numerical experimental results are used to show that the L-2,L-infinity normalization is effective in terms of improving the robustness and accuracy.
关键词	Deep neural network global robustness measure L-2,L-infinity normalization over-fitting Rademacher complexity smooth DNN
DOI	10.1007/s11424-022-1326-y
收录类别	SCI
语种	英语
资助项目	NKRDP[2018YFA0704705] ; National Natural Science Foundation of China[12288201]
WOS研究方向	Mathematics
WOS类目	Mathematics, Interdisciplinary Applications
WOS记录号	WOS:000854858300001
出版者	SPRINGER HEIDELBERG
引用统计
文献类型	期刊论文
条目标识符	http://ir.amss.ac.cn/handle/2S8OKBNM/61005
专题	中国科学院数学与系统科学研究院
通讯作者	Gao Xiao-Shan
作者单位	1.Chinese Acad Sci, Acad Math & Syst Sci, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Yu Lijia,Gao Xiao-Shan. Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization[J]. JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,2022:26.
APA	Yu Lijia,&Gao Xiao-Shan.(2022).Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization.JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,26.
MLA	Yu Lijia,et al."Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization".JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY (2022):26.