Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization
Yu Lijia; Gao Xiao-Shan1
AbstractIn this paper, the L-2,L-infinity normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network (DNN) with Relu as activation functions. It is shown that the L-2,L-infinity normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions, which reduces over-fitting of the DNN. A global measure is proposed for the robustness of a classification DNN, which is the average radius of the maximal robust spheres with the training samples as centers. A lower bound for the robustness measure in terms of the L-2,L-infinity norm is given. Finally, an upper bound for the Rademacher complexity of DNNs with L-2,L-infinity normalization is given. An algorithm is given to train DNNs with the L-2,L-infinity normalization and numerical experimental results are used to show that the L-2,L-infinity normalization is effective in terms of improving the robustness and accuracy.
KeywordDeep neural network global robustness measure L-2,L-infinity normalization over-fitting Rademacher complexity smooth DNN
Indexed BySCI
Funding ProjectNKRDP[2018YFA0704705] ; National Natural Science Foundation of China[12288201]
WOS Research AreaMathematics
WOS SubjectMathematics, Interdisciplinary Applications
WOS IDWOS:000854858300001
Citation statistics
Document Type期刊论文
Corresponding AuthorGao Xiao-Shan
Affiliation1.Chinese Acad Sci, Acad Math & Syst Sci, Beijing 100190, Peoples R China
2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
Recommended Citation
GB/T 7714
Yu Lijia,Gao Xiao-Shan. Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization[J]. JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,2022:26.
APA Yu Lijia,&Gao Xiao-Shan.(2022).Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization.JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,26.
MLA Yu Lijia,et al."Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization".JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY (2022):26.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yu Lijia]'s Articles
[Gao Xiao-Shan]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yu Lijia]'s Articles
[Gao Xiao-Shan]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yu Lijia]'s Articles
[Gao Xiao-Shan]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.