KMS Of Academy of mathematics and systems sciences, CAS
Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization | |
Yu Lijia; Gao Xiao-Shan1 | |
2022-09-15 | |
发表期刊 | JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY |
ISSN | 1009-6124 |
页码 | 26 |
摘要 | In this paper, the L-2,L-infinity normalization of the weight matrices is used to enhance the robustness and accuracy of the deep neural network (DNN) with Relu as activation functions. It is shown that the L-2,L-infinity normalization leads to large dihedral angles between two adjacent faces of the DNN function graph and hence smoother DNN functions, which reduces over-fitting of the DNN. A global measure is proposed for the robustness of a classification DNN, which is the average radius of the maximal robust spheres with the training samples as centers. A lower bound for the robustness measure in terms of the L-2,L-infinity norm is given. Finally, an upper bound for the Rademacher complexity of DNNs with L-2,L-infinity normalization is given. An algorithm is given to train DNNs with the L-2,L-infinity normalization and numerical experimental results are used to show that the L-2,L-infinity normalization is effective in terms of improving the robustness and accuracy. |
关键词 | Deep neural network global robustness measure L-2,L-infinity normalization over-fitting Rademacher complexity smooth DNN |
DOI | 10.1007/s11424-022-1326-y |
收录类别 | SCI |
语种 | 英语 |
资助项目 | NKRDP[2018YFA0704705] ; National Natural Science Foundation of China[12288201] |
WOS研究方向 | Mathematics |
WOS类目 | Mathematics, Interdisciplinary Applications |
WOS记录号 | WOS:000854858300001 |
出版者 | SPRINGER HEIDELBERG |
引用统计 | |
文献类型 | 期刊论文 |
条目标识符 | http://ir.amss.ac.cn/handle/2S8OKBNM/61005 |
专题 | 中国科学院数学与系统科学研究院 |
通讯作者 | Gao Xiao-Shan |
作者单位 | 1.Chinese Acad Sci, Acad Math & Syst Sci, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China |
推荐引用方式 GB/T 7714 | Yu Lijia,Gao Xiao-Shan. Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization[J]. JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,2022:26. |
APA | Yu Lijia,&Gao Xiao-Shan.(2022).Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization.JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY,26. |
MLA | Yu Lijia,et al."Improve Robustness and Accuracy of Deep Neural Network with L-2,L-infinity Normalization".JOURNAL OF SYSTEMS SCIENCE & COMPLEXITY (2022):26. |
条目包含的文件 | 条目无相关文件。 |
个性服务 |
推荐该条目 |
保存到收藏夹 |
查看访问统计 |
导出为Endnote文件 |
谷歌学术 |
谷歌学术中相似的文章 |
[Yu Lijia]的文章 |
[Gao Xiao-Shan]的文章 |
百度学术 |
百度学术中相似的文章 |
[Yu Lijia]的文章 |
[Gao Xiao-Shan]的文章 |
必应学术 |
必应学术中相似的文章 |
[Yu Lijia]的文章 |
[Gao Xiao-Shan]的文章 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论