Adversarial Information Bottleneck

doi:10.1109/TNNLS.2022.3172986

CSpace > 应用数学研究所

	Adversarial Information Bottleneck
	Zhai, Penglong1,2 ; Zhang, Shihua1,2
	2022-05-20
发表期刊	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
ISSN	2162-237X
页码	10
摘要	The information bottleneck (IB) principle has been adopted to explain deep learning in terms of information compression and prediction, which are balanced by a tradeoff hyperparameter. How to optimize the IB principle for better robustness and figure out the effects of compression through the tradeoff hyperparameter are two challenging problems. Previous methods attempted to optimize the IB principle by introducing random noise into learning the representation and achieved the state-of-the-art performance in the nuisance information compression and semantic information extraction. However, their performance on resisting adversarial perturbations is far less impressive. To this end, we propose an adversarial IB (AIB) method without any explicit assumptions about the underlying distribution of the representations, which can be optimized effectively by solving a min-max optimization problem. Numerical experiments on synthetic and real-world datasets demonstrate its effectiveness on learning more invariant representations and mitigating adversarial perturbations compared to several competing IB methods. In addition, we analyze the adversarial robustness of diverse IB methods contrasting with their IB curves and reveal that IB models with the hyperparameter beta corresponding to the knee point in the IB curve achieve the best tradeoff between compression and prediction and has the best robustness against various attacks.
关键词	Robustness Optimization Mutual information Deep learning Perturbation methods Training Task analysis Adversarial robustness deep learning hyperparameter selection information bottleneck (IB)
DOI	10.1109/TNNLS.2022.3172986
收录类别	SCI
语种	英语
资助项目	National Key Research and Development Program of China[2019YFA0709501] ; National Natural Science Foundation of China[12126605] ; National Natural Science Foundation of China[61621003]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Hardware & Architecture ; Computer Science, Theory & Methods ; Engineering, Electrical & Electronic
WOS记录号	WOS:000800769800001
出版者	IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
引用统计
文献类型	期刊论文
条目标识符	http://ir.amss.ac.cn/handle/2S8OKBNM/61455
专题	应用数学研究所
通讯作者	Zhang, Shihua
作者单位	1.Chinese Acad Sci, Acad Math & Syst Sci, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Sch Math Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Zhai, Penglong,Zhang, Shihua. Adversarial Information Bottleneck[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2022:10.
APA	Zhai, Penglong,&Zhang, Shihua.(2022).Adversarial Information Bottleneck.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,10.
MLA	Zhai, Penglong,et al."Adversarial Information Bottleneck".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2022):10.