|
 
|
[BRT18] SHADE: Information-Based Regularization for Deep Learning
Conférence Internationale avec comité de lecture : IEEE International Conference on Image Processing, October 2018, Athens, Grece, Best paper award,Mots clés: Deep Learning, Regularization, Information Bottleneck
Résumé:
Regularization is a big issue for training deep neural networks. In
this paper, we propose a new information-theory-based regularization
scheme named SHADE for SHAnnon DEcay. The originality of the
approach is to define a prior based on conditional entropy, which
explicitly decouples the learning of invariant representations in the
regularizer and the learning of correlations between inputs and labels
in the data fitting term. Our second contribution is to derive a stochas-
tic version of the regularizer compatible with deep learning, resulting
in a tractable training scheme. We empirically validate the efficiency
of our approach to improve classification performances compared to
standard regularization schemes on several standard architectures.
BibTeX
| @inproceedings { | |||
| BRT18, | |||
| title | = | "{SHADE: Information-Based Regularization for Deep Learning}", | |
| author | = | " M. Blot and T. Robert and N. Thome and M. Cord ", | |
| booktitle | = | "{IEEE International Conference on Image Processing}", | |
| year | = | 2018, | |
| month | = | "October", | |
| address | = | "Athens, Grece, Best paper award", | |
| } | |||
