October 1, 2018
Visiting speaker Alex Hernández-García will give a talk on “Data augmentation as a biologically plausible alternative to explicit regularization in CNNs” on Tuesday 9/10.
Abstract
The impressive success of modern deep neural networks on computer vision tasks has been achieved through models of very large capacity compared to the number of available training examples. This over-parameterization is often said to be controlled by means of different regularization techniques, mainly weight decay and dropout. However, since these techniques reduce the effective capacity of the model, typically even deeper and wider architectures are required to compensate for the reduced capacity. Therefore, there seems to be a waste of capacity in this practice. In contrast, data augmentation techniques do not reduce the effective capacity and improve generalization by increasing the number of training examples. This talk will present the results of an ablation study on some popular architectures that conclude that data augmentation alone—without any other explicit regularization techniques—can achieve the same performance or higher than regularized models, especially when training with fewer examples, and exhibits much higher adaptability to changes in the architecture. Besides, a recent study suggests that models trained with heavier data augmentation exhibit more similarity with the human inferior temporal (IT) cortex.