This publication offers a hybrid network compression technique for exploiting the prior knowledge of network parameters by Gaussian scale mixture (GSM) models.
In this paper, the authors propose a hybrid network compression technique for exploiting the prior knowledge of network parameters by Gaussian scale mixture (GSM) models. Specifically, the collection of network parameters are characterized by GSM models and network pruning is formulated as a maximum a posteriori (MAP) estimation problem with a sparsity prior. The key novel insight brought by this work is that groups of parameters associated with the same channel are similar, which is analogous to the grouping of similar patches in natural images. Such observation inspires us to leverage powerful structured sparsity prior from image restoration to network compression - i.e., to develop a flexible filter-grouping strategy that not only promotes structured sparsity but also can be seamlessly integrated with the existing network pruning framework. Extensive experimental results on several popular DCNN models including VGGNet, ResNet and DenseNet have shown that the proposed GSM-based joint grouping and pruning method convincingly outperforms other competing approaches (including both pruning and non-pruning based methods). (Published Abstract Provided)
Downloads
Similar Publications
- The Role of Simulated Data in Making the Best Predictions (from the 87th Annual Meeting of the American Association of Physical Anthropologists - 2018)
- The Effect of Ultraviolet Radiation on Dyed Man-Made Textile Fibers Using UV-Vis Microspectrophotometry (MSP): Technical Aspects on Spectral Alterations in Time
- Detecting Various Burial Scenarios In A Controlled Setting Using Ground-penetrating Radar And Conductivity