Channel Pruning for Accelerating Very Deep Neural Networks

GitHub - yihui-he/channel-pruning: Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)

You can't perform that action at this time. You signed in with another tab or window. You signed out in another tab or window. Reload to refresh your session. Reload to refresh your session.

https://github.com/yihui-he/channel-pruning

Channel Pruning for Accelerating Very Deep Neural Networks

In this paper, we introduce a new channel pruning method to accelerate very deep convolutional neural networks.Given a trained CNN model, we propose an iterative two-step algorithm to effectively prune each layer, by a LASSO regression based channel selection and least square reconstruction. We further generalize this algorithm to multi-layer and multi-branch cases.

https://arxiv.org/abs/1707.06168

• AMC: AutoML for Model Compression and Acceleration on Mobile Devices ECCV’18, which combines channel pruning and reinforcement learning to further accelerate CNN. code and models are available!

• AddressNet: Shift-Based Primitives for Efficient Convolutional Neural Networks WACV’19. We propose a family of efficient networks based on Shift operation.

• MoBiNet: A Mobile Binary Network for Image Classification WACV’20 Binarized MobileNets.

Citation

@InProceedings{He_2017_ICCV,
author = {He, Yihui and Zhang, Xiangyu and Sun, Jian},
title = {Channel Pruning for Accelerating Very Deep Neural Networks},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {Oct},
year = {2017}
}

requirements

1. Python3 packages you might not have: scipy, sklearn, easydict, use sudo pip3 install to install.

2. For finetuning with 128 batch size, 4 GPUs (~11G of memory)

Installation (sufficient for the demo)

1. Clone the repository

# Make sure to clone with --recursive
 git clone --recursive https://github.com/yihui-he/channel-pruning.git

2. Build my Caffe fork (which support bicubic interpolation and resizing image shorter side to 256 then crop to 224x224)

cd caffe

 # If you're experienced with Caffe and have all of the requirements installed, then simply do:
 make all -j8 && make pycaffe
 # Or follow the Caffe installation instructions here:
 # http://caffe.berkeleyvision.org/installation.html

 # you might need to add pycaffe to PYTHONPATH, if you've already had a caffe before

3. Download ImageNet classification dataset http://www.image-net.org/download-images

4. Specify imagenet source path in temp/vgg.prototxt (line 12 and 36)

Channel Pruning

1. Start Channel Pruning

python3 train.py -action c3 -caffe [GPU0]
 # or log it with ./run.sh python3 train.py -action c3 -caffe [GPU0]
 # replace [GPU0] with actual GPU device like 0,1 or 2

2. Combine some factorized layers for further compression, and calculate the acceleration ratio. Replace the ImageData layer of temp/cb_3c_3C4x_mem_bn_vgg.prototxt with temp/vgg.prototxt’s Shell ./combine.sh | xargs ./calflop.sh

3. Finetuning

caffe train -solver temp/solver.prototxt -weights temp/cb_3c_vgg.caffemodel -gpu [GPU0,GPU1,GPU2,GPU3]
 # replace [GPU0,GPU1,GPU2,GPU3] with actual GPU device like 0,1,2,3

4. Testing
Though testing is done while finetuning, you can test anytime with:

caffe test -model path/to/prototxt -weights path/to/caffemodel -iterations 5000 -gpu [GPU0]
 # replace [GPU0] with actual GPU device like 0,1 or 2

Pruned models (for download)
For fast testing, you can directly download pruned model from release: VGG-16 3C 4X, VGG-16 5X, ResNet-50 2X. Or follow Baidu Yun Download link

caffe test -model channel_pruning_VGG-16_3C4x.prototxt -weights channel_pruning_VGG-16_3C4x.caffemodel -iterations 5000 -gpu [GPU0]
# replace [GPU0] with actual GPU device like 0,1 or 2

Pruning faster RCNN

FAQ