Global filter pruning based on multi-source information

To solve the problem that the existing neural network pruning methods do not fully evaluate the importance of filters and there are some differences in the importance of cross-layer filters, a global filter pruning algorithm based on multi-source information is proposed, which establishes the connection between features and weights. Firstly, the relative and absolute importance of the filter are evaluated by the correlation between features and the entropy of weights, respectively, according to the characteristics of rich feature information and low influence of weight information on data noise. Then, the filters with different compression ratios in each layer are considered as a whole, their global importance to the model is evaluated, and the least important parts of the model are cut across layers according to the compression requirements. Finally, knowledge distillation is used to restore the accuracy of the model after pruning, and the model can be compressed and fine-tuned independently of other datasets. To verify the applicability of the proposed method, a large number of experiments are carried out on three semantic segmentation datasets for DeepLabV3, DABNet and U-Net networks. Verification is also carried out on the image classification dataset for various depths of ResNet networks. The experimental results show that the importance of filters in a single layer can be evaluated more accurately by using multi-source information, and the loss of key information can be minimized by using global importance to guide cross-layer pruning.