修辉, 薛丽霞, 汪荣贵, 杨娟. 基于多源信息的全局滤波器剪枝[J]. 微电子学与计算机, 2022, 39(9): 1-10. DOI: 10.19304/J.ISSN1000-7180.2022.0163
引用本文: 修辉, 薛丽霞, 汪荣贵, 杨娟. 基于多源信息的全局滤波器剪枝[J]. 微电子学与计算机, 2022, 39(9): 1-10. DOI: 10.19304/J.ISSN1000-7180.2022.0163
XIU Hui, XUE Lixia, WANG Ronggui, YANG Juan. Global filter pruning based on multi-source information[J]. Microelectronics & Computer, 2022, 39(9): 1-10. DOI: 10.19304/J.ISSN1000-7180.2022.0163
Citation: XIU Hui, XUE Lixia, WANG Ronggui, YANG Juan. Global filter pruning based on multi-source information[J]. Microelectronics & Computer, 2022, 39(9): 1-10. DOI: 10.19304/J.ISSN1000-7180.2022.0163

基于多源信息的全局滤波器剪枝

Global filter pruning based on multi-source information

  • 摘要: 针对现有神经网络剪枝方法未全面评估滤波器的重要性以及跨层滤波器的重要性间存在一定差异的问题,提出了一种基于多源信息的全局滤波器剪枝算法,建立了特征和权重信息间的连接.首先,根据特征信息较为丰富和权重信息受数据噪音影响低的特点,分别以特征间相关性和权重熵来评估滤波器的相对和绝对重要性.然后,将每层中不同压缩比例的滤波器看作一个整体,评估其对模型的全局重要性,按照压缩需求跨层剪掉模型中最不重要的部分.最后,采用知识蒸馏的方式来恢复剪枝后模型的精度,不依赖其他数据集就能完成模型的压缩与微调.为了验证所提方法的适用性,针对DeepLabV3、DABNet和U-Net网络在三个语义分割数据集上进行了大量的实验.也针对多种深度的ResNet网络在图像分类数据集上进行了验证.实验结果表明,通过多源信息可以更精确的评估单层中滤波器的重要性,通过全局重要性来指导跨层剪枝可以使模型的关键信息损失降到最低.

     

    Abstract: To solve the problem that the existing neural network pruning methods do not fully evaluate the importance of filters and there are some differences in the importance of cross-layer filters, a global filter pruning algorithm based on multi-source information is proposed, which establishes the connection between features and weights. Firstly, the relative and absolute importance of the filter are evaluated by the correlation between features and the entropy of weights, respectively, according to the characteristics of rich feature information and low influence of weight information on data noise. Then, the filters with different compression ratios in each layer are considered as a whole, their global importance to the model is evaluated, and the least important parts of the model are cut across layers according to the compression requirements. Finally, knowledge distillation is used to restore the accuracy of the model after pruning, and the model can be compressed and fine-tuned independently of other datasets. To verify the applicability of the proposed method, a large number of experiments are carried out on three semantic segmentation datasets for DeepLabV3, DABNet and U-Net networks. Verification is also carried out on the image classification dataset for various depths of ResNet networks. The experimental results show that the importance of filters in a single layer can be evaluated more accurately by using multi-source information, and the loss of key information can be minimized by using global importance to guide cross-layer pruning.

     

/

返回文章
返回