摘要
Pruning neural networks presents a promising avenue for compressing and accelerating contemporary deep convolutional networks. Recently, researchers have increasingly focused on a new kind of stripe-wise pruning (SWP) methods, which offers finer granularity compared with the traditional techniques. However, most prior network pruning methods, including SWP, suffer from the issue of wrong pruning. During model training, particularly in its early stages, pruning decisions are often sub-optimal, leading to the premature elimination of crucial weights. To mitigate this problem, we propose a novel framework that re-utilizes knowledge from pruned stripe-wise weights for SWP. The unique stripe repair phases are inserted into the standard training process to identify and repair those mis-pruned stripes. Additionally, halfway-pruned models with varying structures are also selectively reused in a self-knowledge distillation manner. Comprehensive experiments conducted on three public datasets demonstrate the effectiveness of our proposed model.
