Abstract: The current ReRAM-based NN acceleratorshave many problems such as high hardwareresource demand and high power consumption. An energy-efficient modelcompression framework consisting of pruning and quantization algorithms is proposed. According to the tightly coupled crossbar structure and unstructured sparsity, a crossbar-aware incrementalstructured pruning algorithm is designedtoachievehigher sparsity and accuracy. A power of two quantizationmethod is designedto reduce ADC resolution requirements and the numberof low resistance states (LRS) ReRAM cells in crossbars to improvethe energy efficiency. Experimental results show thatthe proposed modelcompression framework can achieve 17.2-30.7x energy efficiencyand 4.3-9.3x speedup, compared with ReRAM-based acceleratorsfor dense NN with about 1% accuracy loss