A review of data quantization of deep neural network online training hardware accelerator

With the explosive growth of algorithms and data, the Deep Neural Network (DNN) gradually plays an increasingly important role in practical applications. However, it is difficult for the real scene and offline training data to meet the assumption of independent and identical distribution, resulting in a serious decline in the performance of the pre-training DNN model in practical applications. Therefore, online training of DNN model on the platform with relatively limited resources becomes the guarantee of its effective application. Hence, it is necessary to significantly reduce the computational complexity while ensuring the accuracy. Data quantization is one of the mainstream optimization technologies to reduce computational complexity. Accordingly, we summarize the researches on data quantization of DNN online training accelerator. Firstly, data quantization based on direct fixed-point representation and complex mapping are summarized from the perspective of software. Secondly, the quantization of DNN accelerator for each training step is summarized from the perspective of hardware. Then, the influence of data quantization on accelerator design is described, including memory unit and processing unit. Finally, the researches in this field are summarized and the future development directions of this field are prospected. The classification method proposed in this paper is helpful to classify the previous work of DNN accelerator in data quantization.