Spiking timing dependent plasticity algorithm with mixed reward-modulated signals

In recent years, Spiking Timing-Dependent Plasticity (STDP) rules with physiological basis have been applied more and more in spiking neural networks. The R-STDP (reward-modulated STDP) learning algorithm combining STDP with the reinforcement learning reward modulation embraces great effect on improving the performance of SNN. However, the feedback only reflects on the last layer of spiking deep convolutional neural networks as the R-STDP algorithm works, which means the middle layer cannot get feedback. Inspired by the unsupervised characteristics of the Auto-Encoder, a mix reward-modulatedSTDP (MR-STDP) algorithm with mixed reward/punishment signal was proposed. In this algorithm, the reconstruction layer was added to the middle layer to establish the rewards/punishment signal factor model. The guiding factor signal is the similarity measure of spiking sequences issued by the neurons at the same position of the input layer of the interlayer autoencoder and the reconstruction layer, and it is combined with R-STDP, so that the middle layer can obtain the weight guiding signal. Experiments on MNIST and COVID-19 CT data sets shows that the proposed method achieves higher accuracy than R-STDP, and the efficiency of learning in middle layer is greatly improved.