Abstract: A multi-focus image fusion algorithm is given out, which is based on adaptive sparse representation. According to structure characteristics of image, the algorithm can divide sub-blocks into three models, which are the similar model, the smouth model and the detailed model. The three models get different process mode. The similar model is put into the fusion image directly. The smouth model uses the weighted average method and the detacled model uses sparse representation to realize fusion, so the quantity of blocks for sparse coding is reduced and the efficiecy is improved. The experiment results show that the way can shorten the operation time effectively, at the same time if can guarantee that the subjective effect and objective performance indexes of fused image are both optimal.