Abstract: In order to speed up the operation of RSA coprocessor, an advanced high radix Montgomery module multiplication and Reft-to-Light scan modular exponentiation algorithm are proposed. The modified Montgomery module multiplication eliminates the carry propagation and decreases the number of iteration. The modified modular exponentiation algorithm eliminates the unwanted effect of the factor R^-1 which is introduced by Montgomery modular multiplication. A high performance scalable public-key cipher RSA coprocessor is designed, which is based on them. The result of the hardware implementation shows that the improved RSA coprocessor is synthesized by CSMC 0.18 μm library, and the total logic resources is 26 kgates, Simulation result shows that it takes an average of 330 ms to complete a 1024 bit encryption at 10 MHz. Compared with previous works, the proposed architecture can achieve better performance in chip area and speed.