Abstract: In order to ensure the security of data transmission in wireless and wired heterogeneous networks, a secure data transmission mechanism based on optimized AES-GCM authentication encryption algorithm and SHA-based digital signature algorithm is adopted. This mechanism includes three parts:key generation and management, two-way identity authentication of communicators and efficient and secure data transmission. The key is generated by parameters related to random number and timestamp, and the key sharing among communication entities is realized by using improved ECDH algorithm to prevent replay attacks and ensure key security. It reduces the number of encryption rounds of traditional AES algorithm to ensure the efficiency of message transmission, in order to ensure the security of message transmission, CTR block encryption structure is proposed, and uses the physical address, random number and counter value of communicators as the initialized value to generate the key stream of the optimized AES-GCM authentication encryption algorithm. The communication experiment in heterogeneous networks shows that the secure data transmission mechanism proposed in this paper significantly shortens the encrypted transmission time and guarantees the security and efficiency.