MIAO Y H,FAN Z L,ZHANG M D,et al. A data balanced distribution algorithm based on Ceph storage[J]. Microelectronics & Computer,2024,41(3):90-97. doi: 10.19304/J.ISSN1000-7180.2023.0180
Citation: MIAO Y H,FAN Z L,ZHANG M D,et al. A data balanced distribution algorithm based on Ceph storage[J]. Microelectronics & Computer,2024,41(3):90-97. doi: 10.19304/J.ISSN1000-7180.2023.0180

A data balanced distribution algorithm based on Ceph storage

  • The Controlled Replication Under Scalable Hashing(CRUSH) data distribution algorithm in Ceph distributed storage system causes the difference of storage data capacity between devices to reach 40%, and the so-called "hot spot" becomes the bottleneck of system performance in the case of large data volume and high concurrency. In this paper, CRUSH algorithm is deeply studied, and Writing is designed and implemented Writing_Balance algorithm is used to optimize the performance of data distribution to eliminate the load imbalance caused by "hot spotst" and the high disk utilization. Writing_Balance algorithm is found through experiments ,which can optimize the PG quantity distribution of "hot spotst" to 4.4% compared with storage system that do not use Writing_Balance algorithm. The stability of disk utilization has been improved by about 3% and the overall data balance optimization has also been significantly improved in a small input key space.
  • loading

Catalog

    Turn off MathJax
    Article Contents

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return