李兆兴, 马自堂. 一种多层次Hadoop平台设计[J]. 微电子学与计算机, 2016, 33(2): 27-30.
引用本文: 李兆兴, 马自堂. 一种多层次Hadoop平台设计[J]. 微电子学与计算机, 2016, 33(2): 27-30.
LI Zhao-xing, MA Zi-tang. A Multi-Layer Hadoop Platform[J]. Microelectronics & Computer, 2016, 33(2): 27-30.
Citation: LI Zhao-xing, MA Zi-tang. A Multi-Layer Hadoop Platform[J]. Microelectronics & Computer, 2016, 33(2): 27-30.

一种多层次Hadoop平台设计

A Multi-Layer Hadoop Platform

  • 摘要: 大数据和大数据中心规模的进一步扩大为Hadoop平台的可扩展性提出了新的要求.在系统地分析了制约Hadoop平台性能和可扩展性因素的基础上, 提出了一种多层次Hadoop平台.通过将Hadoop平台划分为若干区域, 形成全局-区域-节点的多层次结构.由Master节点负责系统级的元数据管理和任务分发, 区域管理节点负责区域级的元数据管理和任务分发, 从而提高Hadoop平台可扩展性和容错性.实验表明, 该平台可有效提高Hadoop的可扩展性, 具有一定的可行性.

     

    Abstract: With the further expansion of Big Data and Big Data Centre, new extensibility requirements are proposed on the Hadoop Platform. In the base of systematically analysis constraints on performance and extensibility of Hadoop Platform, this paper presents a multi-level Hadoop Platform which has a structure of Global-Region-Node after being divided. In this platform, Master Node is responsible for metadata management and task development on the global level, and regional management node takes charge of metadata management and task development on the regional level, which improve both scalability and fault tolerance of Hadoop Platform. Experiments show that this platform can effectively promote the scalability of Hadoop and has certain feasibility.

     

/

返回文章
返回