基于KVM虚拟化技术的Hadoop架构

Hadoop Architecture Based-on Virtual Technology of KVM

摘要: 本文提出并实现了一种Hadoop与虚拟化技术相结合的模型.该模型将多核计算机虚拟成多节点集群,最大限度地利用计算资源.在实验测试的基础上,通过分析任务在Hadoop环境中的并行机制,弄清计算任务在节点中与核中的分配方式,以达到提高并行效率、降低硬件开销的目的,最后通过对典型应用问题进行计算,从而对本文所提出模型的正确性和有效性进行了验证.

Abstract: We propose a model which combines the Hadoop with the virtualization technology.In the model,each processing core of multi-cores computer is virtualized as a computing node,then the use of computing resources is maximized.Moreover,parallel working mechanism of the Hadoop is discussed.Experiments are fulfilled by typical applications finally.