基于条件随机场的汉语词性标注

Chinese Part-of-speech Tagging Based on Conditional Random Fields

摘要: 近年来条件随机场广泛应用于各类序列数据标注中, 汉语词性标注中应用条件随机场对上下文建模时会扩展出数以亿计的特征, 在深入分析特征产生机理的基础上对特征模板集进行了优化, 采用条件随机场进一步研究了汉语词性标注中设定的特征模板集、扩展出的特征数、训练后模型大小、词性标注精度等指标之间的关系.实验结果表明, 优化后的特征模板集在模型训练时间、训练后模型大小、标注精度等指标上达到了整体最优.

Abstract: In recent years, conditional random fields is widely used in various types of sequence data labeling, hundreds of millions of features will be extended out in the context modeling using CRFs for Chinese part of speech tagging, feature template set is optimized after in-depth analysis of the context features.We further studied the relations of the feature template set and the training model size, tagging accuracy for Chinese part-of-speech tagging via using maximum entropy model.Experimental results show that optimized feature set of templates is the overall optimum.