Abstract: The application of the improved Hidden Markov Models to text categorization should take the backward dependency as well as forward dependency on states into account.The accuracy of information extraction could be improved by applying the dependency of the current observation value and state on the backward ones into the learning of models.This paper is to preprocess the training samples in text categorization process, to learn the parameters of HMM classifier, establishing one and evaluating its performance through testing set.The improved evaluation criteria could give a fair evaluation of different dataset, make a comparison of different classifiers on the performance of the same dataset and feed back the classifier to improve.