A comparative study of TF*IDF, LSI and multi-words for text classification

doi:10.1016/j.eswa.2010.08.066

CSpace > 系统科学研究所

	*A comparative study of TFIDF, LSI and multi-words for text classification**
	Zhang, Wen 1; Yoshida, Taketoshi 2; Tang, Xijin3
	2011-03-01
发表期刊	EXPERT SYSTEMS WITH APPLICATIONS
ISSN	0957-4174
卷号	38 期号:3 页码:2758-2765
摘要	One of the main themes in text mining is text representation, which is fundamental and indispensable for text-based intellegent information processing. Generally, text representation inludes two tasks: indexing and weighting. This paper has comparatively studied TF*IDF, LSI and multi-word for text representation. We used a Chinese and an English document collection to respectively evaluate the three methods in information retreival and text categorization. Experimental results have demonstrated that in text categorization, LSI has better performance than other methods in both document collections. Also, LSI has produced the best performance in retrieving English documents. This outcome has shown that LSI has both favorable semantic and statistical quality and is different with the claim that LSI can not produce discriminative power for indexing. (C) 2010 Elsevier Ltd. All rights reserved.
关键词	Text representation TF*IDF LSI Multi-word Text classification Information retrieval Text categorization
DOI	10.1016/j.eswa.2010.08.066
语种	英语
资助项目	National Natural Science Foundation of China[90718042] ; National Natural Science Foundation of China[60873072] ; National Natural Science Foundation of China[60803023] ; National Hi-Tech RD Plan of China[2007AA010303] ; National Hi-Tech RD Plan of China[2007AA01Z179] ; National Basic Research Program[2007CB310802] ; Foundation of Young Doctors of Institute of Software, Chinese Academy of Sciences[ISCAS2009-DR03]
WOS研究方向	Computer Science ; Engineering ; Operations Research & Management Science
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Operations Research & Management Science
WOS记录号	WOS:000284863200158
出版者	PERGAMON-ELSEVIER SCIENCE LTD
引用统计
文献类型	期刊论文
条目标识符	http://ir.amss.ac.cn/handle/2S8OKBNM/13040
专题	系统科学研究所
通讯作者	Zhang, Wen
作者单位	1.Chinese Acad Sci, Inst Software, Lab Internet Software Technol, Beijing 100190, Peoples R China 2.Japan Adv Inst Sci & Technol, Sch Knowledge Sci, Nomi, Ishikawa 9231292, Japan 3.Chinese Acad Sci, Acad Math & Syst Sci, Inst Syst Sci, Beijing 100190, Peoples R China
推荐引用方式 GB/T 7714	Zhang, Wen,Yoshida, Taketoshi,Tang, Xijin. A comparative study of TF*IDF, LSI and multi-words for text classification[J]. EXPERT SYSTEMS WITH APPLICATIONS,2011,38(3):2758-2765.
APA	Zhang, Wen,Yoshida, Taketoshi,&Tang, Xijin.(2011).A comparative study of TF*IDF, LSI and multi-words for text classification.EXPERT SYSTEMS WITH APPLICATIONS,38(3),2758-2765.
MLA	Zhang, Wen,et al."A comparative study of TF*IDF, LSI and multi-words for text classification".EXPERT SYSTEMS WITH APPLICATIONS 38.3(2011):2758-2765.