Reinforcement learning behaviors in sponsored search

doi:10.1002/asmb.2157

CSpace

	Reinforcement learning behaviors in sponsored search
	Chen, Wei 1; Liu, Tie-Yan 1; Yang, Xinxin 2
	2016-05-01
发表期刊	APPLIED STOCHASTIC MODELS IN BUSINESS AND INDUSTRY
ISSN	1524-1904
卷号	32 期号:3 页码:358-367
摘要	This paper is concerned with the modeling of advertiser behaviors in sponsored search. Modeling advertiser behaviors can help search engines better serve advertisers, improve auction mechanism, and forecast future revenue. Previous works on this topic either unrealistically assume advertisers to be able to perceive the states of the sponsored search system and the private information of other advertisers or ignore the differences in advertisers' abilities to optimize their bid strategies. To tackle the problems, we propose viewing sponsored search auctions as partially observable multi-agent system with private information. Then, we employ a reinforcement learning behavior model to describe how each advertiser responds to this multi-agent system. The proposed model no longer assumes advertisers to have perfect information access, but instead assumes them to optimize their strategies only based on the partially observed states in the auctions. Furthermore, the model does not specify how the optimization is conducted, but instead uses parameters learned from data to describe different advertisers' abilities in obtaining the optimal strategies. Our experiments on real sponsored search data demonstrate that the proposed model outperforms previous models in predicting the bids and rank positions of the advertisers in the near future. In addition to the accurate prediction of these short-term behaviors, our study shows another nice property of the proposed model. That is, if all the advertisers behave according to the model, the multi-agent system of sponsored search will converge to a locally envy-free equilibrium, under certain conditions. This result establishes a connection between machine-learned behavior models and game-theoretic properties of the system. Copyright (c) 2016 John Wiley & Sons, Ltd.
关键词	advertiser behavior sponsored search generalized second-price auction locally envy-free equilibrium
DOI	10.1002/asmb.2157
语种	英语
WOS研究方向	Operations Research & Management Science ; Mathematics
WOS类目	Operations Research & Management Science ; Mathematics, Interdisciplinary Applications ; Statistics & Probability
WOS记录号	WOS:000379022400010
出版者	WILEY-BLACKWELL
引用统计
文献类型	期刊论文
条目标识符	http://ir.amss.ac.cn/handle/2S8OKBNM/23084
专题	中国科学院数学与系统科学研究院
通讯作者	Chen, Wei
作者单位	1.Microsoft Res, Beijing, Peoples R China 2.Chinese Acad Sci, Acad Math & Syst Sci, Beijing, Peoples R China
推荐引用方式 GB/T 7714	Chen, Wei,Liu, Tie-Yan,Yang, Xinxin. Reinforcement learning behaviors in sponsored search[J]. APPLIED STOCHASTIC MODELS IN BUSINESS AND INDUSTRY,2016,32(3):358-367.
APA	Chen, Wei,Liu, Tie-Yan,&Yang, Xinxin.(2016).Reinforcement learning behaviors in sponsored search.APPLIED STOCHASTIC MODELS IN BUSINESS AND INDUSTRY,32(3),358-367.
MLA	Chen, Wei,et al."Reinforcement learning behaviors in sponsored search".APPLIED STOCHASTIC MODELS IN BUSINESS AND INDUSTRY 32.3(2016):358-367.