• 北大核心期刊(《中文核心期刊要目总览》2017版)
  • 中国科技核心期刊(中国科技论文统计源期刊)
  • JST 日本科学技术振兴机构数据库(日)收录期刊

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于轻量图卷积增强嵌入学习的点击率预测模型

张琳钰 牛存良 姚政

张琳钰, 牛存良, 姚政. 基于轻量图卷积增强嵌入学习的点击率预测模型[J]. 微电子学与计算机, 2022, 39(4): 9-16. doi: 10.19304/J.ISSN1000-7180.2021.1069
引用本文: 张琳钰, 牛存良, 姚政. 基于轻量图卷积增强嵌入学习的点击率预测模型[J]. 微电子学与计算机, 2022, 39(4): 9-16. doi: 10.19304/J.ISSN1000-7180.2021.1069
ZHANG Linyu, NIU Cunliang, YAO Zheng. Enhanced embedding learning for CTR prediction based on LightGCN[J]. Microelectronics & Computer, 2022, 39(4): 9-16. doi: 10.19304/J.ISSN1000-7180.2021.1069
Citation: ZHANG Linyu, NIU Cunliang, YAO Zheng. Enhanced embedding learning for CTR prediction based on LightGCN[J]. Microelectronics & Computer, 2022, 39(4): 9-16. doi: 10.19304/J.ISSN1000-7180.2021.1069

基于轻量图卷积增强嵌入学习的点击率预测模型

doi: 10.19304/J.ISSN1000-7180.2021.1069
基金项目: 

国家自然科学基金 61976242

天津市自然科学基金 19JCZDJC40000

科技部创新方法工作专项项目 2019IM020300

详细信息
    作者简介:

    张琳钰  女,(1998-),硕士研究生.研究方向为推荐系统. E-mail: 202032803106@stu.hebut.edu.cn

    牛存良  男,(1962-),博士,正高级工程师.研究方向为智能信息处理

    姚政  男,(1995-),硕士研究生.研究方向为推荐系统

  • 中图分类号: TP391.9

Enhanced embedding learning for CTR prediction based on LightGCN

  • 摘要:

    基于特征交互建模方法的点击率预测问题经广泛探索已经取得较大进展,它能缓解有效信息损失,但在一定程度上依赖于不同特征的共同出现,存在特征稀疏问题.因此,针对交互过程特征出现次数少不能高效学习特征表示的问题,提出了一个基于轻量图卷积增强嵌入层学习的点击率预测模型LGCDFM(LightGCN with DeepFM).在初始嵌入层采用分而治之的学习策略,提出图结构中区分不同类型节点,首先由同类型节点信息传播确保特征出现频率,再由高阶连通的不同类型节点间交互捕捉多跳邻居信息.轻量图卷积神经结构强大的特征提取和表示学习能力,且摒弃无益于交互的特征变换和非线性激活函数,成为处理简单用户-项目交互数据的协同过滤任务的优势,有效减轻特征稀疏性问题.最后,表示学习层应用点击率预测经典模型DeepFM端到端学习高阶和低阶特征组合,由隐向量从稀疏数据中学习,提升点击率预测任务性能.通过在Criteo、Avazu两个公开数据集上的实验表明,该模型在点击率预测和特征稀疏问题上的性能表现均优于现有方法.

     

  • 图 1  DeepFM架构

    Figure 1.  The architecture of DeepFM

    图 2  LGCDFM架构

    Figure 2.  The architecture of LGCDFM

    表  1  数据集介绍

    Table  1.   Dataset statistics

    数据集 样本数量 特征域数量 特征数量
    Criteo 45 840 617 39 998 960
    Avazu 40 428 967 23 1 544 488
    下载: 导出CSV

    表  2  模型间性能对比

    Table  2.   Performance comparison of models

    Criteo Avazu
    AUC RI-AUC Logloss RI-Logloss AUC RI-AUC Logloss RI-Logloss
    DeepFM 0.801 6 1.21% 0.449 8 1.91% 0.765 3 1.59% 0.385 4 0.83%
    PNN 0.798 3 1.63% 0.453 0 2.60% 0.765 8 1.53% 0.385 6 0.88%
    GIN 0.800 9 1.29% 0.451 7 2.32% 0.775 8 0.22% 0.382 9 0.18%
    FiGNN 0.806 2 0.63% 0.445 3 0.92% 0.776 2 0.17% 0.382 5 0.08%
    LGCDFM 0.811 3 0.00% 0.441 2 0.00% 0.777 5 0.00% 0.382 2 0.00%
    下载: 导出CSV

    表  3  Criteo数据集特征稀疏分析

    Table  3.   Feature sparsity analysis in Criteo

    特征 频率 DeepFM (Logloss) LGCDFM (Logloss)
    F_1 12 0.365 8 0.232 8
    F_2 4 0.321 2 0.3112
    F_3 9 0.623 3 0.569 2
    F_4 10 0.083 2 0.032 6
    下载: 导出CSV
  • [1] Interactive Advertising Bureau. IAB internet advertising revenue report[R]. New York: IAB, 2021.
    [2] RENDLE S. Factorization machines[C]//2010 IEEE International Conference on Data Mining. Sydney, NSW, Australia: IEEE, 2010: 995-1000. DOI: 10.1109/ICDM.2010.127.
    [3] HE X N, CHUA T S. Neural factorization machines for sparse predictive analytics[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'17). New York, NY, USA: Association for Computing Machinery, 2017: 355-364. DOI: 10.1145/3077136.3080777.
    [4] CHENG H T, KOC L, HARMSEN J, et al. Wide & deep learning for recommender systems[C]//Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. New York, NY, USA: Association for Computing Machinery, 2016: 7-10. DOI: 10.1145/2988450.2988454.
    [5] GUO H F, TANG R M, YE Y M, et al. DeepFM: A factorization-machine based neural network for CTR prediction[C]//Twenty-Sixth International Joint Conference on Artificial Intelligence. Melbourne, Australia: AAAI Press, 2017: 1725-1731.
    [6] ZHOU G R, WU K L, BIAN WJ, et al. Res-embedding for deep learning based click-through rate prediction modeling[C]//Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional SparseDate. New York, NY, USA: Association for Computing Machinery, 2019: 1-9. DOI: 10.1145/3326937.3341252.
    [7] ZHOU J, CUI G Q, HU S D, et al. Graph neural networks: a review of methods and applications[J]. AI Open, 2020, 1: 57-81. DOI: 10.1016/j.aiopen.2021.01.001.
    [8] FAN W Q, MA Y, LI Q, et al. Graph neural networks for social recommendation[C]//The World Wide Web Conference (WWW'19). New York, NY, USA: Association for Computing Machinery, 2019: 417-426. DOI: 10.1145/3308558.3313488.
    [9] LI Y J, TARLOW D, BROCKSCHMIDT M, et al. Gated graph sequence neural networks[J]. arXiv: 1511.05493, 2015. https://arxiv.org/abs/1511.05493v4.
    [10] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[C]//ICLR 2017 Conference Submission. Toulon, France: ICLR, 2017.
    [11] HE X N, DENG K, WANG X, et al. LightGCN: simplifying and powering graph convolution network for recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY, USA: Association for Computing Machinery, 2020: 639-648. DOI: 10.1145/3397271.3401063.
    [12] NI Y B, OU D, LIU S C, et al. Perceive your users in depth: learning universal user representations from multiple E-commerce tasks[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD'18). New York, NY, USA: Association for Computing Machinery, 2018: 596-605. DOI: 10.1145/3219819.3219828.
    [13] 王喆. 深度学习推荐系统[M]. 北京: 电子工业出版社, 2020.

    WANG Z. Deep learning recommender system[M]. Beijing: Publishing House of Electronics Industry, 2020.
    [14] WANG X, JI H Y, SHI C, et al. Heterogeneous graph attention network[C]//The World Wide Web Conference (WWW'19). New York, NY, USA: ACM, 2019: 2022-2032. DOI: 10.1145/3308558.3313562.
    [15] 郑诚, 黄夏炎. 联合轻量图卷积网络和注意力机制的推荐方法[J/OL]. 小型微型计算机, 2021: 1-6. http://kns.cnki.net/kcms/detail/21.1106.TP.20201231.1904.019.html.

    ZHENG C, HUANG X Y. A recommendation method combining light graph convolution network and attention[J/OL]. Journal of Chinese Computer System, 2021: 1-6. http://kns.cnki.net/kcms/detail/21.1106.TP.20201231.1904.019.html.
    [16] WANG X, HE X N, WANG M, et al. Neural graph collaborative filtering[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'19). New York, NY, USA: Association for Computing Machinery, 2019: 165-174. DOI: 10.1145/3331184.3331267.
    [17] RENDLE S, FREUDENTHALER C, GANTNERZ, et al. BPR: bayesian personalized ranking from implicit feedback[C]//Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI). Montreal, QC, Canada: AUAI Press, 2009: 452-461. DOI: 10.5555/1795114.1795167.
    [18] QU Y R, FANG B H, ZHANG WN, et al. Product-based neural networks for user response prediction over multi-field categorical data[J]. ACM Transactions on Information Systems, 2019, 37(1): 5. DOI: 10.1145/3233770.
    [19] LI F, CHEN Z R, WANG P J, et al. Graph intention network for click-through rate prediction in sponsored search[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY, USA: Association for Computing Machinery, 2019: 961-964. DOI: 10.1145/3331184.3331283.
    [20] LI Z K, CUI Z Y, WU S, et al. Fi-GNN: modeling feature interactions via graph neural networks for CTR prediction[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York, NY, USA: Association for Computing Machinery, 2019: 539-548. DOI: 10.1145/3357384.3357951.
  • 加载中
图(2) / 表(3)
计量
  • 文章访问数:  28
  • HTML全文浏览量:  25
  • PDF下载量:  5
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-09-04
  • 修回日期:  2021-09-27
  • 网络出版日期:  2022-05-12

目录

    /

    返回文章
    返回