改进的萤火虫算法优化双支持向量机参数

顾佳鑫; 贺兴时; 杨新社

doi:10.19304/J.ISSN1000-7180.2022.0230

改进的萤火虫算法优化双支持向量机参数

doi: 10.19304/J.ISSN1000-7180.2022.0230

1.
西安工程大学理学院, 陕西西安 710600
2.
密德萨斯大学科学与技术学院, 英国伦敦 NW4 4BT 701

基金项目:

国家自然科学基金 12101477

陕西省自然科学基础研究计划 2020JQ-831

详细信息

作者简介:
顾佳鑫女，(1997-)，硕士研究生.研究方向为机器学习与智能优化算法

杨新社男，(1965-)，博士，英国国家物理实验室高级科学家.研究方向为数学建模、工程优化以及科学、数值计算方法等

通讯作者:
贺兴时(通讯作者) 男，(1960-)，硕士，教授.研究方向为智能优化算法、数理统计、数据挖掘等.E-mail: xsh1002@126.com

中图分类号: TP301.6
计量
- 文章访问数: 92
- HTML全文浏览量: 83
- PDF下载量: 18
- 被引次数: 0
出版历程
- 收稿日期: 2022-04-08
- 修回日期: 2022-04-28
- 网络出版日期: 2022-11-29

Improved firefly algorithm optimizes twin support vector machine parameters

1.
College of Science, Xi'an polytechnic University, Xi'an 710600, China
2.
College of Science and Technology, Middlesex University, London NW4 4BT 701, UK

摘要

摘要:
针对原始萤火虫算法(Firefly Algorithm，FA)易陷入局部最优、求解精度低，双支持向量机(Twin Support Vector Machine, TWSVM)参数选择困难的问题，提出基于改进萤火虫算法(DEFA)的双支持向量机模型(DEFA-TWSVM).首先，对原始萤火虫算法进行改进，得到DEFA算法：在萤火虫位置更新公式中结合动态惯性权重，自适应地调整步长控制因子来快速搜索全局和局部最优解，对每次移动后的萤火虫群融入差分进化算法(Differential Evolution，DE)策略，保证种群迭代多样性，通过基准测试函数的仿真结果表明改进后的算法全局寻优能力强，不易陷入局部最优.其次，利用DEFA算法优化TWSVM的参数.最后，在UCI数据集进行测试，得到DEFA-TWSVM和其他模型的分类准确率.通过比较发现：DEFA算法可以在训练过程中自动确定TWSVM参数，解决了TWSVM参数选择盲目的问题，平均分类准确率相较其他模型提高了2到5个百分点.
- 双支持向量机 /
- 参数选择 /
- 萤火虫算法 /
- 动态惯性权重 /
- 差分进化算法 /
- 自适应步长
Abstract:
In view of the problems of the original Firefly Algorithm (FA), which is easy to fall into local optimization, low solution accuracy and difficult parameter selection of twin support vector machine (TWSVM), a dual support vector machine model (DEFA-TWSVM) based on improved firefly algorithm (DEFA) is proposed. Firstly, the original firefly algorithm is improved to obtain DEFA algorithm. In the firefly position update formula, dynamic inertia weight was combined, and the step size control factor was adjusted adaptively to quickly search for global and local optimal solutions. Differential Evolution (DE) strategy was applied to the firefly population after each movement to ensure the iterative diversity of the population. The simulation results of benchmark test function show that the improved algorithm has strong global optimization ability and is not easy to fall into local optimization. Secondly, DEFA algorithm was used to optimize the parameters of TWSVM. Finally, the classification accuracy of DEFA-TWSVM and other models is obtained by testing in UCI data set. By comparison, it is found that DEFA algorithm can automatically determine TWSVM parameters in the training process, which solves the problem of blind TWSVM parameter selection, and the average classification accuracy is increased by 2 to 5 percentage points compared with other models.
- twin support vector machine /
- parameter selection /
- firefly algorithm /
- dynamic inertia weight /
- differential evolution algorithm /
- adaptive step size

HTML全文

图 1 动态惯性权重w变化曲线

Figure 1. Variation diagram of dynamic inertia weight w

下载: 全尺寸图片幻灯片

图 2 α变化曲线

Figure 2. Variation diagram of α

下载: 全尺寸图片幻灯片

图 3 收敛曲线图

Figure 3. Convergence curve

下载: 全尺寸图片幻灯片

图 4 各算法在UCI数据集的分类准确率

Figure 4. Classification accuracy of algorithm on UCI data set

下载: 全尺寸图片幻灯片

表 1 基准测试函数

Table 1. Benchmark function

函数	函数表达式	解空间	最优值
Ackley	$f_1(x)=-20 \exp \left(-0.2 \sqrt{\frac{1}{n} \sum\limits_{i=1}^n x_i^2}\right)-\exp \left(\frac{1}{n} \sum\limits_{i=1}^n \cos \left(2 \pi x_i\right)\right)+20+e $	[－32，32]	0
Rastrigin	$ f_2(x)=\sum\limits_{i=1}^n\left[x_i^2-10 \cos \left(2 \pi x_i\right)+10\right]$	[－5.12，5.12]	0
Griewank	$ f_3(x)=\frac{1}{4\;000} \sum\limits_{i=1}^n x_i^2-\prod\limits_{i=1}^n \cos \left(\frac{x_i}{\sqrt{i}}\right)+1$	[－600，600]	0
Schwefel′s2.2	$ f_4(x)=\sum\limits_{i=1}^n\left\|x_i\right\|+\prod\limits_{i=1}^n\left\|x_i\right\|$	[－10，10]	0
Sphere	$ f_5(x)=\sum\limits_{i=1}^n x_i^2$	[－100，100]	0
Sum square	$ f_6(x)=\sum\nolimits_{i=1}^n i x_i^2$	[－10，10]	0

下载: 导出CSV

表 2 算法参数设置

Table 2. Algorithm parameter settings

算法名称	参数设置
DEFA	γ=1，β₀=0.2，F=0.5，n=30，T=200
FA	α=0.4，γ=1，β₀=0.2，n=30，T=200
BA	A=20，r=0.5，n=30，T=200
FPA	P=0.7，n=30，T=200
CS	Pa=0.25，α₀=0.01，n=30，T=200

下载: 导出CSV

表 3 实验数据集描述

Table 3. Description of experimental data set

数据集名称	样本数	特征数
Sonar	208	60
Heart-c	303	13
Ionosphere	351	34
Monks3	432	6
Australian	690	14
German	1 000	24
Image-seg	2 310	19

下载: 导出CSV

表 4 各算法在不同数据集上的分类准确率对比/%

Table 4. Comparison of classification accuracy of each algorithm on different data sets/%

数据集名称	准确率±标准差
数据集名称	TWSVM	Grid-TWSVM	PSO-TWSVM	FA-TWSVM	DEFA-TWSVM
Sonar	61.17±5.25	64.00±3.71	64.55±3.72	65.95±5.42	67.50±3.34
Heart-c	84.09±7.71	84.46±5.81	88.57±4.92	89.71±4.63	90.86±6.54
Ionosphere	94.31±4.23	95.09±3.11	96.54±4.61	96.01±7.71	96.88±2.52
Monks3	75.83±3.21	77.52±5.83	77.19±4.10	78.36±4.23	81.42±5.61
Australian	79.71±5.30	79.28±5.59	81.45±7.38	80.72±6.27	85.58±5.73
German	68.69±3.09	69.75±4.04	69.89±4.42	71.05±4.31	74.60±3.90
Image-seg	83.81±5.84	85.24±2.17	87.14±2.02	85.71±3.36	87.61±5.69
平均准确率	78.23	79.33	80.76	81.07	83.49

下载: 导出CSV

参考文献(21)

[1]	MANGASARIAN O L, WILD E W. Multisurface proximal support vector machine classification via generalized eigenvalues[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 28(1): 69-74. DOI: 10.1109/TPAMI.2006.17.
[2]	JAYADEVA, KHEMCHANDANI R, CHANDRA S. Twin support vector machines for pattern classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(5): 905-910. DOI: 10.1109/TPAMI.2007.1068.
[3]	HUANG H J, WEI X X, ZHOU Y Q. Twin support vector machines: a survey[J]. Neurocomputing, 2018(300): 34-43. DOI: 10.1016/j.neucom.2018.01.093.
[4]	安悦瑄, 丁世飞, 胡继普. 孪生支持向量机综述[J]. 计算机科学, 2018, 45(11): 29-36. DOI: 10.11896/j.issn.1002-137X.2018.11.003. AN Y X, DING S F, HU J P. Twin support vector machine: a review[J]. Computer Science, 2018, 45(11): 29-36. DOI: 10.11896/j.issn.1002-137X.2018.11.003.
[5]	CHEN S G, WU X J. A new fuzzy twin support vector machine for pattern classification[J]. International Journal of Machine Learning and Cybernetics, 2018, 9(9): 1553-1564. DOI: 10.1007/s13042-017-0664-x.
[6]	TOMAR D, AGARWAL S. An effective weighted multi- class least squares twin support vector machine for imbalanced data classification[J]. International Journal of Computational Intelligence Systems, 2015, 8(4): 761-778. DOI: 10.1080/18756891.2015.1061395.
[7]	CHEN X B, YANG J, YE Q L, et al. Recursive projection twin support vector machine via within-class variance minimization[J]. Pattern Recognition, 2011, 44(10-11): 2643-2655. DOI: 10.1016/j.patcog.2011.03.001.
[8]	TIAN Y J, QI Z Q, JU X C, et al. Nonparallel support vector machines for pattern classification[J]. IEEE Transactions on Cybernetics, 2014, 44(7): 1067-1079. DOI: 10.1109/TCYB.2013.2279167.
[9]	TANVEER M, TIWARI A, CHOUDHARY R, et al. Sparse pinball twin support vector machines[J]. Applied Soft Computing, 2019(78): 164-175. DOI: 10.1016/j.asoc.2019.02.022.
[10]	XU Z B, DAI M W, MENG D Y. Fast and efficient strategies for model selection of Gaussian support vector machine[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2009, 39(5): 1292-1307. DOI: 10.1109/TSMCB.2009.2015672.
[11]	奉国和. SVM分类核函数及参数选择比较[J]. 计算机工程与应用, 2011, 47(3): 123-124. DOI: 10.3778/j.issn.1002-8331.2011.03.037. FENG G H. Parameter optimizing for support vector machines classification[J]. Computer Engineering and Applications, 2011, 47(3): 123-124. DOI: 10.3778/j.issn.1002-8331.2011.03.037.
[12]	李素, 袁志高, 王聪, 等. 群智能算法优化支持向量机参数综述[J]. 智能系统学报, 2018, 13(1): 70-84. DOI: 10.11992/tis.201707011. LI S, YUAN Z G, WANG C, et al. Optimization of support vector machine parameters based on group intelligence algorithm[J]. CAAI Transactions on Intelligent Systems, 2018, 13(1): 70-84. DOI: 10.11992/tis.201707011.
[13]	姜雯. 基于改进粒子群算法的SVM参数优化及应用[D]. 镇江: 江苏科技大学, 2020. JIANG W. Parameter optimization and application of SVM based on improved particle Swarm optimization[D]. Zhenjiang: Jiangsu University of Science and Technology, 2020.
[14]	杨海, 丁毅, 沈海斌. 基于改进萤火虫算法的SVM核参数选取[J]. 计算机应用与软件, 2015, 32(6): 256-258. DOI: 10.3969/j.issn.1000-386x.2015.06.063. YANG H, DING Y, SHEN H B. SVM kernel parameter selection based on improved gso[J]. Computer Applications and Software, 2015, 32(6): 256-258. DOI: 10.3969/j.issn.1000-386x.2015.06.063.
[15]	JINZ Z, HE D Q, CHEN Y J, et al. Research on fault diagnosis method of train rolling bearing based on variational modal decomposition and bat algorithm-support vector machine[J]. Journal of Physics: Conference Series, 2021, 1820: 012170. DOI: 10.1088/1742-6596/1820/1/012170.
[16]	顾佳鑫, 贺兴时, 刘青. 改进的布谷鸟搜索算法对支持向量机参数优化[J]. 西安工程大学学报, 2022, 36(2): 110-118. DOI: 10.13338/j.issn.1674-649x.2022.02.015. GU J X, HE X S, LIU Q. Parameter optimization of support vector machine based on improved cuckoo search algorithm[J]. Journal of Xi'an Polytechnic University, 2022, 36(2): 110-118. DOI: 10.13338/j.issn.1674-649x.2022.02.015.
[17]	洪露, 贺兴时, 杨新社. 基于三重动态调整的花授粉算法[J]. 西安工程大学学报, 2021, 35(2): 97-103. DOI: 10.13338/j.issn.1674-649x.2021.02.016. HONG L, HE X S, YANG X S. The flower pollination algorithm based on triple dynamic adjustment[J]. Journal of Xi'an Polytechnic University, 2021, 35(2): 97-103. DOI: 10.13338/j.issn.1674-649x.2021.02.016.
[18]	SARTAKHTI J S, AFRABANDPEY H, SARAEE M. Simulated annealing least squares twin support vector machine (SA-LSTSVM) for pattern classification[J]. Soft Computing, 2017, 21(15): 4361-4373. DOI: 10.1007/s00500-016-2067-4.
[19]	李景灿, 丁世飞. 基于人工鱼群算法的孪生支持向量机[J]. 智能系统学报, 2019, 14(6): 1121-1126. DOI: 10.11992/tis.201905025. LI J C, DING S F. Twin support vector machine based on artificial fish swarm algorithm[J]. CAAI Transactions on Intelligent Systems, 2019, 14(6): 1121-1126. DOI: 10.11992/tis.201905025.
[20]	BILAL, PANT M. A brief overview of firefly algorithm[M]// PANT M, RAY K, SHARMA T, et al. Soft Computing: Theories and Applications: Proceedings of SoCTA 2016, Volume#2. Singapore: Springer, 2018: 727-738. DOI: 10.1007/978-981-10-5699-4_69.
[21]	BILAL, PANT M, ZAHEER H, et al. Differential evolution: a review of more than two decades of research[J]. Engineering Applications of Artificial Intelligence, 2020(90): 103479. DOI: 10.1016/j.engappai.2020.103479.