基于角度约束与高斯化质量图的抓取检测方法研究

王文俊; 韩慧妍; 郭磊; 韩燮; 李郁峰; 吴伟州

doi:10.19304/J.ISSN1000-7180.2022.0171

基于角度约束与高斯化质量图的抓取检测方法研究

doi: 10.19304/J.ISSN1000-7180.2022.0171

王文俊^{1, 2},
韩慧妍^{1, 2, ,},
郭磊^{1, 2},
韩燮^{1, 2},
李郁峰³,
吴伟州^{1, 2}

1.
中北大学大数据学院, 山西太原 030051
2.
山西省视觉信息处理及智能机器人工程研究中心, 山西太原 030051
3.
中北大学军民融合协同创新研究院, 山西太原 030051

基金项目:

山西省自然科学基金 201901D111150

山西省应用基础研究计划项目 201901D111144

山西省研究生创新项目 2021Y626

详细信息

作者简介:
王文俊  男，(1998-)，硕士研究生.研究方向为智能机器人、计算机视觉等

郭磊  男，(1990-)，博士，讲师.研究方向为计算机视觉等

韩燮  女，(1964-)，博士，教授，博士生导师.研究方向为虚拟现实、仿真可视化等

李郁峰  男，(1978-)，博士，高级工程师.研究方向为军用机器人技术等

吴伟州  男，(1997-)，硕士研究生.研究方向为智能机器人、计算机视觉等

通讯作者:
韩慧妍(通讯作者) 女，(1980-)，博士，副教授.研究方向为人工智能、机器视觉等.E-mail: hhy980344@163.com

中图分类号: TP242
计量
- 文章访问数: 70
- HTML全文浏览量: 52
- PDF下载量: 10
- 被引次数: 0
出版历程
- 收稿日期: 2022-03-13
- 修回日期: 2022-05-09
- 网络出版日期: 2022-11-29

Research on grasp detection method based on angle constraint and gaussian quality map

WANG Wenjun^{1, 2},
HAN Huiyan^{1, 2
, ,},
GUO Lei^{1, 2},
HAN Xie^{1, 2},
LI Yufeng³,
WU Weizhou^{1, 2}

1.
School of Data Science and Technology, North University of China, Taiyuan 030051, China
2.
Shanxi Province's Vision Information Processing and Intelligent Robot Engineering Research Center, Taiyuan 030051, China
3.
Institute for Civ-Mil Integration & Collaborative Innovation, North University of China, Taiyuan 030051, China

摘要

摘要:
针对动态抓取环境中最优抓取点选取不稳定、抓取角度不准确的问题，提出一种基于角度约束与高斯化质量图的抓取检测方法.首先，将抓取角度按角度取值划分为多个类别，约束类别内角度取值范围，解决因密集标注导致的像素级标注丢失问题；再经形态学开运算方法过滤角度图中由于多个标注堆叠产生的碎片，得到标注一致性更强的抓取角度图.其次，利用高斯函数优化抓取质量图，突出可抓取区域中心位置的重要性，提升最优抓取点选取的稳定性.最后，在全卷积网络的基础上，引入抓取点和抓取方向注意力机制，提出融合注意力的生成式抓取检测网络(Attentive Generative Grasping Detection Network, AGGDN).在Jacquard仿真数据集上的实验结果表明：该方法的检测准确率能够达到94.4%，单次检测时间为11ms，能有效提升对复杂物体的抓取检测能力，且具有较好的实时性.对真实环境中不同姿态摆放的异形目标抓取实验结果表明：该方法抓取成功率能够达到88.8%，对训练集中从未出现的新目标有较强的泛化能力，能够应用于机器人抓取的相关任务.
- 抓取图 /
- 抓取检测 /
- 全卷积网络 /
- 注意力机制 /
- 生成式抓取
Abstract:
To solve the problem of unstable selection of optimal grasping point and inaccurate grasping angle in dynamic grasping environment, a grasp detection method based on angle constraint and gaussian quality map was proposed. Firstly, the grasping angle were divided into several categories according to the angle value, and the angle value range within the category was constrained to solve the pixel-level annotation loss caused by intensive annotation. morphological open operation method was used to filter the debris generated by multiple annotation stacking in the angle map, and the grasping angle map with stronger annotation consistency was obtained. Secondly, gaussian function was used to optimize the grasping quality map to highlight the importance of the center position of the grasping region and improve the stability of the selection of the optimal grasping point. Finally, based on the fully convolutional network, grasping point and grasping direction attention mechanisms are introduced, and an Attentive Generative Grasping Detection Network (AGGDN) is proposed. Experimental results on Jacquard simulation dataset show that the detection accuracy of proposed method achieves 94.4%, and the single detection time is 11ms, which can effectively improve the grasping detecting ability of complex objects, and has good real-time performance. The experimental results of grasping irregular targets with different poses in the real environment show that, the grasping success rate of proposed method can reach 88.8%, and it has strong generalization ability to the new targets that never appear in the training set, and can be applied to the relevant tasks of robot grasping.
- grasping map /
- grasping detection /
- fully convolutional network /
- attention mechanism /
- generative grasping

HTML全文

图 1 抓取图示意

Figure 1. Grasping map diagram

下载: 全尺寸图片幻灯片

图 2 抓取检测框架

Figure 2. Overview of grasping detection system

下载: 全尺寸图片幻灯片

图 3 密集标注示意

Figure 3. Dense annotation schematic

下载: 全尺寸图片幻灯片

图 4 抓取角度图改进过程

Figure 4. Improvement process of grasping angle map

下载: 全尺寸图片幻灯片

图 5 抓取质量图处理过程

Figure 5. Improvement process of grasping quality map

下载: 全尺寸图片幻灯片

图 6 根据标注生成抓取图

Figure 6. Grasping map generation from annotations

下载: 全尺寸图片幻灯片

图 7 AGGDN网络结构

Figure 7. Architecture of the AGGDN network.

下载: 全尺寸图片幻灯片

图 8 Jacquard数据集定性比较结果

Figure 8. Qualitative comparison for Jacquard datasets

下载: 全尺寸图片幻灯片

图 9 异形目标

Figure 9. Adversarial objects

下载: 全尺寸图片幻灯片

图 10 真实环境抓取实例

Figure 10. Physical environment grasping instances

下载: 全尺寸图片幻灯片

表 1 Jacquard数据集定量比较结果

Table 1. Quantitative comparison for Jacquard dataset

算法	模态	准确率/%	时间/ms
Jacquard^[14]	RGB-D	74.2	-
Monocular^[9]	RGB	79.1	-
GG-CNN^[5]	D	84.0	19
Depth-wise^[7]	RGB-D	88.0	7
GR-ConvNet^[8]	D	93.7	20
AGGDN	D	94.4	11

下载: 导出CSV

表 2 不同设置对检测准确率的影响

Table 2. The impact of different settings on performance

模型设置	准确率/%		时间/ms
模型设置	阈值0.25	阈值0.30	时间/ms
基线	92.3	89.7	9
基线+高斯	92.7	90.6	9
基线+高斯+分类	94.4	93.1	11

下载: 导出CSV

表 3 异形目标抓取结果

Table 3. Adversarial objects grasping results

算法	抓取成功率/%
GG-CNN^[5]	77.5 (62/80)
Depth-wise^[7]	78.8 (63/80)
GR-ConvNet^[8]	83.8 (67/80)
AGGDN	88.8 (71/80)

下载: 导出CSV

参考文献(15)

[1]	WATKINS-VALLS D, VARLEY J, ALLEN P. Multi-modal geometric learning for grasping and manipulation[C]//Proceedings of 2019 International Conference on Robotics and Automation. Montreal: IEEE, 2019: 7339-7345. DOI: 10.1109/ICRA.2019.8794233.
[2]	苏杰, 张云洲, 房立金, 等. 基于多重几何约束的未知物体抓取位姿估计[J]. 机器人, 2020, 42(2): 129-138. DOI: 10.13973/j.cnki.robot.190261. SU J, ZHANG Y Z, FANG L J, et al. Estimation of the grasping pose of unknown objects based on multiple geometric constraints[J]. Robot, 2020, 42(2): 129-138. DOI: 10.13973/j.cnki.robot.190261.
[3]	刘亚欣, 王斯瑶, 姚玉峰, 等. 机器人抓取检测技术的研究现状[J]. 控制与决策, 2020, 35(12): 2817-2828. DOI: 10.13195/j.kzyjc.2019.1145. LIU Y X, WANG S Y, YAO Y F, et al. Recent researches on robot autonomous grasp technology[J]. Control and Decision, 2020, 35(12): 2817-2828. DOI: 10.13195/j.kzyjc.2019.1145.
[4]	KROEMER O, NIEKUM S, KONIDARIS G. A review of robot learning for manipulation: Challenges, representations, and algorithms[J]. The Journal of Machine Learning Research, 2021, 22(1): 30.
[5]	MORRISON D, CORKE P, LEITNER J. Learning robust, real-time, reactive robotic grasping[J]. The International Journal of Robotics Research, 2020, 39(2-3): 183-201. DOI: 10.1177/0278364919859066.
[6]	WANG S F, JIANG X, ZHAO J, et al. Efficient fully convolution neural network for generating pixel wise robotic grasps with high resolution images[C]//Proceedings of 2019 IEEE International Conference on Robotics and Biomimetics. Dali: IEEE, 2019: 474-480. DOI: 10.1109/ROBIO49542.2019.8961711.
[7]	TENG Y D, GAO P X. Generative robotic grasping using depthwise separable convolution[J]. Computers & Electrical Engineering, 2021(94): 107318. DOI: 10.1016/j.compeleceng.2021.107318.
[8]	KUMRA S, JOSHI S, SAHIN F. Antipodal robotic grasping using generative residual convolutional neural network[C]//Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas: IEEE, 2020: 9626-9633. DOI: 10.1109/IROS45743.2020.9340777.
[9]	PREW W, BRECKON T, BORDEWICH M, et al. Improving robotic grasping on monocular images via multi-task learning and positional loss[C]//Proceedings of the 2020 25th International Conference on Pattern Recognition. Milan: IEEE, 2021: 9843-9850. DOI: 10.1109/ICPR48806.2021.9413197.
[10]	KLEEBERGER K, BORMANN R, KRAUS W, et al. A survey on learning-based robotic grasping[J]. Current Robotics Reports, 2020, 1(4): 239-249. DOI: 10.1007/s43154-020-00021-6.
[11]	CHALVATZAKI G, GKANATSIOS N, MARAGOS P, et al. Orientation attentive robotic grasp synthesis with augmented grasp map representation[Z]. arXiv: 2006.05123, 2021.
[12]	XIE S N, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5987-5995. DOI: 10.1109/CVPR.2017.634.
[13]	李正明, 章金龙. 基于深度学习的抓取目标姿态检测与定位[J]. 信息与控制, 2020, 49(2): 147-153. DOI: 10.13976/j.cnki.xk.2020.9212. LI Z M, ZHANG J L. Detection and positioning of grab target based on deep learning[J]. Information and Control, 2020, 49(2): 147-153. DOI: 10.13976/j.cnki.xk.2020.9212.
[14]	DEPIERRE A, DELLANDRÉA E, CHEN L M. Jacquard: a large scale dataset for robotic grasp detection[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Madrid: IEEE, 2018: 3511-3516. DOI: 10.1109/IROS.2018.8593950.
[15]	DU G G, WANG K, LIAN S G, et al. Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review[J]. Artificial Intelligence Review, 2021, 54(3): 1677-1734. DOI: 10.1007/s10462-020-09888-5.