A progressive filling algorithm for depth image holes based on multi-channel color discrimination
-
摘要:
深度图像在自动驾驶、三维测量等领域发挥着越来越重要的作用，针对深度图像中空洞信息难以准确修复，填充速率较慢等问题，本文提出一种基于多通道颜色判别的深度图像空洞渐进式填充算法.首先根据深度图像和彩色图像设置筛选条件，对空洞点邻域内像素进行准确筛选，然后计算邻域内像素在空间域和值域下的双边权值并得到带有权重的二维填充模板，进行填充时将二维模板化简为两个互相垂直的一维模板以提高填充速度并采用渐进式填充方法对空洞进行填充.在公开数据集上对实验结果在主观视觉上进行定性对比分析，客观上通过均方根误差和峰值信噪比两个评价指标对本文算法处理效果进行准确分析.实验结果表明，本文方法能较好地保留物体的边界信息，有效防止填充后物体边缘模糊的现象，填充结果准确，填充速率得到优化.
Abstract:Depth images are playing an increasingly important role in areas such as autonomous driving and three-dimensional measurement. Aiming at the problems of difficulty in accurately repairing the information of holes in depth images and slow filling rate, this paper proposes a progressive depth image based on multi-channel color discrimination. Type filling algorithm. First, set the filter conditions according to the depth image and the color image, accurately filter the pixels in the neighborhood of the hole, and then calculate the bilateral weights of the pixels in the neighborhood in the spatial domain and the value domain, and obtain a two-dimensional filling template with weights for filling At this time, the two-dimensional template is reduced to two perpendicular one-dimensional templates to increase the filling speed and a progressive filling method is used to fill the cavity. The experimental results are qualitatively compared and analyzed subjectively on the public data set. Objectively, the two evaluation indicators of the root mean square error and the peak signal-to-noise ratio are used to accurately analyze the processing effect of the algorithm in this paper. The experimental results show that the method in this paper can better retain the boundary information of the object, effectively prevent the blur of the object edge after filling, the filling result is accurate, and the filling rate is optimized.
-
Key words:
- depth image /
- multi-channel /
- hole repair /
- conditional screening /
- progressive filling
-
图 5 小目标物体边界局部采样放大对比
Figure 5. Comparison of local sampling and zooming in the boundary of small target objects
图 7 室内场景图边界局部采样放大对比
Figure 7. Enlarged comparison of local sampling at the boundary of indoor scene graph
图 9 两种场景下不同算法的PSNR值对比
Figure 9. Comparison of PSNR values of different algorithms in two scenarios
表 1 不同滤波算法运算时间比较(ms/帧)
Table 1. Comparison of operation time of different filtering algorithms (ms/frame)
算法 小目标图像 室内场景图像 GF 150 927 BF 160 930 PFMD 110 520 
下载: 导出CSV
-
[1] JAIMES A. Computer vision startups tackleAI[J]. IEEE Multimedia, 2016, 23(4): 94-96. DOI: 10.1109/MMUL.2016.62. [2] 佟帅, 徐晓刚, 易成涛, 等. 基于视觉的三维重建技术综述[J]. 计算机应用研究, 2011, 28(7): 2411-2417. DOI: 10.3969/j.issn.1001-3695.2011.07.003.TONG S, XU X G, YI C T, et al. Overview on vision-based 3D reconstruction[J]. Application Research of Computers, 2011, 28(7): 2411-2417. DOI: 10.3969/j.issn.1001-3695.2011.07.003. [3] ALEXIADIS D S, ZARPALAS D, DARAS P. Real-time, full 3-D reconstruction of moving foreground objects from multiple consumer depth cameras[J]. IEEE Transactions on Multimedia, 2013, 15(2): 339-358. DOI: 10.1109/TMM.2012.2229264. [4] WANG S, BONOMI L, DAI W R, et al. Big data privacy in biomedical research[J]. IEEE Transactions on Big Data, 2020, 6(2): 296-308. DOI: 10.1109/TBDATA.2016.2608848. [5] YANG J, HUANG Z X, GAO Y X, et al. Dynamic learning style prediction method based on a pattern recognition technique[J]. IEEE Transactions on Learning Technologies, 2014, 7(2): 165-177. DOI: 10.1109/TLT.2014.2307858. [6] 沈彤, 刘文波, 王京. 基于双目立体视觉的目标测距系统[J]. 电子测量技术, 2015, 38(4): 52-54. DOI: 10.3969/j.issn.1002-7300.2015.04.013.SHEN T, LIU W B, WANG J. Distance measurement system based on binocular stereo vision[J]. Electronic Measurement Technology, 2015, 38(4): 52-54. DOI: 10.3969/j.issn.1002-7300.2015.04.013. [7] JIAO J B, WANG R G, WANG W M, et al. Local stereo matching with improved matching cost and disparity refinement[J]. IEEE MultiMedia, 2014, 21(4): 16-27. DOI: 10.1109/MMUL.2014.51. [8] 陈千颂, 杨成伟, 潘志文, 等. 激光飞行时间测距关键技术进展[J]. 激光与红外, 2002, 32(1): 7-10. DOI: 10.3969/j.issn.1001-5078.2002.01.002.CHEN Q S, YANG C W, PAN Z W, et al. A brief introduction on the development of laser time-of-flight distance measurement technology[J]. Laser & Infrared, 2002, 32(1): 7-10. DOI: 10.3969/j.issn.1001-5078.2002.01.002. [9] 余兆凯, 彭晓峰, 邱昌杰. 基于Kinect三维重建的地下井室可视化方法研究[J]. 导航定位与授时, 2021, 8(4): 112-119. DOI: 10.19306/j.cnki.2095-8110.2021.04.014.YU Z K, PENG X F, QIU C J. Research on visualization method of underground well chamber based on Kinect 3D reconstruction[J]. Navigation Positioning and Timing, 2021, 8(4): 112-119. DOI: 10.19306/j.cnki.2095-8110.2021.04.014. [10] BHATEJA A, SHRIVASTAV A, CHAUDHARY H, et al. Depth analysis of Kinect v2 sensor in different mediums[J]. Multimedia Tools and Applications, 2021. DOI: 10.1007/s11042-021-11392-z. [11] 刘田间, 郭连朋, 朱禛, 等. 一种深度图像修复算法研究[J]. 信息技术, 2017(6): 107-111. DOI: 10.13274/j.cnki.hdzj.2017.06.025.LIU T J, GUO L P, ZHU Z, et al. A depth image inpainting algorithm[J]. Information Technology, 2017(6): 107-111. DOI: 10.13274/j.cnki.hdzj.2017.06.025. [12] 隋修武, 胡秀兵, 程国栋, 等. 基于模糊推理的图像椒盐噪声自适应滤波算法[J]. 微电子学与计算机, 2018, 35(6): 69-74. DOI: 10.19304/j.cnki.issn1000-7180.2018.06.015.SUI X W, HU X B, CHENG G D, et al. Image salt and pepper noise adaptive filtering algorithm based on fuzzy reasoning[J]. Microelectronics & Computer, 2018, 35(6): 69-74. DOI: 10.19304/j.cnki.issn1000-7180.2018.06.015. [13] KUBICEK J, STRYCEK M, CERNY M, et al. Quantitative and comparative analysis of effectivity and robustness for enhanced and optimized non-local mean filter combining pixel and patch information on MR images of musculoskeletal system[J]. Sensors (Basel, Switzerland), 2021, 21(12): 4161. DOI: 10.3390/S21124161. [14] JANA R B, THOTAKURA H, BALIYAN A, et al. Pixel density based trimmed median filter for removal of noise from surface image[J]. Applied Nanoscience, 2021. DOI: 10.1007/S13204-021-01950-0. [15] NYEMEESHA V, ISMAIL B M. Implementation of noise and hair removals from dermoscopy images using hybrid Gaussian filter[J]. Network Modeling Analysis in Health Informatics and Bioinformatics, 2021, 10(1): 49. DOI: 10.1007/S13721-021-00318-2. [16] 余博, 郭雷, 钱晓亮, 等. 一种新的自适应双边滤波算法[J]. 应用科学学报, 2012, 30(5): 517-523. DOI: 10.3969/j.issn.0255-8297.2012.05.013.YU B, GUO Z, QIAN X L, et al. A new adaptive bilateral filtering[J]. Journal of Applied Sciences, 2012, 30(5): 517-523. DOI: 10.3969/j.issn.0255-8297.2012.05.013. [17] 万琴, 朱晓林, 陈国泉, 等. 分层联合双边滤波的深度图修复算法研究[J]. 计算机工程与应用, 2021, 57(6): 184-190. DOI: 10.3778/j.issn.1002-8331.1912-0463.WAN Q, ZHU X L, CHEN G Q, et al. Research on depth map restoration algorithm based on hierarchical joint bilateral filter[J]. Computer Engineering and Applications, 2021, 57(6): 184-190. DOI: 10.3778/j.issn.1002-8331.1912-0463. [18] SALEHI H, VAHIDI J. A novel hybrid filter for image despeckling based on improved adaptive wiener filter, bilateral filter and wavelet filter[J]. International Journal of Image and Graphics, 2021, 21(3): 2150036. DOI: 10.1142/S0219467821500364. [19] 潘波, 范祺红, 曹雪玮, 等. 基于超像素的深度图修复算法[J]. 计算机应用研究, 2020, 37(6): 1863-1866. DOI: 10.19734/j.issn.1001-3695.2018.11.0913.PAN B, FAN Q H, CAO X W, et al. Depth image inpainting algorithm based on superpixel[J]. Application Research of Computers, 2020, 37(6): 1863-1866. DOI: 10.19734/j.issn.1001-3695.2018.11.0913. [20] 李知菲, 陈源. 基于联合双边滤波器的Kinect深度图像滤波算法[J]. 计算机应用, 2014, 34(8): 2231-2234. DOI: 10.11772/j.issn.1001-9081.2014.08.2231.LI Z F, CHEN Y. Kinect depth image filtering algorithm based on joint bilateral filter[J]. Journal of Computer Applications, 2014, 34(8): 2231-2234. DOI: 10.11772/j.issn.1001-9081.2014.08.2231. -
下载:
点击查看大图
计量
- 文章访问数: 24
- HTML全文浏览量: 20
- PDF下载量: 1
- 被引次数: 0
