Development of Vision-Based Human Tracking for Drone’s Gimbal

Bayat, Mohammad Hossein; Shahbazi, Mohammad; Tarvirdizadeh, Bahram

doi:10.22034/jast.2022.312829.1105

Document Type : Original Article

Authors

¹ Faculty of New Sciences and Technologies

² Manufacturing engineering, School of Mechanical engineering, Iran University of Science and Technology

https://doi.org/10.22034/jast.2022.312829.1105

Abstract

The use of Unmanned Aerial Vehicles (UAVs) with different features and for a variety of applications has grown significantly. Tracking generic targets, especially human, using the UAV's camera is one of the most active and demanding fields in this area. In this paper we implement two vision-based tracking algorithms to track a human by using a 2D gimbal which can be mounted on UAVs. To ensure smooth movements and reduce the effect of common jumps on the trackers output, the gimbal motion control system is equipped with a Kalman filter followed by a proportional-derivative (PD) controller. Various experimental tests have been designed and implemented to track a human. The evaluation results show success in tracking the high speed movements with one of the algorithms and high accuracy in tracking the challenging movements in the other algorithm. Also in both methods, the tracking computation time is short enough and suitable for real-time implementation. The favorable performance of both algorithms indicate the ability of designed system to be implemented on the UAVs for practical applications.

Keywords

20.1001.1.17352134.2022.15.1.7.5

Main Subjects

Flight Mechanics Stability & Contorl

Article Title [Persian]

Development of Vision-Based Human Tracking for Drone’s Gimbal

Authors [Persian]

Mohammad Hossein Bayat ¹
Mohammad Shahbazi ²
Bahram Tarvirdizadeh ¹

¹ Faculty of New Sciences and Technologies

² Manufacturing engineering, School of Mechanical engineering, Iran University of Science and Technology

Abstract [Persian]

Keywords [Persian]

Vision-based Human Tracking
2D Gimbal
Visual Servoing
Unmanned Aerial Vehicles
Kalman Filter

References

[1] C. H. Lin, F. Y. Hsiao, and F. Bin Hsiao, “Vision-based tracking and position estimation of moving targets for unmanned helicopter systems,” Asian J. Control, vol. 15, no. 5, pp. 1270–1283, 2013, doi: 10.1002/asjc.654.

[2] V. N. Dobrokhodov, I. I. Kaminer, K. D. Jones, and R. Ghabcheloo, “Vision-based tracking and motion estimation for moving targets using unmanned air vehicles,” J. Guid. Control. Dyn., vol. 31, no. 4, pp. 907–917, 2008, doi: 10.2514/1.33206.

[3] Y. Wu, Y. Sui, and G. Wang, “Vision-Based Real-Time Aerial Object Localization and Tracking for UAV Sensing System,” IEEE Access, vol. 5, pp. 23969–23978, 2017, doi: 10.1109/ACCESS.2017.2764419.

[4] R. Cunha et al., “Gimbal Control for Vision-based Target Tracking,” no. 3, pp. 1–5.

[5] N. Wojke, A. Bewley, and D. Paulus, “Simple online and realtime tracking with a deep association metric,” Proc. - Int. Conf. Image Process. ICIP, vol. 2017-Septe, pp. 3645–3649, 2018, doi: 10.1109/ICIP.2017.8296962.

[6] D. Gordon, A. Farhadi, and D. Fox, “Re3 : Real-Time Recurrent Regression Networks for Object Tracking,” IEEE Robot. Autom. Lett., vol. 3, pp. 788–795, 2018.

[7] W. Liu et al., “SSD: Single shot multibox detector,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016, vol. 9905 LNCS, pp. 21–37, doi: 10.1007/978-3-319-46448-0_2.

[8] DJI, “DJI Active Track: Make the Drones Follow You,” 2017. https://store.dji.com/guides/film-like-a-pro-with-activetrack (accessed Jan. 15, 2021).

[9] H. Kang, H. Li, J. Zhang, X. Lu, and B. Benes, “FlyCam: Multitouch Gesture Controlled Drone Gimbal Photography,” IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 3717–3724, 2018, doi: 10.1109/LRA.2018.2856271.

[10] C. Huang et al., “ACT: An Autonomous Drone Cinematography System for Action Scenes,” Proc. - IEEE Int. Conf. Robot. Autom., pp. 7039–7046, 2018, doi: 10.1109/ICRA.2018.8460703.

[11] C. R. Wren, A. Azarbayejani, T. Darrell, and A. P. Pentland, “P finder: real-time tracking of the human body,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 780–785, 1997, doi: 10.1109/34.598236.

[12] X. Zhou, S. Liu, G. Pavlakos, V. Kumar, and K. Daniilidis, “Human Motion Capture Using a Drone,” Proc. - IEEE Int. Conf. Robot. Autom., pp. 2027–2033, 2018, doi: 10.1109/ICRA.2018.8462830.

[13] H. Zhang, Z. Lei, G. Wang, and J. N. Hwang, “Eye in the sky: Drone-based object tracking and 3D localization,” in MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia, 2019, no. 1, pp. 899–907, doi: 10.1145/3343031.3350933.

[14] R. Bartak and A. Vykovsky, “Any object tracking and following by a flying drone,” in Proceedings - 14th Mexican International Conference on Artificial Intelligence: Advances in Artificial Intelligence, MICAI 2015, 2016, pp. 35–41, doi: 10.1109/MICAI.2015.12.

[15] A. Chakrabarty, R. Morris, X. Bouyssounouse, and R. Hunt, “Autonomous indoor object tracking with the Parrot AR.Drone,” in 2016 International Conference on Unmanned Aircraft Systems, ICUAS 2016, 2016, pp. 25–30, doi: 10.1109/ICUAS.2016.7502612.

[16] D. Held, S. Thrun, and S. Savarese, “Learning to track at 100 FPS with deep regression networks,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 9905 LNCS, pp. 749–765, 2016, doi: 10.1007/978-3-319-46448-0_45.

[17] J. F. Henriques, R. Caseiro, P. Martins, and J. Batista, “Exploiting the circulant structure of tracking-by-detection with kernels,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2012, vol. 7575 LNCS, no. PART 4, pp. 702–715, doi: 10.1007/978-3-642-33765-9_50.

[18] A. G. Howard et al., “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” 2017, [Online]. Available: http://arxiv.org/abs/1704.04861.

[19] Y. Jia et al., “Caffe: Convolutional architecture for fast feature embedding,” in MM 2014 - Proc. 2014 ACM Conf. Multimed., 2014, pp. 675–678, doi: 10.1145/2647868.2654889.

[20] Y. Wu, J. Lim, and M. H. Yang, “Online object tracking: A benchmark,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 2411–2418, 2013, doi: 10.1109/CVPR.2013.312.

Journal of Aerospace Science and Technology

Development of Vision-Based Human Tracking for Drone’s Gimbal

References

References

Volume 15, Issue 1
June 2022
Pages 63-72

Development of Vision-Based Human Tracking for Drone’s Gimbal

References

References

Volume 15, Issue 1June 2022Pages 63-72

Volume 15, Issue 1
June 2022
Pages 63-72