Gesture Interface for the Blind Using the Leap Motion Controller

Krzysztof Dobosz, Rafał Pomocnik


The aim of described preliminary studies was the evaluation of prototype gesture user interface using the Leap Motion controller. Provided experiment used a modified version of the file explorer that was supported by gesture controller and synthesized voice feedback. Obtained results shows that visual disability does not have a significant influence on the correctness of controlling the application using the proposed contact-free interface.


gesture interface; visually impaired; assistive technology; Leap Motion

Full Text:



Mitra S., Acharya T.: Gesture recognition: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2007, vol. 37 (3), p. 311÷324.

Anusuya M.A., Katti S.K.: Speech recognition by machine, a review. International Journal of Computer Science and Information Security, 2010, vol. 6 (3), p. 181÷205.

Morimoto C.H., Mimica M.R.: Eye gaze tracking techniques for interactive applications. Computer Vision and Image Understanding, 2005, vol. 98 (1), p. 4÷24.

Premaratne P., Nguyen Q., Premaratne M.: Human computer interaction using hand ges-tures. In Advanced Intelligent Computing Theories and Applications - 6th International Conference on Intelligent Computing, Communications in Computer and Information Science, Springer 2010, vol. 93, p. 381÷386.

Leap motion - mac & pc motion controller for games, design, virtual reality & more. (accessed: September 2015).

Guna J., Jakus G., Pogacnik M., Tomazic S., Sodnik J.: An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors, 2014, vol. 14 (2), p. 3702÷3720.

Weichert F., Bachmann D., Rudak B., Fisseler D.: Analysis of the accuracy and robust-ness of the leap motion controller. Sensors, 2013, vol. 13 (5), p. 6380÷6393.

Tung J.Y., Lulic T., Gonzalez D.A., Tran J., Dickerson C.R., Roy E.A.: Evaluation of a portable markerless finger position capture device: accuracy of the leap motion controller in healthy adults. Physiological Measurement, 2015, vol. 36 (5), p. 1025÷1035.

Sturman M.M., Vaillancourt D.E., Corcos D.M.: Effects of aging on the regularity of physiological tremor. Journal of Neurophysiology, 2005, vol. 93 (6), p. 3064÷3074.

Bachmann D., Weichert F., Rinkenauer G.: Evaluation of the leap motion controller as a new contact-free pointing device. Sensors, 2015, vol. 15 (1), p. 214÷233.

Zaiti I.-A., Pentiuc S.-G., Vatavu R.-D.: On free-hand tv control: experimental results on user-elicited gestures with leap motion. Personal and Ubiquitous Computing, 2015, vol. 19 (5), p. 821÷838.

Jakus G., Guna J., Tomazic S., Sodnik J.: Evaluation of leap motion controller with a high precision optical tracking system. In Human-Computer Interaction. Advanced Interaction Modalities and Techniques – 16th International Conference, HCI International, Lecture Notes in Computer Science, Springer, 2014, vol. 8511, p. 254÷263.

He L., Hu X. Q., Wei D.: The study of interaction about the online virtual learning com-munity based on leap motion. In Proceedings of the International Conference on Computational and Information Sciences, 2014, p. 1225÷1229.

Liu D.: Research on digital exhibition old summer palace somatosensory interaction based on leap motion. In Proceedings of the 2nd International Conference on Electrical, Computer Engineering and Electronics, Advances in Comptuer Science Research 2015, vol. 24, p. 1079÷1083.

Mohandes M., Aliyu S., Deriche M.: Arabic sign language recognition using the leap motion controller. In Proceedings of the IEEE International Symposium on Industrial Electronics, 2014, p. 960÷965.

Iosa M., Morone G., Fusco A., Castagnoli M., Fusco F.R., Pratesi L., Paolucci S.: Leap motion controlled videogame-based therapy for rehabilitation of elderly patients with sub-acute stroke: a feasibility pilot study. Topics in Stroke Rehabilitation, 2015, vol. 22 (4), p. 306÷316.

Mauser S., Burgert O.: Touch-free, gesture-based control of medical devices and software based on the leap motion controller. Studies in health technology and informatics 2014, vol. 196 (1), p. 265÷270.

Rosa G.M., Elizondo M.L.: Use of a gesture user interface as a touchless image navigation system in dental surgery: Case series report. Imaging Science in Dentistry, 2014, vol. 44 (2), p. 155÷160.