Virtual mouse using hand gestures by skin recognition

Main Article Content

ArulMurugan S
Somaiswariy S
Rosshan Banu S
Ruby Angel R


CNN, virtual mouse, deep learning


Motion-controlled PCs and PCs have recently gained ground. This technique is known as the jump movement. Putting our hand in front of the computer and waving, we can control all of its features. Introductions made using a PC have important advantages over slides and overheads. You can use sound, video, and, surprisingly, intuitive projects to expand introductions further. Unfortunately, these techniques are more difficult to use than overheads or slides. With the new controls, the speaker should be able to manage a variety of gadgets (e.g., console, mouse, VCR controller).These devices are difficult to notice in the shadows, and using them upsets the presentation. The most common and convenient form of communication is hand signals. The camera's results will be shown on the screen. Instead of using a traditional mouse or piece of art to manage the mouse cursor, the idea is to use a straightforward camera. With the use of The Virtual Mouse, which is just a camera, establishes a foundation between the user and the system. It enables interaction between users and machines without the need of mechanical or physical mouse equipment, and even manage functions. The technique for controlling where the cursor is placed without the use of any electronics is presented in this paper. Whereas various hand gestures will be used to execute tasks like clicking and lugging stuff. The suggested design will only require a webcam as an information device. The suggested framework necessitates the use of Python, OpenCV, and other hardware. The client can further align the output from the camera by viewing it on the framework's screen. With the correct technology and programming, it is probably conceivable to create a virtual mouse using hand motions and skin detection. The main concept is to translate the movement of your hand and fingers into the equivalent movement of the pointer on the screen by using a camera to track those movements.

Abstract 321 | pdf Downloads 162


1. N. Shaker and M. A. Zliekha, "Real-time Finger Tracking for Interaction," 2007 5th International Symposium on Image and Signal Processing and Analysis, Istanbul, 2007, pp. 141-145.
2. R. M. Prakash, T. Deepa, T. Gunasundari and N. Kasthuri, "Gesture recognition and fingertip detection for human computer interaction," 2017 International Conference on Innovations in Information, Embedded and Communication Systems (ICIIECS), Coimbatore, 2017, pp. 1-4.
3. Dekate, A. Kamal and K. S. Surekha, "Magic Glove – wireless hand gesture hardware controller," 2014 International Conference on Electronics and Communication Systems (ICECS), Coimbatore, 2014, pp. 1-4.
4. J. Suh, M. Amjadi, I. Park and H. Yoo, "Finger motion detection glove toward human-machine interface," 2015 IEEE SENSORS, Busan, 2015, pp. 1-4.
5. R. S. Batu, B. Yeilkaya, M. Unay and A. Akan, "Virtual Mouse Control by Webcam for the Disabled," 2018 Medical Technologies National Congress (TIPTEKNO), Magusa, 2018, pp. 1-4.
6. Mhetar, B. K. Sriroop, A. G. S. Kavya, R. Nayak, R. Javali and K. V. Suma, "Virtual mouse," International Conference on Circuits, Communication, Control and Computing, Bangalore, 2014, pp. 69-72.
7. S. K. Kang, M. Y. Nam and P. K. Rhee, "Color Based Hand and Finger Detection Technology for
User Interaction," 2008 International Conference on Convergence and Hybrid Information Technology, Daejeon, 2008, pp. 229-236.
8. Y. Fang, K. Wang, J. Cheng and H. Lu, "A Real-Time Hand Gesture Recognition Method," in 2007 International Conference on Multimedia & Expo, Beijing, 2007 pp.
9. M. Han, J. Chen, L. Li and Y. Chang, "Visual hand gesture recognition with convolution neural network," in 2016 17th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), Shanghai, China, 2016 pp. 287-291.
10. T. D. Grove, K. D. Baker and T. N. Tan, "color based object tracking," Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170), Brisbane, Queensland, Australia, 1998, pp. 1442-1444 vol.2.
11. N. Vo, Q. Tran, T. B. Dinh, T. B. Dinh and Q. M. Nguyen, "An Efficient Human-Computer Interaction Framework Using Skin Color Tracking and Gesture Recognition," 2010 IEEE RIVF International Conference on Computing & Communication Technologies, Research, Innovation, and Vision for the Future (RIVF), Hanoi, 2010, pp. 1-6.
12. D. A. Barhate and K. P. Rane, "A Survey of Fingertip Character Identification in Open-Air Using Image Processing and HCI," 20183rd International Conference for Convergence in Technology (I2CT), Pune, 2018, pp. 1-4.
13. Riza Sande, Neha Marathe, Neha Bhegade, Akanksha Lugade, Prof. S. S. Jogdand PCET’s Pimpri Chinchwad Polytechnic International Journal of Advanced Research in Science Engineering and Technology Vol. 8, Issue 4 , April 2021
14. Murthy R. S. G., Jadon R. S. A., “Review Of Vision Based Hand Gestures Recognition”, International Journal of Information Technology, 2, 2 (2009) 405-410.
15. K. Sandeep, A. N. Rajagopalan, “Human Face Detection in Cluttered Color Images Using Skin Color and Edge Information”, Indian Conference on Computer Vision, Graphics and Image Processing, Aralık 2002, Ahmedabad, Bildiriler Kitabı, 122-128.
16. Yao Y., Li C. T., “Hand Posture Recognation Using SURF with Adaptive Boosting”, British Machine Vision Conference (BMVC), Eylül 2012, London, Bildiriler Kitabı, 50-60.
17. Denzler J., Paulus D. W., “Active Motion Detection and Object Tracking”, International Conference on Image Processing (ICIP), Kasım 1994, Austin, Bildiriler Kitabı, 635-639.
18. Isard M. A., “Visual Motion Analyses by Probabilistic Propagation of Conditional Density”, Ph.D. thesis, Robotics Research Group Department of Engineering Science, University of Oxford, Oxford, 1998.
19. Jiang L., Wang D., Cai Z., Yan X., “Survey of Improving Naive Bayes for Classification”, Advanced Data Mining and Applications Lecture Notes in Computer Science, 4632, (2007) 134-145.
20. Asaari M. S. M., Suandi S. A., “Hand Gesture Tracking System Using Adaptive Kalman Filter”, Intelligent systems design and applications (ISDA), International Conference on IEEE, Aralık 2010, Cairo, Bildiriler Kitabı, 166 -171.
21. Isard M., Blake A., "CONDENSATION-- conditional density propagation of visual tracking". International Journal of Computer Vision 29 (1): 5–28. [9] Zhou H., Xie L., Fang X., “Visual Mouse: SIFT Detection and PCA Recognition”, Computational Intelligence and Security Workshops (CISW). International Conference on IEEE, Aralık 2007, Harbin, Bildiriler Kitabı, 263-266.
22. Dai J., Song W., Pei L., Zhang J., “Remote Sensing Image Matching via Harris Detector and SIFT Discriptor”, Image and Signal Processing (CISP), International Conference on IEEE, Ekim 2010, Yantai, Bildiriler Kitabı, 2221-2224.
23. Burges C. J. C., “A Tutorial on Support Vector Machines for Pattern Recognation”, Data Mining and Knowledge Discovery, 2, 2 (1998) 121-167. [12] Abe S., “Support Vector Machines for Pattern Classification”, Second Edition, Springer, Kobe, 2010.
24. Byun H., Lee S. W., “Applications of Support Vector Machines for Pattern Recognation: A Survey”, Pattern Recognition with Support Vector Machines, 2388, (2002) 213-23