Built a touchless gesture control system using Python, OpenCV, and MediaPipe to enable hands-free computer interaction for scenarios where physical contact with peripherals is impractical. The system utilizes a deep learning pipeline to translate live video feed into real-time OS-level mouse events.
The architecture leverages Google MediaPipe's neural networks to perform 21-point 3D hand landmarking, identifying specific knuckle coordinates at sub-30ms latency. By applying vector math and Euclidean distance calculations, the logic layer maps finger "pinches" and palm orientations to standard primary actions: index-thumb pinches for left clicks, ring-finger pinches for right clicks, and middle-finger grabs for precision vertical scrolling.
To ensure precise navigation, the system implements coordinate transformations that map the camera's resolution to the monitor's pixel grid through linear interpolation. A specialized jitter buffer and exponential moving average (EMA) smoothing algorithm were designed to filter raw sensor noise, reducing cursor jitter and maintaining stability during active use.