UNSUPERVISED HUMAN EMOTION RECOGNITION VIA BODY POSE DYNAMICS BASED ON SELF-SUPERVISED CONTRASTIVE LEARNING

Abstract

The article presents the results of a study aimed at solving the current problem of automated emotion recognition in the absence of large amounts of labeled data. The core idea of the work lies in the use of an unsupervised approach to training neural networks, which allows for the detection of emotional patterns directly from the geometry and kinetics of the human body. The introductory part substantiates the need for a transition from classical supervised learning methods to self-supervised learning approaches, driven by the high cost and subjectivity of manual emotional state labeling. The object, subject, and goal of the study are defined, focusing on creating a fast and accurate system for recognizing seven basic emotions in a video stream.

The literature review section demonstrates that existing solutions (OpenPose, MoveNet) successfully solve the task of pose estimation; however, their application for affective state analysis is usually limited by the need for massive datasets. A scientific gap has been identified regarding insufficient attention to the unsupervised learning of the emotional component of movements. The methodology section describes in detail the proposed hybrid architecture, which combines a Convolutional Neural Network (CNN) for spatial pose encoding and Recurrent Neural Network blocks (LSTM) for temporal dynamics analysis. A key element of the methodology is the implementation of the SimCLR framework, where training occurs through the minimization of the NT-Xent contrastive loss function. The mathematical influence of the temperature parameter $\tau$ on the model's ability to distinguish visually similar poses (Hard Negatives) is substantiated, ensuring high-quality feature formation in the latent space.

The experimental part of the article contains a description of the testing process based on international datasets (RAVDESS, CK+). It outlines the stages of video preprocessing using MediaPipe Holistic, coordinate normalization, and the creation of positive data pairs for contrastive learning. The experimental results confirmed the high efficiency of the method: using only 10% of labeled data for final fine-tuning achieved an accuracy of 78.5%, which is comparable to the performance of full supervised learning. Particular attention is paid to the system's performance, which is 42–45 FPS, confirming the possibility of its real-time use. The conclusions summarize the scientific novelty of the work, which lies in the adaptation of contrastive learning methods for the task of emotional body kinetics, and outline the practical prospects for implementing the development in the fields of social robotics, security systems, and human-machine interfaces.