RESEARCH ON MACHINE LEARNING MODELS FOR TOXIC CONTENT DETECTION

Abstract

The rapid growth of digital communication and the proliferation of online platforms have intensified the need to address the problem of detecting toxic content, including hate speech, cyberbullying, threats, and discriminatory statements. Such manifestations negatively affect both individual users and the broader information environment, undermining trust in digital services and contributing to the spread of social bias. Given the impracticality of large-scale manual moderation, automated methods based on Deep Learning and Natural Language Processing (NLP) have become increasingly important.

This study presents a comparative analysis of modern transformer-based models for toxicity detection in text, including Toxic BERT, Toxic Comment Model, RoBERTa Toxicity Classifier, DehaBERT Mono English, as well as the general-purpose large language model Phi-3 mini 4k without task-specific fine-tuning. The evaluation was conducted on two publicly available datasets – Measuring Hate Speech and the Jigsaw Toxic Comment Dataset – using the metrics Accuracy, Precision, Recall, and F1-score. The primary focus was placed on optimizing the F1-score as a balanced measure between precision and recall. Additionally, a threshold-based filtering mechanism was applied to enable objective comparison of models with different sensitivity levels to toxic content.

The experimental results demonstrate that specialized transformer-based models, particularly the RoBERTa Toxicity Classifier, achieve the highest effectiveness in detecting explicit forms of toxic language, reaching accuracy above 90% for categories such as severe hate speech, threats, and calls for violence. However, performance decreases when addressing context-dependent and indirect manifestations of toxicity, including subtle aggression, disrespect, and offensive humor. It was also shown that class imbalance within the datasets significantly affects classification quality, resulting in lower performance for underrepresented categories. Furthermore, model bias toward specific identity subgroups and sensitivity to profanity were identified.

The study additionally demonstrates that applying a general-purpose LLM without task-specific fine-tuning is both ineffective and computationally inefficient during inference. The obtained results confirm the appropriateness of using specialized models for content moderation tasks and highlight the перспективity of combining general-purpose large language models with domain-specific classifiers to improve context-aware toxicity detection. Future research should focus on developing ensemble approaches and reducing model bias to ensure fairer and more reliable automated moderation systems.