COMPARATIVE ANALYSIS OF DEEP AND MACHINE LEARNING METHODS FOR NETWORK INTRUSION DETECTION

Abstract

This paper presents the results of a comprehensive comparative study of six machine learning and deep learning methods for the task of multi-class network attack classification. We evaluate the effectiveness of a convolutional neural network (CNN-IDS), long short-term memory network (LSTM-IDS), LightGBM and CatBoost gradient boosting, Transformer-IDS based on the self-attention mechanism, and Mamba-IDS based on Selective State Space Models (S6). Experiments are conducted on four benchmark network traffic datasets: CIC-IDS2017, CIC-IDS2018, UNSW-NB15 and CICIoT2023. To ensure reproducibility, we apply a unified preprocessing protocol with feature standardization, stratified 70/15/15 splitting, and weighted loss functions to address class imbalance. Evaluation is performed using Accuracy, Macro F1-score, MCC (Matthews Correlation Coefficient), and Weighted F1-score. Results show that LightGBM gradient boosting achieves the highest accuracy across all four datasets. Deep learning models (CNN, LSTM, Transformer, Mamba) demonstrate better generalization on imbalanced datasets, particularly higher Macro Recall for rare attack classes. Mamba-IDS shows competitive results compared to Transformer-IDS with linear O(n) computational complexity instead of quadratic O(n²), making it promising for real-time processing of long network traffic sequences. Per-class F1-score analysis reveals significant differences in models' ability to recognize rare attack classes, emphasizing the need for multi-class evaluation beyond overall accuracy. The study contributes to understanding the strengths and limitations of modern neural network architectures for intrusion detection systems and provides practical recommendations for selecting the optimal method depending on dataset characteristics and processing time requirements.