FORMALIZATION OF A SYSTEM OF INDICATORS FOR EVALUATING THE EFFECTIVENESS OF PERSONNEL TRAINING PROGRAMS IN INFORMATION SECURITY
DOI:
https://doi.org/10.28925/2663-4023.2026.33.1179Keywords:
cybersecurity, information security, information security management, personnel training, effectiveness evaluation, system of indicators, entropy method, modelingAbstract
The article addresses the problem of formalizing a system of indicators for evaluating the effectiveness of personnel training programs. The relevance of the study is due to the fact that, in organizational security practice, the evaluation of such programs is often reduced to the use of separate fragmentary characteristics, including training coverage, testing results, or the frequency of training sessions, which does not provide a holistic quantitative representation of the actual effectiveness of a program. This complicates the comparison of programs, the analysis of their development dynamics, and the substantiation of managerial decisions aimed at improving the training process. The purpose of the article is to formalize a system of indicators for evaluating the effectiveness of personnel training programs by determining the composition of quantitative indicators, substantiating their formal representation, and constructing an integral evaluation. The paper proposes a system of six interrelated indicators covering the main aspects of personnel training program effectiveness, namely training coverage, coverage of relevant thematic blocks, final testing result, program updating, behavioral effectiveness, and timeliness of recurrent training. A formal expression is provided for each indicator, ensuring the possibility of its quantitative determination within a unified evaluation model. The expediency of using the entropy method to determine indicator weight coefficients is substantiated, making it possible to account for their relative discriminatory ability on a set of actual or simulated observations. To demonstrate the methodology, the paper develops a scenario-analytical model based on characteristic ranges of indicator values for training programs with different maturity levels. The scientific novelty of the study lies in the formalization of a system of indicators for evaluating the effectiveness of personnel training programs and in substantiating an approach to constructing an integral indicator based on entropy-derived weight coefficients. The obtained results showed that the greatest contribution to the differentiation of training programs by effectiveness level is made by the indicators of behavioral effectiveness, timeliness of recurrent training, and coverage of relevant thematic blocks. The practical significance of the study lies in the possibility of using the proposed model for comparative analysis of training programs, evaluation of their effectiveness dynamics, and support of decisions on improving the training process.
Downloads
References
Kannelønning, K., & Katsikas, S. K. (2023). A systematic literature review of how cybersecurity-related behavior has been assessed. Information & Computer Security, 31(4), 463-477. https://doi.org/10.1108/ics-08-2022-0139
Haney, J., & Lutters, W. (2020). Security awareness training for the workforce: Moving beyond "check-the-box" compliance. Computer, 53(10), 91-95. https://doi.org/10.1109/mc.2020.3001959
Alnajim, A. M., Habib, S., Islam, M., AlRawashdeh, H. S., & Wasim, M. (2023). Exploring cybersecurity education and training techniques: A comprehensive review of traditional, virtual reality, and augmented reality approaches. Symmetry, 15(12), Article 2175. https://doi.org/10.3390/sym15122175
National Institute of Standards and Technology. (2024a). Building a cybersecurity and privacy awareness and training program (NIST Special Publication 800-50r1). U.S. Department of Commerce. https://doi.org/10.6028/NIST.SP.800-50r1
Bada, M., & Nurse, J. R. C. (2019). Developing cybersecurity education and awareness programmes for small- and medium-sized enterprises (SMEs). Information & Computer Security, 27(3), 393-410. https://doi.org/10.1108/ics-07-2018-0080
Dahabiyeh, L. (2021). Factors affecting organizational adoption and acceptance of computer-based security awareness training tools. Information & Computer Security. Advance online publication. https://doi.org/10.1108/ics-12-2020-0200
Chaudhary, S., Gkioulos, V., & Katsikas, S. (2022). Developing metrics to assess the effectiveness of cybersecurity awareness program. Journal of Cybersecurity, 8(1), tyac006. https://doi.org/10.1093/cybsec/tyac006
Hijji, M., & Alam, G. (2022). Cybersecurity awareness and training (CAT) framework for remote working employees. Sensors, 22(22), 8663. https://doi.org/10.3390/s22228663
Hillman, D., Harel, Y., & Toch, E. (2023). Evaluating organizational phishing awareness training on an enterprise scale. Computers & Security, 134, 103364. https://doi.org/10.1016/j.cose.2023.103364
Prümmer, J., van Steen, T., & van den Berg, B. (2023). A systematic review of current cybersecurity training methods. Computers & Security, 134, 103585. https://doi.org/10.1016/j.cose.2023.103585
Prümmer, J., van Steen, T., & van den Berg, B. (2024). Assessing the effect of cybersecurity training on end-users: A meta-analysis. Computers & Security, 137, 104206. https://doi.org/10.1016/j.cose.2024.104206
National Institute of Standards and Technology. (2024b). The NIST cybersecurity framework 2.0 (NIST Special Publication 1299). U.S. Department of Commerce. https://doi.org/10.6028/NIST.SP.1299
Kő, A., Tarján, G., & Mitev, A. (2023). Information security awareness maturity: Conceptual and practical aspects in Hungarian organizations. Information Technology & People, 36(8), 174-195. https://doi.org/10.1108/itp-11-2021-0849
Yigit Ozkan, B., van Lingen, S., & Spruit, M. (2021). The cybersecurity focus area maturity (CYSFAM) model. Journal of Cybersecurity and Privacy, 1(1), 119–139. https://doi.org/10.3390/jcp1010007
Marshall, N., Sturman, D., & Auton, J. C. (2024). Exploring the evidence for email phishing training: A scoping review. Computers & Security, 139, 103695. https://doi.org/10.1016/j.cose.2023.103695
Dornheim, P., & Zarnekow, R. (2023). Determining cybersecurity culture maturity and deriving verifiable improvement measures. Information & Computer Security. Advance online publication. https://doi.org/10.1108/ics-07-2023-0116
Neri, M., Benevento, E., Stefanini, A., Aloini, D., Niccolini, F., Carducci, A., Federigi, I., & Dini, G. (2024). Understanding information security awareness: Evidence from the public healthcare sector. Information & Computer Security. Advance online publication. https://doi.org/10.1108/ics-04-2024-0094
Roszkowska, E., & Wachowicz, T. (2024). Impact of normalization on entropy-based weights in Hellwig’s method: A case study on evaluating sustainable development in the education area. Entropy, 26(5), 365. https://doi.org/10.3390/e26050365
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Михайло Запорожченко, Світлана Легомінова, Дмитро Рабчун
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
