METHODOLOGY FOR AUTOMATING CYBER INCIDENT REPORTS USING LLM

Abstract

The work is devoted to the issues of automation of reporting as part of Threat Intelligence processes. The purpose of the work is to develop a methodology that allows to reduce the burden on employees who process and document the results of cyber incidents in accordance with the requirements of regulatory documents. Among the main results of the work, a reusable instruction template for a large language model (LLM) is proposed. The presented template allows to provide clear instructions, namely required and optional fields, permissible values that are entered into the report fields. Software models based on the Pydantic library are proposed for generating and checking the response in JSON format from LLM. This allows to reduce the length of instructions for LLM by approximately 3 times. A RAG pipeline architecture is proposed to take into account the specific context of regulatory documents in the field of cyber incident reporting. Such a pipeline allows to follow the requirements of legislation and standards without the need to manually prescribe these requirements in the instructions, which speeds up the generation process and improves the quality of reports. A software model has been developed that allows automated generation of a cyber incident report. Such a model does not require manual filling in of incident characteristics, user interaction with the Threat Intelligence platform using the example of MISP (Malware Information Sharing Platform). This approach allows reducing the time for creating a report from hours to minutes, and improving the efficiency of exchanging threat data, avoiding time and financial investments. Another result of the work is a comparative analysis of report generation when using different LLMs, in particular Claude Sonnet 4.5, Gemini 2.5 pro, Grok xAI, GPT 5, DeepSeek, Llama in terms of quality and cost of report generation. For comparison, report quality criteria were proposed, and compliance with the criteria was assessed by an expert method. As a result, the Claude Sonnet 4.5, Gemini 2.5 pro models were identified as leaders in terms of the quality of generated reports. It was established that LLMs are a promising tool for implementation in processing and communication processes in the field of cybersecurity incidents, their use allows fully automating the Threat Intelligence reporting process in an organization.