EVALUATING CRYPTOGRAPHIC RESILIENCE IN THE FREDHOLM CRYPTOSYSTEM: METHODOLOGY AND RESULTS ANALYSIS

Abstract

The rapid advancement of quantum cryptanalysis technologies poses a growing threat to the security of existing asymmetric and symmetric cryptosystems. This challenge has accelerated the global and Ukrainian development of post-quantum cryptography, including the creation of new algorithms and innovative cryptosystems based on unconventional mathematical principles. One such system is the Fredholm cryptosystem, which is both conceptually novel and largely unexplored from a cybersecurity standpoint. Unlike conventional cryptographic solutions grounded in discrete mathematics, the Fredholm system operates on the basis of integral calculus. Due to the absence of theoretical and practical evaluations of its cryptographic strength, the system’s implementation remains limited. To address this gap, the present study introduces a methodology for assessing the cryptographic resilience of the Fredholm cryptosystem. The proposed framework focuses on its weakest component—the encryption key’s resistance to brute-force attacks. The primary theoretical metric is the number of operations required to exhaustively search the key space. Using combinatorial analysis, the study proves a lemma demonstrating that resilience depends solely on the number of discrete elements in the key’s differential spectrum, rather than their permutation order. Time required to compromise the key is considered as an additional metric. For practical resilience, the study employs the norm of the solution to the inverse ill-posed decryption problem, which determines the regularization parameter affecting both stability and accuracy. The probability of successful key recovery is also introduced as a supplementary criterion. The results show that the Fredholm cryptosystem’s practical resilience increases with the number of operations needed to identify the discrete differential spectrum of the encryption key. Based on the findings, the paper outlines the system’s strengths and limitations and offers recommendations for its application in critical information and communication infrastructures.