Even today, the most commonly used password in Germany is “123456”. Most people should be aware by 2019 that this string does not offer adequate protection. Since a security system is only as strong as its weakest link, humans are usually the greatest security risk. But even automated systems are not always error-free. In contrast to humans, however, machine systems can be tested and, if necessary, improved. This security testing, i.e. the systematic testing of the extent to which the security of a system is appropriate and can be guaranteed sustainably by appropriate concepts, is a relatively young sub-discipline of security research of systems. However, it is becoming increasingly important. As the dependency on software and networked systems continues to increase and more and more, critical processes such as human safety, industrial production or even driving a car are transferred to software systems and therefore, the reliability of such systems must be throughly ensured. The development of the Internet and the worldwide tendency to connect more and more systems to this public network of networks means that unauthorized persons are given access to systems to which they should not have access to. In this respect, every company, every institution and every private household should test its IT systems for possible security gaps. With the help of security tests, it is possible to assess the extent to which protective measures taken can successfully ward off external threats and thus create confidence in the security of the system Existing standards and best practices such as Microsoft SDL or the ETSI Guide ETSI EG 203 25, co-designed by Fraunhofer FOKUS, assign security-related test activities to individual phases of a software lifecycle and thus ensure a continuous and early integration of security into the software development process. In principle, the more critical a system is, the more intensively it should be tested. 

Errors in the software or the IT infrastructure allow various attack strategies such as SQL injections or buffer overflows. However, these are avoidable deficiencies that can be prevented by security testing measures. Almost all common database systems use SQL (the Structured Query Language) to edit their databases. Errors in querying such an SQL database may allow an attacker to send their own database commands through the application that provides access to the database. The aim is often to spy out data, to change it in one's own sense, to gain control over servers or simply to cause the greatest possible damage. This was the case, for example, in August 2014, when the security company Hold Security discovered a data leak on over 420,000 websites that led to the theft of approximately 1.2 billion user data, including email addresses, passwords and other data. But not all security measures are appropriate for all circumstances. In the context of IT security, security objectives should be aligned with the company's objectives. However, it is important to stress that absolute security can never be achieved. No system is unassailable, and security testing helps to find existing bugs and can therefore only provide an approximate assessment of the security situation within a pre-defined framework.

The increasing complexity of the software architectures and the high quality expectations of the customers usually result in more complex security tests and the associated rising costs. It must therefore be attempted to reduce the effort by focusing on critical systems areas and structures without neglecting security standards. Risk-based testing provides a remedy. Security risk analyses serve to identify the potential threats to a system in order to determine the probability of its occurrence and its damaging effect. As a first step, the security risks to be tested are identified. This assessment takes into account the situation of the market and competitors as well as system requirements, architectural components, processes and people involved.

After such a correlation, the test objectives can be prioritized according to the risk value. Based on the risk analysis, security test patterns can then be used to derive suitable test cases for a specific security testing problem, e.g. finding a specific SQL injection vulnerability. Once these have been found, the actual strength of such security test patterns comes to light in connection with the automation of the test design and test execution: Once certain gaps have been detected by the security test patterns, special tools can be used to test for specific security gaps. Security test patterns provide a declarative description of the strategies that can be used to adequately test for security vulnerabilities.

Fuzz testing or fuzzing is a technique for the automated detection of security gaps. Fuzzing stimulates a system with invalid or unexpected inputs and tests its robustness. Fuzzing is a very efficient method to detect weak points in a system. It has proven to be a particularly effective technique for detecting 0-day vulnerabilities. Fuzzing can be used, for example, to detect the widespread buffer overflows that occur when the length of an input date is not checked before processing. There are a variety of fuzzing-tools, both commercial and open source. All these tools have in common that they combine protocol modeling, test generation and test execution. Fraunhofer FOKUS pursues a lightweight approach with Fuzzino. With the help of the open source fuzzing-library Fuzzino it is possible to perform fuzzing-based safety tests with existing functional test tools. The goal in the development of Fuzzino was to make fuzzing accessible to a broad audience without requiring detailed prior knowledge. Fuzzino does a lot of the work for the user by automatically making many configuration options based on the test object. This means that even inexperienced testers can be quickly enabled to use this technique with Fuzzing.

Fraunhofer FOKUS also supports the German Federal Office for Information Security (BSI) in carrying out security certifications of IT products according to the certification criteria “Common Criteria for Information Technology Security Evaluation” (ISO/IEC 15408, CC for short). Due to the high practical relevance of fuzzing for CC procedures, BSI has commissioned Fraunhofer FOKUS to investigate the systematic use of fuzzing for security certifications. The main result of the project is a guideline which establishes a relationship between CC and fuzzing and gives recommendations for the use of fuzzing depending on the certification goal and the evaluation object. The project will be presented at the 16th German IT Security Congress in May 2019.

Another tool developed by FOKUS scientists is RACOMAT. This tool is an ISO 31000 compliant risk management tool that combines risk assessment with safety testing. While classical risk assessments depend on subjective expert opinions, the RACOMAT tool enables an objective, holistic analysis of the systems to be evaluated. Based on an overall budget for the risk analysis and initial risk assessments, the RACOMAT tool calculates how much effort should be invested in security testing to improve the quality of the risk picture by reducing uncertainties. Security testing can be used to reveal the vulnerability of systems to attacks such as SQL injection. The RACOMAT tool prioritizes the potential security tests in order to efficiently analyze the most dangerous threats of the most important assets with the existing budget. Fully automated or at least semi-automated testing can be achieved with the RACOMAT tool using reusable security test patterns for common attack patterns. Once testing is complete, the tool helps users interpret the results against the risk picture to supplement newly identified vulnerabilities or adjust the probability of occurrence. The improved risk model can then be used to reschedule the use of remaining resources for security testing. This cycle continues until the total budget for the assessment is exhausted. RACOMAT can then be used to start planning the treatment of unacceptably highly rated risks based on the final risk picture. For this purpose, the expected costs of the untreated risks are compared with the costs of treatment measures over a certain period of time.

Fuzzing and RACOMAT, however, only represent a small part of the variety of different activities in a security test process. To determine different testing approaches, Fraunhofer FOKUS has developed the Security Testing Improvement Profile (STIP), an evaluation scheme to systematically evaluate the influence of the techniques on the maturity of the security test processes and in particular on the model-based security test processes. STIP can be used both independently and in combination with established methods for evaluating security tests - the best-known representatives are TPI (Test Process Improvement) or its successor TPI NEXT and Test Maturity Model Integration (TMMI). Key areas are used to identify various activities that evaluate the maturity level of a software in a hierarchical order. The higher this maturity level, the better the process is characterized with regard to its performance and the quality of the results for the key area. The systematic structuring and definition of dependencies between the maturity levels in the key areas also allows targeted identification and efficient structuring of improvement measures.