Automotive Supplier Finds 32% of Bugs Through Fuzz Testing

Scope of the project

An automotive supplier employs static analysis and Code Intelligence's Fuzz Testing to assess software within a separate business unit. The project comprises 10 million lines of C code, adhering to Classic AUTOSAR standards.

Testing approaches used

Initially, the company implemented static analysis (SAST). When activating all security rules (checkers) in the SAST, millions of issues were reported. The company selectively enabled rules in SAST based on MISRA C and CERT C Secure Coding standards.

To uncover missing vulnerabilities without false positives, the company adopted Code Intelligence’s Fuzzing Testing solution, CI Fuzz. Conducting fuzzing at the system level assists the supplier in identifying critical issues easily reproducible on hardware by sending accurate CAN, SPI, and other data to electronic control units (ECUs) without false positives.

Results

The integration of both tools led to the identification of a wider spectrum of vulnerabilities in their code. Approximately one-third of issues are exclusively identified through static analysis, another third through Code Intelligence's fuzz testing, and the remaining third by both solutions.

Certain vulnerabilities, such as buffer overflows, are detected by both SAST and fuzz testing. When flagged by SAST, the team might overlook them and consider them as false positives. However, immediate attention is given to these issues when they are uncovered by CI Fuzz as it provides the inputs triggering vulnerabilities, along with links to the exact code lines, which makes the issue easily reproducible.

Screenshot at Apr 23 10-26-59

Approximate Bug Detection Split Between SAST and Code Intelligence's Fuzz Testing (CI Fuzz)

Group 20

How companies benefit from using both static analysis and fuzz testing

The best security practice for automotive software involves leveraging both static and dynamic testing, such as fuzz testing. Integrating fuzz testing and SAST helps cover a broader range of potential issues early in development, reducing false positives and negatives and ensuring compliance with requirements.

	Static Code Analysis	Static Analysis + Code Fuzz Testing
End users	Developers and Testing teams	Developers and Testing teams
Application knowledge	Full Knowledge Analyzes the internal structure of a program, understanding the entire context of the code.	Full Knowledge Analyzes the internal structure of a program, understanding the entire context of the code.
Access to source code	Required	Required
Type of code analysis	Static Scans the source code without actually executing it.	Static + Dynamic Scans the source code and the running application.
When executed	Early in development	Early in development and testing
Bugs&Vulnerability detection	Insecure coding practices Malicious Code Authentication and Authorization Issues Code Tampering and Injection Flaws	Insecure coding practices Malicious Code Authentication and Authorization Issues Code Tampering and Injection Flaws. Memory Corruption Problems Input Validation Issues Protocol Parsing Flaws Complex Logic Flaws Interaction with External Systems Environment-specific Issues Security Misconfigurations
Results reliability	Low Produces a large number of false positives and duplicates.	High Fuzz Testing only flags actual issues.
Issues reproducibility	Limited Struggle to offer inputs that allow developers to reproduce identified vulnerabilities easily.	Full Code Fuzz Testing provides inputs that developers can use to reproduce bugs and pinpoint the exact location in the source code.
Integration into the development process	Full Runs directly from IDE.	Full Static analysis runs directly from an IDE, whereas code fuzz testing can be run through any IDE in the same way as unit testing.

Learn more about fuzz testing for automotive software

Learn more about how fuzzing helps automotive companies comply with the ISO 21434 and identify critical bugs that other tools miss here. Or book a call with us to see how it works.