Due to the new ISO/SAE 21434 many car manufacturers (OEMs) are expanding their software testing activities. The standard contains regulations for software devices within vehicles, as well as their connectivity to external systems. This ISO recommends OEMs to integrate feedback-based fuzz testing into the DevOps process and defines new requirements for software security engineering.
| Fuzzing or fuzz testing is an automated security testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks. One of the main advantages of fuzzing is that it can be done during development, similar to dynamic application security testing (DAST) and interactive application security testing (IAST). |
Common Challenges
As a Customer Success Engineer who deals with automotive customers on a daily basis, I see three main challenges that developers and security teams face when implementing fuzzing for their automotive software.
1. Where to start fuzzing?When developers implement fuzz testing for the first time, they often start with a low budget, with little to no fuzzing experience, and no professional support. Besides, they often feel a lot of pressure that their first fuzzing project must deliver great results. Therefore, it is critical to choose the right project to start with.
2. How to fuzz complex systems with dependencies?
Infotainment systems in modern vehicles usually communicate with a whole range of external sensors. The code gets messed up and complicated very quickly. The dependencies in the software make it difficult for developers to fuzz the applications properly and the manual effort remains very high. Therefore, your long-term goal should be to fuzz the application continuously and to simulate the input from those embedded systems simultaneously. This way you will be able to automate your testing process far more than you could with traditional DAST or IAST and achieve a higher level of code coverage.
3. How to integrate fuzz testing into the CI/CD?
Integrating fuzz testing into the CI/CD can help scale the benefits of this method. But for successful integration, communication is key. Developers, security professionals, and managers should be on the same page about the implications of these changes and how they affect certain processes.
This article is a part of a series on automotive security testing in which I will touch all three questions. For more articles on how to build secure automotive software, check out the Code Intelligence - Security Blog. In this article, I will now primarily focus on the question: “Where to start fuzz testing?”
My Advice: Start Simple
Start simple and proceed to more complex challenges. Developers (including myself) sometimes tend to start with the most complicated project or the one that promises the most bugs. But I made the experience that when it comes to fuzzing it is often better to keep it simple.
For example, I would always recommend starting fuzz testing automotive software with a simple C/C++ library that already got tested by your security experts. This way you can get a better feeling for the fuzzing process. Later, you can work your way up to more complex projects step-by-step.
In many cases, fuzz testing will find a lot of new bugs in these already tested C/C++ libraries, even if you thought this code was safe. (Great, a first success! But please be gentle with the person who tested this code last time. Remember… there is also a human component to automated bug finding.)
Roadmap to Successful Fuzz Testing (for Automotive)
I would always recommend automotive developers implement fuzz testing in three simple steps. With each step, you will become more confident and more proficient in what you do. This way you will get the bests results. And each little success will lead you to even more advanced fuzzing approaches that help you make your software more secure.
*Click to enlarge
Step 1: Unit Fuzzing
When initially setting up fuzzing in an automotive environment, the easiest way to do this is using pre-existing unit tests as a sort of template. In most projects, these unit tests already exist, since many automotive developers use unit tests to test the functionality of their software.
During the first hours after setting up the fuzzer, you will usually detect the majority of security issues, just by testing with these self-contained units. Ideal candidates for these low-hanging fruits are libraries, API functions, and different kinds of parsers.
In this stage, you can also set up a continuous integration very easily, by simply repeating this already-defined procedure with every upcoming merge request (similar to DAST or IAST).
Step 2: Interface Fuzzing
On the network level, fuzzing requires more effort as different network interface protocols have to be taken into account (TCP/IP, Ethernet, CANBus, etc.). To simulate this, the fuzzing engines have to be able to understand certain protocols, such as IPSec, UDS, and Protobuf. That's why, interface fuzzing can take several days to weeks, but as the complexity increases, you will also find more bugs.
If you would start with interface fuzz testing before unit test fuzzing, you would most likely also detect all the findings that you could detect with unit test fuzzing. However, it might take you more time to debug and analyze the bugs when they are found via interface fuzzing. Therefore, it's usually better to start with unit fuzzing first.
Step 3: System(-like) Fuzzing
Especially in automotive and complex embedded projects, system fuzzing is the long-term goal. At a certain point all tests should be connected to the hardware to find out what actually happens on the operating system with the commercial compiler. Luckily, it's possible to simulate the input of those systems. The simulations built during unit testing can serve as a fundament for testing the actual hardware.
Recap
Long story short: start with the low-hanging fruits and work your way up. You can already do a lot with a pretty basic fuzzing setup. But in the long game, you should find a way to implement fuzz testing into your CI/CD.
If you liked this article, feel free to join me at my next live coding session, where I will demonstrate how to fuzz embedded systems with dependencies.
www.code-intelligence.com/webinar
About Daniel
Being an engineer at Code Intelligence, Daniel Teuchert holds a master's degree in IT Security. Together with the Customer Success Team of Code Intelligence, he makes great efforts to revolutionize the way to test the software by supporting companies in integrating modern fuzz testing.