Secure Coding in C and C++ Using Fuzz Testing

Google Found 25 000 Bugs in Chrome With Fuzzing

Fuzz testing is currently considered one of the most effective ways to find functional bugs and security issues in software. 

With the same security testing approach, engineers at Google have already found more than 25,000 bugs in Chrome. 75% of these vulnerabilities were memory issues, such as heap buffer overflows and use-after-free. 

The Chromium team has almost completely automated its security testing efforts for C/C++ for memory issues with fuzz testing.

In this article, I will show you how to apply fuzz testing to your own code. And the best part: all code examples and tools I will use are 100% open-source. 

What is Fuzz Testing, and How Does it Help Find C/C++ Security Vulnerabilities?

Let’s start with the basics.

Fuzz Testing is a dynamic testing method for finding functional bugs and security issues in software. 

During a fuzz test, a program or a function under test gets executed with thousands of invalid, unexpected, or random inputs in order to crash the application, like a stress test for your application code.

Feedback-based Fuzzing

However, the fuzzer gets feedback on the structure of the code and the states reached by the automatically generated inputs. 

With this information, the fuzzer can adjust and mutate its inputs to produce additional test cases that are highly likely to trigger even more errors and crashes in the system under test. Learn more.

What are Good Use Cases for Fuzz Testing? 

In general, any library or application that handles untrusted or complicated data is well suited for fuzz testing. For example:

• Media Codecs
• Network Protocols, RPC Libraries
• Crypto
• Compression
• Compilers and Interpreters
• Regular Expression Matchers
• Text/UTF processing
• Databases
• Browsers
• Text Editors/Processors

• Embedded System

• OS Kernel, Drivers, Supervisors and VMs

What Bugs Can You Find With Fuzzing in C/C++? 

With fuzzing, you can find all sorts of C/C++ vulnerabilities, that can cause your application to crash, for example:

• Memory Exhaustion
• Hangs or Infinite Loops
• Infinite Recursions
• Denial of Services (DoS) 

• Discrepancies between two implementations of the same protocol
• Round-trip consistency bugs (e.g. compress the input decompress back, -compare with the original)

• NULL deference
• Uncaught Exceptions

• Buffer Overflows
• Use After Free
• Out of Bounds
• Memory Leaks
• Uninitialized reads

How to Trigger Memory Corruptions With Fuzzing? 

With fuzzing, you can also find out-of-bounds errors, all kinds of buffer overflows, and other memory allocation errors, but you may have to install some additional libraries before running your first fuzz test. 

As an example, if a fuzzer generates inputs that exceed the allocated buffer size, these inputs do not necessarily cause the application to crash, since accessing information in the unallocated area might well be allowed.

So, to reliably trigger memory corruptions during a fuzz test, you will need to restrict the program execution, by applying sanitizers to your source code. 

What is a Sanitizer? 

A sanitizer is a software library that you compile to your code to make your program crash more often.

The idea behind sanitizers is to trigger crashes when normal application behavior would not do that.

For example, the AddressSanitizer (ASan) replaces the malloc and free functions to create poisoned areas around the memory you want to allocate.

Example of an AddressSanitizer

So if your application at some point is trying to access the red areas in your system, the sanitizer will throw exceptions or error messages.

List of Sanitizers for Fuzzing

Here is a list of four common C/C++ sanitizers and the vulnerabilities, you can find with them, if you apply those sanitizers in combination with fuzzing: 

• Use after free (dangling pointer dereference)
• Heap buffer overflow
• Stack buffer overflow
• Global buffer overflow
• Use after return
• Use after scope
• Initialization order bugs
• Memory leaks
  
  

2. MemorySanitizer (MSan) is a detector of uninitialized memory reads in C/C++ programs.

3. UndefinedBehaviorSanitizer (UBSan) is a fast undefined behavior detector implemented in Clang and Compiler-rt.

4. ThreadSanitizer (TSan) is a data race detector for C/C++. It finds:

• Signed integer overflow
• Out of Bounds Array
• Out of Bounds BitShifts
• Floating Point Conversion Overflow
• Dereferencing Misaligned or Null Pointers
• Data Races

Good Fuzz Testing Tools

If you are completely new to fuzzing, I would recommend starting with an easy-to-use fuzzer like CI Fuzz or AFL++ for C/C++. 

Here’s a list of popular open-source fuzzers: 

• AFL++ a C/C++ fuzzer that employs genetic algorithms to increase code coverage of the test cases efficiently.
• BFuzz an input-based fuzzer tool that takes .html as input.
• CI Fuzz an easy-to-use fuzzing tool that helps you to integrate and run fuzz tests directly from your command line.
• Go-fuzz gofuzz is a library for populating go objects with random values.
• Honggfuzz a feedback-driven, easy-to-use fuzzer with interesting analysis options.
• Jazzer a coverage-guided fuzzer for Java and other JVM-based languages.
• Jazzer.js a coverage-guided fuzzer for JavaScript and the Node.js platform.
• KernelFuzzer a Kernel Fuzzer, for fuzzing Windows.
• libFuzzer a library for coverage-guided fuzz testing.
• Radamsa a general-purpose fuzzer.

Fuzz Testing From Your Command Line

Now that you have learned about the basics of fuzz testing let’s have a look at how to perform fuzz testing in practice, to enable secure coding of C/C++ applications.  In the next chapter, I will show you how to set up and run a fuzz test, with CI Fuzz. 

CI Fuzz is an easy-to-use fuzzing tool, that enables you to integrate and run fuzz tests directly from your command line. I chose this tool for this tutorial, as it is particularly user-friendly and allows developers to set up and run a fuzz test with only three commands. 

# Initialize fuzzing
$ cifuzz init

# Create your first fuzz test
$ cifuzz create my_fuzz_test

# Run fuzz test and find bugs
$ cifuzz run my_fuzz_test

CI Fuzz supports Linux, MacOS and Windows. If you want to fuzz a CMake based C/C++ application the only prerequisites to run your first fuzz test are CMake (version >= 3.16) and LLVM (version >= 11). You can download CI Fuzz here, or by running the installation script in GitHub:

sh -c "$(curl -fsSL https://raw.githubusercontent.com/CodeIntelligenceTesting/cifuzz/main/install.sh)"

Fuzzing Use Case: How to Find Bugs in an Open Source Library

This example library contains two C/C++ vulnerabilities.

explore_me.cpp
#include "explore_me.h"
#include <cstdio>
#include <string.h>
using namespace std;

// just a function with multiple paths that can be discoverd by a fuzzer
void exploreMe(int a, int b, string c) {
    printf("a: %d; b: %d; c: %s\n", a, b, c.c_str());

    if (a >= 20000) {
           if (b >= 2000000) {
                 if (b - a < 100000) {
                   // Trigger the undefined behavior sanitizer
                   int n = 23;
                   n <<= 32;
                   if (c == "FUZZING") {
                     // Trigger a heap buffer overflow
                     char *s = (char *)malloc(1);
                     strcpy(s, "too long");
                   }
                 }
               }
    }
} 

As you can see, the library has multiple branches that are reached under different conditions. It contains an Undefined Behavior, where the BitShift is larger than the size of the type, and a Heap Buffer Overflow in the branch below.

Now, I want to show you how to write a fuzz tests with CI Fuzz that triggers both C/C++ vulnerabilities. 

How to Set Up a Fuzz Test in Easy 6 Steps

To set up CI Fuzz, just follow the instruction in the documentation. 

How to set up CI Fuzz in CMake

CI Fuzz commands will interactively guide you through the needed options and give you instructions on what to do next. You can find a complete list of commands with all options and parameters by calling cifuzz command --help.

1. Initialize the Project for CI Fuzz

After the installation, the first step would be to initialize the project, with cifuzz init in the root directory of your project. This will create a file named cifuzz.yaml containing the needed configuration and print out any necessary steps to set up your project.

$ cifuzz init
configuration saved in /home/cmake/cifuzz.yaml

Enable fuzz testing in your CMake project by adding the following lines to the top level CMakeLists.txt before any add_subdirectory( … ), 
add_library ( … ) or add_executable( … ) calls: 

find_package(cifuzz)
enable_fuzz_testing()

Use 'cifuzz create' to create your first fuzz test 

2. Add CI Fuzz Integration to CMake Config

Enable fuzz testing and build the project in CMake.

CMakeList.txt
cmake_minimum_required(Version 3.19)

project(cmake_example)

enable_testing()
find_package(cifuzz)
enable_fuzz_testing()

add_subdirectory(src)

add_executable(${PROJECT_NAME} main.cpp )
target_link_libraries (${PROJECT_NAME} LINK_PUBLIC exploreMe)

3. Create Stub for First Fuzz Test

Now, the next step would be to create your first fuzz test. Execute cifuzz create and follow the instructions given by the command. This will create a stub for your fuzz test, let's say it is called my_fuzz_test_1.cpp and tell you how to integrate it into your project.

$ cifuzz create
Select type of the fuzz test:
     > C/C++

☑️ Created fuzz tests stub my_fuzz_test_1.cpp

Note: Fuzz tests can be put anywhere in your repository, but it makes sense to keep them close to the tested code - just like regular unit tests. 

Create CMake target for the fuzz test as follows - it behaves just like a regular add_executable( … ): 
    add_fuzz_tests(my_fuzz_tests_1 my_fuzz_tests_1.cpp)

4. Add Fuzz Test to CMake Config and Link Test

Usually you also have to add instructions in your CMakeLists.txt file to link the fuzz test with the software under test (e.g. use the target_link_libraries directive). The add_fuzz_test directive can be treated like add_executable.

CMakeList.txt
cmake_minimum_required(Version 3.19)

project(cmake_example)

enable_testing()
find_package(cifuzz)
enable_fuzz_testing()

add_subdirectory(src)

add_executable(${PROJECT_NAME} main.cpp )
target_link_libraries (${PROJECT_NAME} LINK_PUBLIC exploreMe)

add_fuzz_test(my_fuzz_test_1 my_fuzz_test_1.cpp)
target_link_libraries (my_fuzz_test_1 LINK_PUBLIC exploreMe)

5. Adjust Fuzz Test to Call the Application With Fuzz Data

Edit my_fuzz_test_1.cpp, so it actually calls the function you want to test with the input generated by the fuzzer. To learn more about writing fuzz tests, you can take a look at our tutorial or one of the example projects.

my_fuzz_test_1.cpp
#include <assert.h>
#include <cifuzz/cifuzz.h>  
#include <fuzzer/FuzzDataProvider.h>
#include <src/explore.me.h>

#include <cifuzz/cifuzz.h>

FUZZ_TEST_SETUP() {
	// Perform any one-time setup requried by the FUZZ_TEST function
}

FUZZ_TEST(const unit8_t *data, size_t size) {  
    // Call the functions you want to test with the provided data and optionally
    // assert that the results are as expected.
    // int res = DoSomething(data, size);
    // assert(res != 1); 
    FuzzDataProvider fuzzed_data(data, size); 
    int a = fuzzed_data.ConsumeIntegral();
    int b = fuzzed_data.ConsumeIntegral();
    std::string c = fuzzed_data.ConsumeRandomLengthString();
    exploreMe(a, b, c);	
}

6. Run the Fuzz Test

Start the fuzzing by executing cifuzz run my_fuzz_test_1. CI Fuzz now tries to build the fuzz test and starts a fuzzing run.

$ cifuzz run_my_fuzz_test_1
[...]

Use ‘cifuzz finding ’ for details on a finding. 

💥[eglated_puma] undefined behavior in exploreMe (src/explore_me.cpp:21:11)
💥[great_protozoa] heap buffer overflow in exploreMe (src/exploreMe:28:11)

Note: The crashing input has been copied to the seed corpus at: 

	my_fuzz_test_1_inputs/great_protoza

It will now be used as a seed input for all runs of the fuzz test, including remote runs with artifacts created via ‘cifuzz bundle’ and regression tests. For more information, see:

	https://github.com/CodeIntelligenceTesting/cifuzz#regression-testing

Execution time: 1s
Average exec/s: 22140
Findings: 	    2
New seeds:      13 (total: 13)

Coverage Reporting and Debugging

What you can do now is take a closer look at the findings. cifuzz findings shows you a list of all findings and cifuzz finding [name] will show you detailed information about a particular finding which is useful for understanding and fixing a bug. Here we have the stack trace for the undefined behavior: 

[eglated_puma] undefined behavior in exploreMe (src/explore_me.cpp:21:11)
Date: 2022-10-04 16:31:11.67212394 +0200 CEST

/home/jochen/dev/talks/webinar-cifuzz/cmake/src/explire_me.cpp:21:11 runtime error shift exponent 32 is too large for 32-bit type 'int'
	#0 0x55a43e491854 in exploreMe(int, int, std::_cxx11::basic_sting<char, std::char_traits <char>, std::allocator<char>>) /home/talks/webinar-cifuzz/cmake/src/explire_me.cpp:21:11
	#1 0x55a43e48ff8c in LLVMFuzzerTestOneInputNoReturn(unsigned char const*, unsigned ling) /home/talks/webinar-cifuzz/cmake/src/explire_me.cpp:16:3
	#2 0x55a43e48ff8c in LLVMFuzzerOneTestInput /home/talks/webinar-cifuzz/cmake/src/explire_me.cpp:11:1
	#3 0x55a43e372cf8 in fuzzer::Fuzzer::ExecuteCallBack(unsigned char const*, unsigned long) (/home/talks/webinar-cifuzz/cmake/.cifuzz-build/libfuzzer/address+undefined/my_fuzz_test_01+0x51cf8)
	#4 0x55a43e372650 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) ((/home/talks/webinar-cifuzz/cmake/.cifuzz-build/libfuzzer/address+undefined/my_fuzz_test_01+0x52650)
	#5 0x55a43e374661 in fuzzer:Fuzzer::MutateAndTestOne() (/home/talks/webinar-cifuzz/cmake/.cifuzz-build/libfuzzer/address+undefined/my_fuzz_test_01+0x53661)
	#6 0x55a4e3376017 in fuzzer::Fuzzer::Loop(std::vector<fuzzer::sizedfile, std::allocator#fuzzer::sizedfile="">) (/home/talks/webinar-cifuzz/cmake/.cifuzz-build/libfuzzer/address+undefined/my_fuzz_test_01+0x55017)
#7 0x55a43e359de4 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) ((/home/talks/webinar-cifuzz/cmake/.cifuzz-build/libfuzzer/address+undefined/my_fuzz_test_01+0x38de4)
#8 0x55a43e3492d7 in main (/home/talks/webinar-cifuzz/cmake/.cifuzz-build/libfuzzer/address+undefined/my_fuzz_test_01+0x282d7)
#9 0x7f199725c2cf (usr/lib/libc.so.6+0x232cf) (BuildId: 9c28cfc869012ebbd43cdb0f1eebcd14e1b8bdd8)
#10 0x7f199735c289 in __lib_start_main (usr/lib/libc.so.6+0x23389) (BuildId: 9c28cfc869012ebbd43cdb0f1eebcd14e1b8bdd8)
#11 0x55a43e349314 in _start ../sysdeps/x86_64/start.S:115

SUMMARY: UndefinedBehaviorSanitizer: undifined-behavior /home/jochen/dev/talks/webinar-cifuzz/cmake/src/exploreMe.cpp:21:11

We also get a report of the the CLI how many lines and function got, and how many of the branches were covered.
Coverage Report
              File |  Functions Hit/Found |  Lines Hit/Found | Branches Hit/Found
my_fuzz_test_1.cpp |         2/2 (100.0%) |     9/9 (100.0%) |       0/0 (100.0%)
src/explore_me.cpp |         1/1 (100.0%) |   23/23 (100.0%) |       8/8 (100.0%)
                   |                      |                  | 	
                   |  Functions Hit/Found |  Lines Hit/Found | Branches Hit/Found                                      
             total |                  3/3 |            32    |                8/8

If you can, you should always try to optimize your fuzzer for code coverage. The idea behind this recommendation is to enable the fuzzer to explore your application, by taking as many different paths through your code as possible. This way, the fuzzer will be able to learn if a specific data structure or inputs are required, to execute the code. The more information the fuzzer gets about the structure of the code, the more effective your fuzz tests will be. 

Closing Thoughts

Getting started with fuzzing is not always easy, and many open-source fuzzers require a lot of skills and knowledge to set up your first test. But I hope this article and CI Fuzz will make it easier for you to get started with fuzzing, as it’s really one of the most effective (and fun) ways to test C/C++ for memory corruptions and other vulnerabilities.

If you have any questions about this article or CI Fuzz, please feel free to reach out via @jochil!