Fuzzing for developers

Stockholm C++ 20190314

Paul Dreik

About me

• C++ since 2004
• Independent contractor
• twitter: @pauldreik
• web: www.pauldreik.se
• slides: www.pauldreik.se/talks/20190314_fuzzing/
• code: github.com/pauldreik/presentations

What is fuzzing?

Why bother to fuzz?

• finds corner cases
• finds stuff you couldn't think of
• automatic
• someone else will

Google fuzz track record

Curl

The simplest fuzzer

• Generational fuzzing
• works!
• inefficient
• How do you choose N?

Generation based fuzzing - improved

• use knowledge of the input format
• example: csmith
• nice, but still never better than the tool!

Improvement: add corpus

1. supply set of interesting files
2. corrupt them

Mutational fuzzing

Mutational fuzzing

• much better!
• no source needed!
• radamsa
• has found lots of bugs

Use the source, Luke!

Improve crash probability

• sprinkle with assert()
• c++20 contracts?
• hardening flags
• sanitizers

Use a sensitive memory allocator

guard malloc

electric fence

libdislocator

So far

• Execution reaches far into the program
• More likely to crash

FEEDBACK

The biggest improvement!

• Crash/no crash is too blunt
• Monitor execution path
• Feedback interesting input to the corpus

Feedback

• Monitor
• Feedback

Monitoring techniques

• debugger
• hardware support
• virtual machine
• compiler instrumentation

State of the art fuzzers

• AFL - American Fuzzy Lop
• LibFuzzer - (within llvm)

Both use feedback, compiler instrumentation

Fuzzer comparison

Disclaimer: this is *my opinion* :-)

DEMO

bdecode, by Arvid Norberg

parses a tree from a serialized format

From a bdecode unit test:

char b[] = "i12453e";
bdecode_node e;
error_code ec;
int ret = bdecode(b, b + sizeof(b)-1, e, ec);

Install the tools

$apt install afl g++ afl-clang clang++-6.0
        

entrypoint_afl_plain.cpp

#include "bdecode.hpp"
#include <array>
#include <cassert>
#include <fstream>

int
main(int argc, char* argv[]) {
  std::ifstream ifs(argv[1]);
  std::array<char, 8192> buf;
  ifs.read(buf.data(), buf.size());
  auto nbytes = ifs.gcount();
  bdecode_node e;
  error_code ec;
  int ret = bdecode(buf.begin(),buf.begin() + nbytes, e, ec);
}

Compile

afl-g++ entrypoint_afl_plain.cpp \
bdecode.cpp -lboost_system -o afl_plain

Initial corpus

mkdir -p corpus
echo "" >corpus/empty

Note: putting more effort into this can pay off well

Output directory

mkdir -p output-afl-plain

Start fuzzing!

afl-fuzz -i corpus/ -o output-afl-plain/ -- ./afl_plain @@

(switch to terminal)

Decoding the status screen

LibFuzzer

• part of llvm
• in-process
• very fast
• works very well with sanitizers
• no cool user interface

entrypoint_libfuzzer.cpp

#include "bdecode.hpp"

extern "C" int
LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size)
{
  bdecode_node e;
  error_code ec;
  auto b = reinterpret_cast<const char*>(Data);
  int ret = bdecode(b, b + Size, e, ec);

  return 0;
}

Compile

clang++-6.0 -std=c++14 -g -O3  -fsanitize=fuzzer \
entrypoint_libfuzzer.cpp bdecode.cpp -lboost_system -o libfuzzer_plain

Run

mkdir -p output-libfuzzer-plain

./libfuzzer_plain output-libfuzzer-plain/

(switch to terminal)

make run-libfuzzer-plain

Decoding the output

#1450971 REDUCE cov: 96 ft: 295 corp: 112/5838b \
exec/s: 483657 rss: 33Mb L: 293/1072 MS: 1 EraseBytes-

reproducer

Minimal program to replay an input

For running in a debugger, valgrind

There are caveats!

Sanitizer+reproducer demo

Resources

Thank you!

https://www.pauldreik.se/