Fuzzing: what’s behind the automated testing technique

People make mistakes irrespective of the type of work they’re doing, and software development is no exception. Increasingly complex programs that use modules or functions from different vendors contribute to the problem. More often, finished components are continuously integrated. As a result, it becomes hard to predict program behaviour in advance. This poses risks, especially if software has to process external data following a manual input or once a file is opened. In the worst-case scenario, if the format of the input does not match the expected format, the program may even crash.

Attackers often use exploits to take advantage of these types of bugs, which is why it’s always a good idea to test software for weaknesses.

Note

An exploit takes advantage of vulnerabilities in software to gain elevated system privileges or execute malicious code. As an administrator, you can stay up to date on known vulnerabilities by consulting exploit databases.

Fuzzing, also known as fuzz testing, is an automated method that is ideal for detecting vulnerabilities in software. In this article, you’ll learn what the testing technique involves, its advantages and disadvantages, and the different fuzzing tools you can use.

What is fuzzing (fuzz testing)?

Fuzzing or fuzz testing was originally developed by computer scientist Barton Miller and is a method used to systematically test software for vulnerabilities. Fuzzing does not attempt to interpret the source code of the program. Instead, it treats the software as a black box and its content as given. In fuzz testing, all possible data input interfaces are automatically invoked and fed random data. This process can take hours or even days, depending on the size of the software project you’re testing.

The aim of fuzz testing is to determine whether the program contains the necessary reactions for all possible input variants. Nonsensical or incorrect inputs should be caught by error-handling routines, if possible. If error-handling routines do not exist for certain inputs or do not work correctly, the program can crash. The technique of testing software for all eventualities has long been an integral part of web development environments. For example, cross-browser testing has been used for many years to test web applications across multiple web clients or browser versions.

What types of fuzzing are there?

There are basically three types of fuzzing:

Application fuzzing

This fuzzing approach tests functions such as buttons and input fields in graphical user interfaces or options of command line programs. For example, functions are accessed with an abnormally high frequency or speed, or input fields are filled with too much content.

Protocol fuzzing

Protocols such as the Hypertext Transfer Protocol (HTTP) are used to exchange data on the web. The transferred data must be in a specific format. Protocol fuzzing is used to test the behaviour of the software when incorrectly formatted content is sent. The main goal of this type of fuzzing is to ensure that sent content is not accidentally interpreted as commands and then executed on a server.

File format fuzzing

Fuzzing tools that follow the approach of file-format fuzzing generate corrupted files and then feed them to the target software for processing. Files typically have a standard format such as .jpg for image files so that they can be exchanged between different applications. If the format of the opened file does not match the expected format, it can cause problems. An advanced version of file-format fuzzing allows you to test implemented features such as compression methods for video files.

How exactly does fuzzing work?

Analysing how a piece of software operates is time-consuming. Software often consists of several thousand lines of source code written in a programming language. Branch instructions and integrated functions make the code difficult to follow. In addition, once the source code has been compiled, it has to be decompiled, which is not always possible.

Fuzzing takes a different approach: It generates all conceivable random data of a given type. The goal is to cover as many input variants as possible. Fuzzing is the interface for automated input. This can happen at different levels. For example, in application fuzzing, different data types are mimicked and passed into the software. Possible lengths and data formats of the inputs can also be tested. Protocol fuzzing uses data packets that you create yourself. Another option involves intercepting, manipulating, and returning contents. File format fuzzing uses randomly generated test files. It tests factors such as size, structure, or set flags.

Applications of fuzz testing

The primary application of fuzzing is quality assurance in software development. Because fuzzing is highly automated, it allows you to test software regularly in a prepared test environment. You can integrate this environment into existing agile development processes. You can also fuzz-test previously released software as part of a security audit to find out whether the software has security vulnerabilities.

Of course, fuzzing can also help you to detect exploits. However, it only provides clues about errors in program functions, since fuzzing tools ultimately cannot access the internal structures of the software being tested.

Pros and cons of fuzzing at a glance

Advantages of fuzzing Disadvantages of fuzzing
Allows you to ensure a uniform standard of quality through prepared tests Software handling is sometimes complex
Boosts software stability Additional work
Improves security Does not narrow down the causes of detected bugs
Tools are free of charge and well-established Can be misused by malware developers

The best fuzzing tools at a glance

A variety of free and paid fuzzing tools with different features are available on the market. Here are just a few of them:

  • American Fuzz Lop: Can compile the source code on its own and then test it, which is why this type of program is called a grey-box fuzzer. If the source code isn’t available, it is emulated using QEMU (short for Quick Emulator).
  • Fuzzino: Provides a library for generating input data that can be used to test applications using models of protocols.
  • LibFuzzer: Library that is part of the LLVM compiler infrastructure.
  • ClusterFuzz: Test environment originally developed by Google to test the Chrome browser.
  • Sulley: Provides a collection of tools in the Python scripting language; especially useful for simple test cases such as generating random data.
  • Peach: Highly automated solution for fuzz-testing hardware and software.
  • Powerfuzzer: Provides different attack scenarios, such as SQL injections; features a web-based graphical user interface.
Fuzz-testing tool Developer License Category Unique features
American Fuzz Lop (AFL) Michael Zalewski Open source Application Grey-box approach
Fuzzino Fraunhofer Institut Open source Protocol Suitable for special test cases
LibFuzzer LLVM team Open source Application Integrated into compiler
ClusterFuzz Google Open source Application Integrates tools such as AFL
Sulley OpenRCE Open source Protocol File, easy to integrate
Peach Peachtech Commercial Application Highly automated
Powerfuzzer Marcin Kozlowski Open source Application Integrated into Kali Linux

Possible alternatives to fuzz testing

Depending on the intended application, there are alternatives to fuzz testing that take a different approach and deliver different results. We’ve summarised the most important ones for you below.

Reverse engineering

Reverse engineering is an indispensable tool for security researchers. Reverse engineering is the process of analysing code after decompilation (reverse translation into programming languages) or disassembly (translation into the low-level assembler language). Reverse engineering is a very complex and error-prone process. When successful, however, it allows you to examine all the details of program functionality.

Debugging

Debugging is the process of finding bugs during software development. Programs are executed temporarily in the development environment before they are completed. The programmer can set breakpoints to interrupt the program at certain places during execution. Next, the programmer can view the contents of variables and check them for correctness. Debugging is also used in reverse engineering.

Module tests

Complex software often has a modular design. Module testing is a well-established method for checking individual components in test environments.

Summary

As a programmer, bugs and security vulnerabilities in your software can damage your reputation and cause others to lose trust in your work. Expecting a program to be absolutely flawless is unrealistic, and most software users know this. However, by fuzz-testing your software, you have the opportunity to find bugs before others report them. Beyond that, this technique will boost the stability and security of your software. These are all good reasons to make fuzz testing part of your quality assurance process.


Own your online success
IONOS takes care of your online challenges
Create your website with the perfect domain and the UK's fastest hosting!