Firmadyne-System for emulation and dynamic analysis of Linux-based firmware.

Introduction

FIRMADYNE is an automated and scalable system for performing emulation and dynamic analysis of Linux-based embedded firmware. It includes the following components:

• modified kernels (MIPS: v2.6.32, ARM: v4.1, v3.10) for instrumentation of firmware execution;
• a userspace NVRAM library to emulate a hardware NVRAM peripheral;
• an extractor to extract a filesystem and kernel from downloaded firmware;
• a small console application to spawn an additional shell for debugging;
• and a scraper to download firmware from 42+ different vendors.

We have also written the following three basic automated analyses using the FIRMADYNE system.

• Accessible Webpages: This script iterates through each file within the filesystem of a firmware image that appears to be served by a webserver, and aggregates the results based on whether they appear to required authentication.
• SNMP Information: This script dumps the contents of the public and private SNMP v2c communities to disk using no credentials.
• Vulnerability Check: This script tests for the presence of 60 known vulnerabilities using exploits from Metasploit. In addition, it also checks for 14 previously-unknown vulnerbailities that we discovered. For more information, including affected products and CVE's, refer to analyses/README.md.

In our 2016 Network and Distributed System Security Symposium (NDSS) paper, titled Towards Automated Dynamic Analysis for Linux-based Embedded Firmware, we evaluated the FIRMADYNE system over a dataset of 23,035 firmware images, of which we were able to extract 9,486. Using 60 exploits from the Metasploit Framework, and 14 previously-unknown vulnerabilities that we discovered, we showed that 846 out of 1,971 (43%) firmware images were vulnerable to at least one exploit, which we estimate to affect 89+ different products. For more details, refer to our paper linked above.

Note: This project is a research tool, and is currently not production ready. In particular, some components are quite immature and rough. We suggest running the system within a virtual machine. No support is offered, but pull requests are greatly appreciated, whether for documentation, tests, or code!

Setup

The following has been tested on a Ubuntu 14.04 machine. Other Debian-based systems should also be compatible.

First, clone this repository recursively and install its dependencies.

1. sudo apt-get install busybox-static fakeroot git dmsetup kpartx netcat-openbsd nmap python-psycopg2 python3-psycopg2 snmp uml-utilities util-linux vlan
2. git clone --recursive https://github.com/firmadyne/firmadyne.git

Extractor

The extractor depends on the binwalk tool, so we need to install that and its dependencies.

1. git clone https://github.com/devttys0/binwalk.git
2. cd binwalk
3. sudo ./deps.sh
4. sudo python ./setup.py install

• For Python 2.x, sudo apt-get install python-lzma

4. sudo -H pip install git+https://github.com/ahupp/python-magic
5. sudo -H pip install git+https://github.com/sviehb/jefferson.
6. Optionally, instead of upstream sasquatch, our sasquatch fork can be used to prevent false positives by making errors fatal.

Database

Next, install, set up, and configure the database.

1. sudo apt-get install postgresql
2. sudo -u postgres createuser -P firmadyne, with password firmadyne
3. sudo -u postgres createdb -O firmadyne firmware
4. sudo -u postgres psql -d firmware < ./firmadyne/database/schema

Binaries

To download our pre-built binaries for all components, run the following script:

• cd ./firmadyne; ./download.sh

Alternatively, refer to the instructions below to compile from source.

QEMU

To use QEMU provided by your distribution:

• sudo apt-get install qemu-system-arm qemu-system-mips qemu-system-x86 qemu-utils

Note that emulation of x86-based firmware is not currently supported, but installing qemu-system-x86 resolves a packaging issue on certain Debian-based distributions.

Alternatively, use our modified version of qemu-linaro for certain firmware with an alphafs webserver that assumes a fixed memory mapping (not recommended), or upstream qemu.

Usage

1. Set FIRMWARE_DIR in firmadyne.config to point to the root of this repository.
2. Download a firmware image, e.g. v2.0.3 for Netgear WNAP320.
   • wget http://www.downloads.netgear.com/files/GDC/WNAP320/WNAP320%20Firmware%20Version%202.0.3.zip
3. Use the extractor to recover only the filesystem, no kernel (-nk), no parallel operation (-np), populating the imagetable in the SQL server at 127.0.0.1 (-sql) with the Netgear brand (-b), and storing the tarball in images.
   • ./sources/extractor/extractor.py -b Netgear -sql 127.0.0.1 -np -nk "WNAP320 Firmware Version 2.0.3.zip" images
4. Identify the architecture of firmware 1 and store the result in the image table of the database.
   • ./scripts/getArch.sh ./images/1.tar.gz
5. Load the contents of the filesystem for firmware 1 into the database, populating the object and object_to_imagetables.
   • ./scripts/tar2db.py -i 1 -f ./images/1.tar.gz
6. Create the QEMU disk image for firmware 1.
   • sudo ./scripts/makeImage.sh 1
7. Infer the network configuration for firmware 1. Kernel messages are logged to ./scratch/1/qemu.initial.serial.log.
   • ./scripts/inferNetwork.sh 1
8. Emulate firmware 1 with the inferred network configuration. This will modify the configuration of the host system by creating a TAP device and adding a route.
   • ./scratch/1/run.sh
9. The system should be available over the network, and is ready for analysis. Kernel messages are mirrored to ./scratch/1/qemu.final.serial.log.
   • ./analyses/snmpwalk.sh 192.168.0.100
   • ./analyses/webAccess.py 1 192.168.0.100 log.txt
   • mkdir exploits; ./analyses/runExploits.py -t 192.168.0.100 -o exploits/exploit -e x (requires Metasploit Framework)
   • sudo nmap -O -sV 192.168.0.100
10. The default console should be automatically connected to the terminal. Note that Ctrl-c is sent to the guest; use the QEMU monitor command Ctrl-a + x to terminate emulation. For the sample firmware above, you will first need to delete the file /etc/securetty from the filesystem to login as root with password password.
11. The following scripts can be used to mount/unmount the filesystem of firmware 1. Ensure that the emulated firmware is not running, and remember to unmount before performing any other operations.

• sudo ./scripts/mount.sh 1
• sudo ./scripts/umount.sh 1

                                Source and download