Suricata-Suricata is a network IDS, IPS and NSM engine.
Suricata is a complex piece of software dealing with mostly untrusted input. Mishandling this input will have serious consequences:
For this reason, we have developed a QA process that is quite extensive. A consequence is that contributing to Suricata can be a somewhat lengthy process.
On a high level, the steps are:
The final QA run takes a few hours minimally, and is started by Victor. It currently runs:
One step of the QA is currently run post-merge. We submit builds to the Coverity Scan program. Due to limitations of this (free) service, we can submit once a day max. Of course it can happen that after the merge the community will find issues. For both cases we request you to help address the issues as they may come up.
- in IPS mode a crash may knock a network offline;
- in passive mode a compromise of the IDS may lead to loss of critical and confidential data;
- missed detection may lead to undetected compromise of the network.
For this reason, we have developed a QA process that is quite extensive. A consequence is that contributing to Suricata can be a somewhat lengthy process.
On a high level, the steps are:
- Travis-CI based build & unit testing. This runs automatically when a pull request is made.
- Review by devs from the team and community
- QA runs
Trusted devs and core team members are able to submit builds to our (semi) public Buildbot instance. It will run a series of build tests and a regression suite to confirm no existing features break. Overview of Suricata's QA steps
The final QA run takes a few hours minimally, and is started by Victor. It currently runs:
- extensive build tests on different OS', compilers, optimization levels, configure features
- static code analysis using cppcheck, scan-build
- runtime code analysis using valgrind, DrMemory, AddressSanitizer, LeakSanitizer
- regression tests for past bugs
- output validation of logging
- unix socket testing
- pcap based fuzz testing using ASAN and LSAN
- traffic replay testing (multi-gigabit)
- large pcap collection processing (multi-terabytes)
- AFL based fuzz testing (might take multiple days or even weeks)
- pcap based performance testing
- live performance testing
- various other manual tests based on evaluation of the proposed changes
One step of the QA is currently run post-merge. We submit builds to the Coverity Scan program. Due to limitations of this (free) service, we can submit once a day max. Of course it can happen that after the merge the community will find issues. For both cases we request you to help address the issues as they may come up.
Post a Comment