Sunday, September 23, 2012

Protest -- unit testing in C++ made slick

I've tried so many unit testing framework in C++, yet nothing really impressed me. Most are bad clones of JUnit, others are just silly (macros are not always bad). A few get close to what I'd like to have, but all frameworks really fall short on how fixtures are handled.

So, this weekend I decided to see if I could come up with something better. Here's what I got so far. I call it Protest, and it's a unit testing framework that a simple, powerful and slick. Here's an example:
#include <protest.hh>
suite("my first suite of tests.") {
  test("my first test") {
    int i = 2;
    expect(1 != i - 1) << "Intentionally wrong";
  }
}
which will print the following when run:
example.cc:4: expectation '1 != i - 1' (1 != 1) failed [my first suite][my first test][Intentionally wrong].

Fixtures are handled differently in Protest than in most other framework. There is no need to create a separate class or declare any extra stuff:
#include <protest.hh>
suite("tests with fixture 1") {
  int i = 2; // Can be used in all tests.
  test("test 1") {
    expect(i != 1);
  }
  test("test 2") {
    expect(i != 3);
  }
  // If needed, any tear-down code goes here.
}

However, sometimes there is a need for a more traditional approach to fixture (that is, inheriting from a base class). This is also supported in Protest:
#include <protest.hh> 
struct Fixture {
  int two() { return 2; }
}
suite("tests with fixture 2") {
  int i = two();
  test("test 1") {
    expect(two() == i);
  }
}

In addition, Protest supports ignoring test-cases, expected failures, and logging parts of expressions (not yet implemented: will only be logged if test-case fails). It also handles when test-cases crashed (SIGSEGV) and reports the last known line that was executed (usually the last expect that was executed).

Note that I've used lower-case for suite, test, etc, which are macros. These are just development names, and I intend that to be configurable.

3 comments:

Nikola Valentinov Petrov said...

Hi,

I must say that the end result is really impressive. I should have a look how the nested scoping was implemented but the end result is a nice DSL.

I am not a C++ user for some time but I really will look at this next time I write something in C++ and need testing.

Once again - nice job and keep it up ;)

Best, Nikola ;)

Torgny said...

Thank you for your comment Nikola.
Right now, the macro implementation is (very) platform dependent, which is unfortunate. I'm looking into doing it less platform dependent, but I'm not sure it'll be possible. If nothing else work the only solution is to have different macro implementations for different platforms/compilers.

Torgny said...

The latest version in the repo fixes some platform dependencies. Only two left. :)