Monday, October 8, 2012

An update on Protest (the testing framework that doesn't suck)

Protest (see wiki for more information) is a unit test framework for C++ that is like most other test frameworks, except that it does checks in a innovative way and handles test fixtures really well. I first wrote about Protest here, and since then I've done some more work. Well, actually I rewrote the whole thing.

Why rewriting? Well, the initial solution was a proof-of-concept and not worth spending any effort making production worthy. The rewrite is cleaner, but not as clean as I want it.

Anyway, the version that's in the repository has a proper main function, a memory leak detector, and handles of segmentation faults and similar error. It also has preliminary support for JUnit XML output. None of these features distinguish Protest from the rest of the testing framework pack. The distinguishing feature of Protest is how fixtures and assertions are handled. Here's an example:
suite("my suite") {
  int one = 1; // This is the fixture!
  test("1 should equal 1") {
    check(one == 1);
  test("1 should be less than 2") {
    check(one < 2) << "What? your compiler is broken";
Note that there is only one assert macro, check, which handles all kinds of comparisons. If the check fails, the expression is split into left-hand side and right-hand side before printed.
Also, note how the fixture is setup just as local variables -- this is because fixtures in Protest is local variables. This is much more convenient than class-based fixtures that all (to my knowledge) other test-framework uses

Actually, Protest support class-based fixtures as well. This is done as follows:
struct Fixture { int one() { return 1; } };
suite("my suite", Fixture) {
   test("1 == 1") {
      check(one() == 1);
This is where I'm not yet fully happy with Protest -- I'd like to make it possible for the test-case to provide the fixture with arguments. Something along the lines:
struct Fixture {
  int m_value;
  Fixture(int value) : m_value(value) { }
  int value() { return m_value; }
suite("my suite", Fixture) {
  test("1 == 1", (1)) { check(value() == 1); }
  test("2 <= 4", (4)) { check(2 <= value()); }
That is, the test macro takes optional arguments that is used for initializing the fixture. I'm not fully sure this is possible right now, but I'll give it a go soon.

No comments: