Wednesday, January 12, 2011

When all magic goes wrong: std::vector of incomplete type

I have recently been working on an API. I put great effort into separating the implementation from the interface, which in this case means that the header file of the API strictly contains declarations. No executable code at all. This makes it easier to hide implementation details, which is something we should always aim for, especially for APIs.

In C++ there are several ways to hide implementation. One way is to forward declare types and simply use pointers and references to those types in header files. However, when you need to use a type by-value it is not possible to use a forward declared. For example:
class CompleteType { };
class IncompleteType;
class HoldsTheAboveTypes {
  CompleteType value0;     // Ok.
  IncompleteType* pointer; // Ok.
  IncompleteType value1;   // Compilation error!
In my experience, there are usually ways to avoid having types by-value that are implementation details. Usually its a matter of thinking hard about the life-time or ownership of an object. However, when I implemented the API mentioned above I ran into a problem that seemed to be unsolvable.

I needed to have a class with a field of type std::vector of an incomplete type, that is:
class StdVectorOfIncompleteType {
  std::vector<IncompleteType> value;
This code fails to compile, though, giving some error message about "invalid use of incomplete type" (just as the code above). However, IncompleteType isn't used anywhere! So it should compile, shouldn't it?

(Well, I guess you could argue that it should compile if C++ would be designed properly, but it not so let's not go into that...)

The reason the above code doesn't compile is because the following special methods are automagically generated by the compiler:
  • zero-argument constructor
  • copy constructor
  • destructor
  • assignment operator
The default implementations of these special methods are nice to have in most cases. However, in the example with std::vector<IncompleteType> above these default implementation doesn't work at all. It is these default implementation that causes the compilation error, which is very much non-obvious. All (auto-)magic goes wrong.

So to fix the compilation error given above, we simply need to declare these special methods, and provide an implementation to them in a separate .cc-file, where the declaration of IncompleteType is available.

I've been fiddeling with programming for more 15 years (professionally much shorter, though) and I've run into this problem several times before but never tried to understand the cause for it. Today I did.


sithhell said...

I don't know what exactly you are trying to say ... std::vector is missing its template arguments. Having that said, the below code works just fine.


class foo {
std::vector value;

int main() {}

Torgny said...

Thanks for you comment.

What compiler are you using? That code does not compile with GCC 4.3.2 on my machine (and does probably not compile with most compilers I know about). the reason is that std::vector needs a template argument. Could you clarify?

Having said that, I think you misunderstood my post a bit. I was writing about std::vector[IncompleteType], not std::vector without template argument (which is not the same as _zero_ template arguments, i.e., std::vector[], by the way).

To see the problem I describen in my post, try compiling the following code (with "[" and "]" replaced with the appropriate characters):
#include [vector]

class incomplete;
class foo {
std::vector[incomplete] value;

int main() {
foo f;
This code should result in a compilation error due the class "incomplete" only being forward declared. The solution to this problem is what I describe in my post. Sorry if I was unclear.

Torgny said...

Sorry, I just realize that the template argument to std::vector was missing in my blog post. I totally missed that when proof-reading it. Sorry.

I updated the code in the blog post. I hope it makes more sense now. Thank you for pointing out my mistake.

Kim Gräsman said...

Hej Torgny,

I think you're misinterpreting the problem...

It doesn't have much to do with the automatic generation of ctor/dtor/assignment.

Rather, the compiler needs to be able to see the complete type of std::vector's template argument in order to instantiate the template.

As it happens, since all std::vector does is allocate, deallocate and move around objects of its template argument type, ctor/dtor/assignment must be visible in order to instantiate std::vector, but it has nothing to with whether they were compiler-generated or user-defined.

Torgny said...

Thank you for you comment, Kim.

>It doesn't have much to do with the automatic generation of ctor/dtor/assignment.
I agree, the compilation error has nothing to do with whether the ctor/dtor is compiler-generated or user-defined.
However, note the phrase "when all magic goes wrong" in the blog. :) The C++ magic for automatically generating default ctor/dtor/copy/assign, is nice to have in most cases. But in the example described this magic totally blows up in the programmers face and the error message emitted by the compiler is hard to understand...

Technically there is nothing strange with the compilation error; everything is according to the spec. However, the error can be very surprising and the solution is non-obvious for many.

Even worse, these magic methods (defualt ctr/dtor/copy/assign) are, as far as I know, generated by the compiler on-demand. That is, if there is no code that instantiates StdVectorOfIncompleteType, then the code compiles.

As I see it, there are many intricate C++ details that works together here that makes it less-than-obvious that the code is broken, why it's broken, and how to fix it.