Features are social processes

For a while now I've been thinking about how human languages evolve compared to how computer languages are designed and how that relates to the features of the respective languages. In this post I will ramble at bit about how the meaning of software related terms is defined. I'll also discuss hard and soft features in programming languages and how the community surrounding the language affects, and is affected by, those features.

This is mostly a bunch of ideas and observations that I'm trying to put into words to 1) make me understand them better, and 2) make sure I don't forget them. If you expect a scientific survey, then I'm sorry to disappoint you. Maybe, though, you'll find food for your own thought and ideas.

What's a language?

As I see it, languages (be it the human or programming kind) are mutual agreement between the communicating parts of what a statement means. If everyone have a different opinion of what the following statement means, then it effectively doesn't have any meaning at all since we can't use it to communicate with anyone:

You need wood from a forest to create a fire.

or in computer-speak:

Fire fire = new Fire(forest.getWood());

On the other hand, when we agree upon what this phrase mean, then we can do all kinds of things: discuss its correctness, use it in another context, abstract it to cover more and general cases, write a compiler for it, etc.

For example, the common breed of C compiler accepts a lot of code most C programmers won't. It's the users of the C language that defines the subset of allowed-by-the-compiler-C that is acceptable for us to use. In other words, the C standard can say all it wants; its the C users who in the end defines the (practical) C language. It a bit like if physics would say "Our universe have 11 dimensions. Go use all of them!", but all known users of the universe are three-dimensional beings, thus, the accepted subset of the universe is three-dimensional. Sorry to break it to you, physics, but that's how it is.

Language features are all about what the community around the language make of the language. In a way, language features are just as much a technical aspect of the language as a social aspect of it. Example: is a language feature that's not accepted by the community really a feature, or is it just useless language complexity? More extreme example: a language without users but with extremely powerful features; effectively, does that language have any features at all?

I would also say that anyone aiming to develop a successful programming language (without the backing of a *caugh* Sun huge *caugh* Microsoft corporation) needs to have equally good eye for the technical aspect as well as the social aspect. (S)he needs to understand the social processes involved for getting a community of users who agree (hopefully with the language designer) on how the language should be used. (I think python is good example of such community, by the way).

What about software?

Developing software is also a social process. For example, you get requirements from your customer, you discuss the requirements in order to understand them, and you implement them. Implementing requirement are also a social process: you design the code by discussing it with your colleagues. And what words do you use for doing that?

You use words like object, generic, sort, inheritance, stack, tree, operation, method, message, reuse, client, algorithm, allocation, port, framework, mapping, service, channel, process, decoupled, assumption, resource, provider, input, interface... I could go on forever, but the point is that none of these words really mean anything if we humans don't agree on what they mean. The computer, framework, or programming language has no opinion on what "decouple the client's mapping algorithm from the port allocation" means, but programmers do. It's important it means that same to all programmers involved.

Soft and hard

How does this relate to programming language features? I think there two different kinds of features: hard features that was (deliberately) designed into the language, and soft features that are concepts and idioms that have evolved from using the language.

Hard features are concrete. You can make a list of hard features by reading the language specification. Soft features, on the other hand, are not. They are embedded in the community and to enumerate them you need to vibe with it for a while. Hard features are taught in classes and in books; soft features are learned by hacking, hacking, living and breathing, and some more hacking.

Example: C++ templates. Originally intended to provide better type-safety when writing generic code, like std::vector. The C++ community has then discovered that templates can be used for much, much more (like Boost.Spirit). There are a lot of template code written to implement various kinds of abstract features, e.g., compile-time if, compile-time strings, domain specific languages, etc. The hard feature is "write type-safe generic code". The soft features are "compile-time if", "embedded DSL", and even "it's hard, but you can evaluate everything at compile-time".

The D language took these soft features of C++ templates (e.g., compile-time if, embedded DSL) and integrated them into the core language. Thus, enabled more programmers to use them, because of easier syntax, error messages, compiler support, documentation, etc.

So when a C++ programmer talks about enabling or disabling some piece of code (s)he needs to think about some abstract concept like the enable-if template, while a D programming just thinks "static if". In fact, I don't think the D programmer even thinks "static if" because it's so natural to them, just as the more common "dynamic if" is so natural to all of us. The D programmer probably thinks in entirely other abstract concepts because his/her mind is free from the horrible details of C++ templates meta-programming.

You may argue that our mind is very good at abstracting and that this isn't a problem in practice, but I don't think that's true at all. Example: very few C++ programmer have every done something like an template computing the sinus of an angle, so when they're told to optimize a piece of code doing trigonometry what they'll do is to use some kind of table look-up. A D programmer, on the other hand, will simply slap together a sinus function that can be evaluated statically by the compiler because compile-time evaluation is nothing magic to her/him. (In fact, in D 2.0 match function will automatically be evaluated at compile-time if their arguments are constants).

What I'm saying here is that compile-time evaluation is a (hard) language feature of D but not of C++ (where it is an soft feature). Sure, you can in theory do compile-time evaluation of everything you need in C++ (templates are Turing complete), but not in practice because it's so hard that you actively avoid it. Thus, in most programmers conscious mind, C++ does not have compile-time evaluation. Similarly, you can do object-oriented programming in C, but you probably don't because it's hard and littered with horrible details. Thus, C is in most programmers mind not a object-oriented language. It's possible to do structured programing in Commodore BASIC, but most of us don't think of it like that. Got my point yet? Good.

Ok, so what?

By now you are probably thinking "that's all very interesting, but how is this useful?". Well, I did say that this post was a rambling, didn't I? :)

Seriously though, I don't really think any of this will make you a better software developer, but I think it could be useful if you are developing an tool and there's a community of users around it. Be sure to note what kinds of words the users uses, what features they ignore, what idioms they invent, etc. Integrate the words and idioms into the tool and it's documentation to make the experience for a new user of the application more consistent.

Design patterns are to software development what McDonald's is to cooking

I remember reading the GoF design patterns book and thinking gosh, this is really good stuff. Now I can write program like a real master! I liked the whole idea so much that I went on reading the xUnit Patterns book, and a few more like Refactoring to Patterns.

Looking back on these books now and what I learned from them, I realize that it's not the patterns described in the books that I value the most. It's the reason for their existents; the motivation for using them. For example, the Factory pattern exists because it's often desirable to separate object construction from domain logic. Why? Because it reduces coupling, which means code are easier to enhance, reuse, extend, and test. So when you understand why a pattern exists, then you know when to use and when not to use it.

The problem is that you don't need to understand why a design pattern is needed in order to use a design pattern in you code. Code with a misused design pattern is worse than code without that pattern. As an example, here is some code taken basically directly from an application I worked with:

Thing t = new ThingFactory().create(arg);

with ThingFactory defined as

class ThingFactory {
  Thing create(int arg) { return new Thing(arg); }
}

This is a prime example of code that misuses a design pattern. Clearly, (s)he who wrote this code did not understanding why and when a Factory should be used, (s)he simply used used a Factory without thinking. Probably because (s)he just read some fancy-named design-pattern book.

This is one big problem I see with design patterns. It makes it easy to write code that looks good and professional, when in fact it's horribly bad and convoluted. The Design Patterns book is the software equivalent to MacDonald's Big Book of Burgers: you need to be a good cook/developer already in order to learn anything that will actually make you burgers/software skills better. A less-than-good cook/developer will only learn how to make burgers/software that look good on the surface.

I recently read Object-Oriented Design Heuristics by Arthur J. Riel, and I must say that this book is much better than the Design Patterns book. First of all, it more than just a dictionary of patterns, it's actually a proper book you can read (without being bored to death). Second, the design rules (what the author calls "heuristics") are much more deep and applicable than design patterns. These rules are like Maxwell's equations for good software design. Understand and apply them, and your software will be well designed.

Let me illustrate with an example how I think Riel is different from GoF. Where GoF say "this is a hammer, use it on nails", Riel says "to attach to wooden objects you can either use hammer+nails or screwdriver+screws, both has pros and cons." Sure GoF is easier to read and you'll learn some fancy word you can say if you running out of buzz-words, but Riel actually makes you understand. Understanding is underrated.


But let's give the GoF book et. al, some slack. To be honest I actually did learn something useful and important from those books, and I couldn't do my daily programming work properly without that knowledge:

• favor composition over inheritance,
• separating construction logic from domain logic, and
• code towards an interface, not an implementation.

To me, these three ideas are the essence of most design pattern that are worth knowing and if you keep those three ideas in you head all the time you won't screw up to much.

Oh, there is actually one more idea you should keep in your head or you will definitely screw up big time (literately).

But of course there is one more important thing about knowing design patterns. It helps you communicating with your colleagues. Design patterns defines a pattern language, so instead of saying "...you know that class that instantiates a bunch of classes an connects them..."you can say "...you know that builder class...". Honestly, for me this language is way more important than the patterns themselves.

Actually, there is one thing that I learned from all those design-patterns books (not including Riel). They taught me something important that I could only have learned from few other places: I learned that if an author tries hard enough, (s)he can write a 500 pages book consisting of some common sense and two or three good ideas repeated over and over again. The xUnit Patterns book is the prime example of this. Don't read it. Read Riel.

If it hurts do it more often

As a kid, most of what you did was related to learning. Some call it "playing" or "being curious", but whatever you call it, it's learning in one form or another. And learning hurts. A lot.

So you start going to school to make learning not hurt so much. As a 7-8 year-old you enjoy school, because it makes learning much easier. Also, you can play with your class mates, like say, play soccer. Oh, by the way, playing soccer makes you run faster longer, gives you better balance, gives you better feeling of how flying objects behaves (like a ball), better timing your own movements to others (like your team mates). It also makes you better winning and loosing (you aren't a bad looser, are you? How about a bad winner?), and it makes you better at focusing on a particular task, and making other (your team mates) focusing on the same task as you. It's all learning.

As you grow older you become more comfortable with most things you do because you know them. When you finally leave school to start working as a programmer you lost all will to do things that hurts... because you lost your will to learn (in comparison to how much you wanted to learn stuff as a kid). You lost your will to discover new things.

Stupidity hurts

So, when stated with a problem that forces you to do something you don't know or something you really don't like, you stall. Stall, stall, stall, because you don't want to do it. You want to do something fun. Something you know how to do. Something that makes you feel secure and comfortable. Something that makes you feel less stupid!

Yes, you are stupid. I'm stupid. We are all stupid. No one know everything, so everyone is stupid at something. And we will remain stupid at those things unless we learn how to do them better. But... Ouch! Remember? Learning hurts! So we think to yourselfs "Ooh, I don't want to do that!", or put slightly different "I may get hurt doing that! I'd rather stay stupid!". Imagine if you thought that way as a 8-year-old kid in school... or at the playground, or at the soccer field. We wouldn't get many marathon runners or Nobel prize winners that way, would we?

The cure

It's really easy: dare to be stupid. Dare to ask stupid questions. After a while, you'll notice that you start to ask really hard questions, and before you know it you'll ask questions no one (you know) have answers to. And then, as it happens, people will ask you instead. If they dare to of course (hint: they should).

In fact, I think that people will be more likely to ask you (stupid) questions because they'll seen or heard (of) you ask (stupid) questions. And you know what, people do as other people do. Especially like people they look up to, and since you know so much (from asking stupid questions) they look up to you. Isn't that neat?

The workplace

If you have colleagues who don't mind potentially looking stupid by asking a question, you've work at a good company, I think. But I'm not really someone who read or thinks about this a lot, so I may be wrong. There may be effect here that I'm clueless about. Anyway, my feelings are that I'd be more at home at a company like that than a company that encourages silence and really intelligent questions only. But I'm just me. You are you, and you may feel entirely different about this.

A story of being humble with confidence

I just read a post over at SKORKS, a blog I read every now and then. He writes a bit about The Art Of Unix Programming, which got me reminded of when I first heard of that book. I then remembered reading The Art of Humble Confidence and I felt that I really had to write something along those lines. Here goes.

It was 2006 and I was working with a handful experienced colleagues on a project trying to radically increase the usefulness two large applications by making them work together. This was an extremely interesting time for me and they taught me a lot that I today value very highly.

One day one of my colleagues who was sitting next door to my office knocked gently on my open door to get my attention. He said, "Sorry, am I interrupting you? Do you have a moment?" He was humble as always, speaking slowly and silently to make sure that I wouldn't be anxious about what he'd say next.

"Do you remember what we talked about before?" Even though I wasn't really sure to what he was referring I replied "sure", thinking that I'd get what he means when he starts to talk about it.

While he slowly pulled up a chair and sat down next to me he said, "did you consider what we said about the MessageReceiver class?" I now realized that he was referring to our discussion over a coffee they day before. I nodded, remembering that he didn't really liked how I designed some part of the system we were working on.

Though I couldn't really understand his argument from yesterday I had redesigned it anyway to be more in line with his suggestions. Making a poor design made me feel a bit bad and not understanding why it was bad made me feel worse. But I didn't want to ask him explaining it (again) because I didn't want to look (more) stupid. That would be even worse. Or so I thought.

I guess he realized my anxiety about not properly understanding his design, because he next said "I did a pretty crappy job explaining why that class needed to be changed, right?" He smiling and chuckled, "I was always better at instructing computers than people." We laughed.

"Anyway", he said, "I read this book a bit yesterday and I think chapter 10 explains what I mean much better than I ever can." He handed me the book and said "you can borrow it if you like." He laughed again and added "but not for too long. I need to read it soon again since you ask so incredibly interesting and hard questions." He got up from the chair and said "let's go and get some coffee." He smiled and added "I'm 'starving'".

I grabbed my cup and we walked over to our colleagues offices and asked them to joined us. As we walked to the coffee machine I felt like I was in the greatest group of developers there was. Everyone was working for our mutual goal while having fun, learning, and teaching at the same time.

My colleague had basically just asked me questions, yet managed to tell me something. Yes, he evened managed to tell me what to do. But more importantly, he taught me that you will never know everything and that working in software is a constant process of learning.

Best TED-talk ever

TED is an extremely good place to find interesting (short) talks about various topics. You can literally spend days there watching really high quality presentations ranging from musical performances to science talks. Today, though, I saw something that bets every other TED-talks I've ever seen before.

It was short and witty, it was science and humor, and it was refreshingly different. And best of all, you actually learnt something useful! Stop wasting your time here. Get over there and see it now!

Testing generated code

I started to write this post roughly one and a half years ago. I never finished writing it until now for whatever reason. Here's the post.

<one and a half years ago>
At work I'm currently devloping a little tool that generates Java code for decoding messages (deeply nested data structures) received over a network. To be more precise, the tool generates code that wraps a third-party library, which provides generic access to the messages' data structures. Slightly amusing is that the library consists of generated Java code.

The third-party library is... clumpsy to use, to say the least. Its basic problem is that is so generic/general that it's feels like programming in assembler when using it. Ok, I'm exaggerating a bit, but you get the point: its API is so general it fits no one.

There are several reasons for hiding a library like this:

• simplifynig the API such that it fits your purpose;
• removing messy boilerplate code that's hard to read, understand, and test;
• less code to write, debug, and maintain;
  
• introducing logging, range checks, improved exception handling, Javadoc, etc, is cheap;
• removing the direct dependency to the third-party library; and
• you get a warm fuzzy feeling knowing that 3 lines of trivial DSL code corresponds to something like 60-80 lines of messy Java code.

I'm developing the code generator using Ruby, which have had its pros and cons. The good thing with Ruby is that is easy to prototype things and ou can do fairly complex stuff really easy.

On the bad side is that I reqularly get confused about what types go where. This isn't to much of a problem from a bug point-of-view since test-cases catches the mistakes I make, but it's a bit of an annoyance if you're used to developing Java with Eclipse like I am. The Ruby IDE I'm using (Eclipse RDT) is not nearly as nice as Eclipse JDT, which is natural since an IDE for a dynamic language has less information available for refactorings, content assist, etc.

I've discovered that a functional style is really the way to go when writing code generators, especially when there are no requirements on performance. This is a nice fit, beacuse Ruby encurage a functional programming style -- or rather, it encurage me.

What keeps biting me when it come to code generators is how to test them. Sure, the intermediate steps are fairly easy to test, that is, while things are still objects and not just a chunk of characters. But how is the output tested?

Usually I test the output (the generated code) by matching it against a few regual expressions or similar, but this isn't a very good solutions as the test-cases are hard to read. Also, if an assertion fails the error message isn't very informative (e.g., <true> was not <false>). Furthermore, test-cases like these are still just an approximation how the code really should look like. For example, I've found no general and simple way of testing that all used classes (and no other classes) are imported. So, it is possible that a bug such as:
import tv.muppets.DanishChef;
wouldn't be found by my test-cases, even though the resulting code wouldn't even compile (do'h! The chef's from Sweden not Denmark!).

Ok, this could be fixed by having some test-cases that actually compiles the output by invoking a Java compiler. Not very beautiful but still possible. Even better, the generated-and-compiled code could be tested with a few hand-written test-cases. These test-cases would test the generated code, alright, but would not document the code at all (since "the code" here means the uby code that genreated the executed Java code). This problem, I'm sad to say, I think is impossible to solve with reasonable effort.

This approach is good and covers a lot of pitfalls. However, it misses one important aspect: generated documentation. The code generator I wrote generated Javadoc for all the methods and classes, but how should such documentation be tested? The generated documentation is definitely an important part of the output since it's basically the only human-readable part of it (in other words, the generated code that implements methods are really hard to read and understand).

Another approach is to simply compared the output with the output from a previous "Golden Run" that is known to be good. The problem here is, of course, to know what is 'good'. Also, when the Golden Run needs to be updated, the entire output of the (potential) new Golden Run has to be manually inspected to be sure it really is good/a Golden Run. The good thing with a "Golden Run" is that documentation in the generated code is also tested and not only the generated code.

The approach I'm using is to have a lot of simple test-cases that exercises the entire code generator (from input to output). Each of these test-cases verifies a certain aspect of the generated code, for instance:

• number of methods;
• number of public methods;
• name of the methods;
• a bunch of key code sequences exists in method implementations (e.g., foo.doIt(), new Bar(beer), baz.ooka(boom), etc);
• for a certain method the code looks exactly like a certain string (e.g., return (Property) ((SomethingImpl) s).propertyOf(obj););
• name of the class;
• name of the implemented interfaces;
• number of imports;
• that used classes are imported;
• key frases exist in Javadoc.

In addition to these test-cases, there is are test-cases that simply generates a lot of Java classes and compiles them. The test-cases pass if everything compiles. Finially, some of these generated classes are tested using hand-written JUnit test-cases. For me, this approach worked good and I didn't experience any problems (meaning more problem than normally) with testing the generated code. For the generated documentation, however, there were much more bugs. These bugs didn't get fixed either because it wasn't high priority to fix them. Too bad.
</one and a half years ago>

Looking back at this story now, I would have done things differently. First of all I would not use Ruby. Creating (internal) DSLs with Ruby is really easy and most of the time turn out really nice. I've tried doing similar things with Python, Java and C++, but the syntax of these languages just isn't as forgiving as Ruby's. However, the internal DSL I created for this tool never used the power of Ruby, so it just be an external DSL instead. An external DSL could just as well be implemented in some other language than Ruby.

Second, I would consider just generating helper methods instead of entire classes and packages of classes. So instead of doing:
void setValueFrom(Message m) {
this.value = new GeneratedMessageWrapper(m).fieldA().fieldB().value();
}
you would do
void setValueFrom(Message m) {
this.value = getValue(m);
}
@MessageDecoder("Message.fieldA.fieldB.value")
int getValue(Message m) {
}
where the code for the getValue method would be generated and inserted into the file when the class is built. This way, much less code would be needed to be generated, no DSL would be needed (annotations are used instead), and things like name collisions would not be such a big problem as it was (believe it or not).

At any rate, writing this tool was a really valuable experience for meany reasons that I wouldn't wish to have undone. Considering how good (meaning how much code it saved us from writing) it was a sucess. On the other hand, considering how much time we spent on getting it to work it was a huge failure. As it turns out, this tool has been replaced with a much simpler variant written entierly in Java. Although simpler, it gives the user much more value, though, so its arguable much better then that stuff I wrote 1,5 years ago. My mistake.

Designing (your)Self

I just saw a very interesting lecture by Davig Unger. Its main focus is the programming language Self, which is a prototyped-based object-oriented language. He also talks about other things: getting the right people to work with you, some of his guiding principles, and even a few comments on getting to know yourself.

David Ungar received the Dahl-Nygaard Senior Prize in 2009 for his work in the field of object-orientation, most notably for Self. He currenlty works at IBM Research.

The most important skill

As a software developer you need to have large skill set: knowing the right tools, languages, frameworks, platforms, and so on. To complicate things you need to keep every piece of knowledge updated since the field of computing is changing so rapidly. But what is the most important skill among all these? I argue that it's not knowledge of some key technology that make you more than an average programmer, nor is it how many fixed point combinators you dreamt up, how many open-source project you've started, or how fast you type. If non of these, what is the most important skill?


First let me note that with developer I do not mean someone like Alice who write shell one-liners for her husband Bob to search his photo collection of dogs with hats. I don't mean teenanger Charlie who uses barbed wire and duct tape to hack together scripts to play suitably random mp3s to match his parents' photo slide show from their vacation last summer. What I mean is someone who write code that other developers will maintain, test, read, enhance, etc. In essens I mean developers that work with other developers either directly or through their code.

So, the important part here is work with other developers, which leads me to the heart of this post: the most important skill for a developer is communication. Err, I'm sorry, let me correct that: the most important skill for every developer is effective communication. By communication I mean source code and comments, documentation and specifications, power-point presentations and white-board explanations, mail and phone calls, and so on. I could continue, but I think you've got the point.

A small disclaimer before we continue. I assume that the ones you communicate with want to understand what you say or write. If that's not the case then you have an entierly different problem, and I don't think much in this post can help you. Sorry. Ok, that's that, let's continue.

I've met many very technically skilled developers that, when I tell them I don't understand what you mean, literally repeated what they just said (only louder). Is this good communication? I don't think so. I've also got mails from otherwise good programmers which basically required mind reading ability in order to be understood. Even worse, the mails were sent to 10-50 people, meaning (10 to 50) * (mail length) / (reading speed) * (average speed reduction factor due to lack of mind reading ability) man-hours was spent reading that mail. Effective communication? No.

I think there are two kinds of communication: one-way and two-way. Explaning some intricate technical detail to you pair-programming collaegue is a two-way communication since you (should) instantly realize if you collaegue understands you (by her facial expressions and body language). Phone calls are also two-way since the vocal intonation (should) indicate directly to you whether or not your explanation is good enough.

On the other hand, I consider mail to be a one-way communication (especially if sent to many people) since you don't get any indications when you write the mail if it will be understood or not when received. The same is true for documentation, specifications, code comments and source code. All these kinds of communication is one-way since it takes long time until you get any feedback on if your colleagues understand it or not.

So, how to become a better developer? How to communicate more effectively? For two-way communication I suggest to practice. For one-way communication I suggest... practice, as well actually. That is. Move along, nothing more to see here. Hit the back-button on you favourite browser.

Just kidding. Well, kind of any way. I do think that practice is the one thing will make any noticable difference, so my suggestion is to try to explain whatever technical work you're currently doing to your aunt Mathilda who lives in that small red cottage and knits hats all day long. Oh, it's impossible you say? Your aunt has never heard of numerical stability? She thinks monte carlo simulations has something to do with SimCity? That's doesn't mean you can't make her understand the ideas behind your work. It only means that you have to try harder to explain and simplify it using her language and concepts! Remember that the whole idea with this exercise is to make you better at explaning to other people, not make your aunt capable of challenging the next Turing award winner. To be blunt, she is just an exercise tool. A excerise tool that makes extremely good blueberry pies, though. Yum.

In fact, you could do this exercise by explaning anything. For instance, describe the story of the last book you read, a news article, or the story behind the best photo you ever took. But remember, you're not just telling the story, you're explaning what happend. You should "listen" (with your ears and eyes) to your audience to make sure they understand every word you say.

Some simple concrete tips:

• Be clear about the background. Just because you remember the context, does not mean every one else do. Start the explanation by giving the relevant context. If you have discussed it before with the audience, refer to that. If you haven't, start by giving a hand-waving overview of the problem.
• Use you hands. If you talk about many different objects, place them on different places in the air and "hold" the object you talk about by making a fist on that spot. Much less confusing for those who listen to you. Also, if you need to move your arms around to refer to different objects, you will probably speak more slowly which is also makes it easier to take in everything you say.
• Pop the stack explicitly. Often in a discussion you need to get back to something you talked about earlier. In that case, say "as I said before" before you start talking about it to indicate to your audience that you pop the conversation stack. If you just change the subject people could get a bit confused and miss some parts of your explanation.
• Be utterly clear when something is important. Say "this is the key point", or "this is important". Use short sentences. Repeat yourself. Use short sentences and repeat yourself. Make a pause to let your audience take in what you just said and let them think about it. Use some uncommon phrase or a silly name when explaning this key point that you can use later on to refer to it.
• Speak clearly or stay quiet. If you don't know what to say, don't mumble something incomprehensible. Just shut up. Scratch your head a bit if you feel akward to let the audience know that you're thinking about what to say. People won't mind anyway, unless you do it twice every minute. They will probably thank you for the moment of mental relief. In fact, you can make pauses like this work for your advantage. (Read the bullet above one more time if you don't understand why. (You still don't get it? I'm refering to the 6th sentence, silly.))

That it for this post. But what about the one-way communications? How to improve those? I'll get back to that in my next post on this subject. Until then, if you have any other tips please let me know by leaving a comment!

And oh, before I forget, say hello to aunt Mathilda for me!

Unskilled and unaware of it

I just read the Unskilled and unaware of it post at Successful Software. It's a good post that (should) make you think about how good you consider yourself in various fields such as software development and driving.

I find it a bit funny that there are some skills that you probably don't overestimate your own skill level. Have you ever heard of someone saying he/she is a great gardener when in fact every flower they touch die?

Update: See Dunning-Kruger effect on Wikipedia.

Lessons from a debugger

A few days ago I got undefined method `some_method' for nil:NilClass when I executed a test-case I've just had written for a quite well-tested class. The test-case tested input that the class hadn't been designed to handled, but now I needed the class to handle it.

Knowing that there was a suit of test-cases making sure I didn't break anything, I just added return if (!thing) (where thing is the object the some_method was called on) and run the suit again. And gues what? All test-passed -- including the one I just written that didn't pass. I wrote another one testing a similar scenario and it also passed. I was satisfied.

Why did I add that nil-check? Well, the error appeared deep in a recursive call-chain, and I simply guessed that the recursion should be stopped if thing was nil. I didn't know -- I just guessed.

The point is that I didn't have to know, because if I was wrong any of the class test or multi-class tests would tell me I was. This is all good right? Well, it's not all good. Why? I'll tell you in a moment.

But first, think about a hard problem that you solved by putting in more effort than usual -- persuading someone to test more, parsing a proprietary file format, writing C++ or reading Perl -- any hard problem will do. I'm pretty sure you learnt something really valueable from that experience (even though it didn't feel that way while solving it...). I'm sure we learn a lot from solving any problem by putting in more effort than ususal.

Now, back to my story about the nil fix that made the test-case pass. What did I learn from fixing it by adding a line that I simply guessed should be there? Zip, nothing, ingenting. Of course, this shows that well-tested code is a Good Thing, but this isn't news to anyone. (By the way, note that a Good Thing isn't trademarked, registered, closed-sourced, or anything like that. Its free to use and I encurage you to do so as often as possible. :) Of course, any improvements you make to a Good Thing have to be shared with the community.)

On the other hand, what would I have to do if the class was poorly tested? I would have to understand what the code did by reading, test/run it by hand, debug it, etc, before I added the nil-check. Then I would have to repeate the process to make sure everything worked as before. What would the outcome of all this work be? Probably the same nil-check as before, but I would also understand the code much better. Also, I might picked up some good design ideas, learnt to use the tools better, etc.

Now, I'm not saying that poorly tested code is good. What I am saying is that working with poorly tested code that forces you to fire up the debugger and step through the program will make you debug programs better. I'm saying that working with deep inheritage hiearchies will make you realize that inheritance isn't always a good thing. I'm also saying that reading code with a lot a mutable instance variables and class variables will make you appreciate (and use) 'final' or 'const' more.

To take this a bit further, I think you actually get worse at debugging if you developing in an environment where the code is well-tested, because you never have to debug anything. This is true for me, at least.

I've completely stopped using the debugger. I write test-cases that narrow down the problematic code instead. This combined with a few print-outs is all I need. I think this is easier, and more valuable in the long-run because my efforts are mirrored in a few test-cases that document that there was a bug that was fixed. Would I simply had fired up the debugger and found (and fixed) the problem, there would be no (exectubale) documentation of the bug-fix.

So, one way of becoming a better developer is, I think, to improving quality of untested code because it'll forces you to reason about the program and it's control flow based on scarce information (e.g., logs and stack-traces) among other things. On the other hand, working with well-tested code is much easier: write a test, make the change you have to make to the production code and run all tests. Do they still pass? Great! Code is ready to be checked-in. Good for the project's progress. What have you learnt? Nothing! Bad for you. In some sense.

The Zen of regular expressions

I'm a proud owner, and sometimes wearer, of this (scroll down to Regular Expressions Shirt). On my way home from work today I started to think about whether I really know regular expressions. Sure, I can write expressions that match fairly complex patterns... but do I really know them? I came to the conclusion that I know regexp in the same sense as most seven-year-olds (i.e., first graders) can read and write: they know letter and short words, but not much more.

The funny thing is that if I had been asked this question a few years ago I would have answered of course I know regexps without much thought. Does that mean that I know less about regular expression now than I did then? No, I know more. I now know enough to know that I don't know them.

The Zen of regular expressions:

The first step towards knowing regular expressions is to realize you do not know them.

Since I'm just starting to reach this first step, I cannot tell what the next step will be... or how many steps there are. :)