Design Is Learning

Given that, as software designers, we don't know exactly what we're doing, we must be learning what we're doing as we go. By implication, design can be (and, in my opinion, should be) approached as a learning opportunity: the design process should support learning as much as possible about the form of the system under development, as rapidly as possible. An effective design process will include an effective learning environment. (For the record, I also subscribe to Jack Reeves's idea that code is design. As such, I think this idea also applies to software construction.)

Learning is facilitated when there is rapid and easy visibility into the link from action to consequence. This is partly (primarily?) why methodologies like test-driven development, continuous integration, and continuous delivery focus on decreasing the turn-around time from making a change to the system, to seeing the results of that change: by seeing the results of a change while the thoughts that lead to the change are still in working memory, it becomes much easier to see why the change did or did not have the desired effect. Sometimes, an automated test will give the fastest form of feedback from idea to consequence... More often, the fastest form of feedback comes from thinking through the consequences (thinking "if I change this loop's end-condition, it should resolve my off-by-one error" will usually be faster than making the change and re-running the test), with the automated tests acting as a check on the model of the software you maintain in your mind as you work.

Big Design Up Front will often fail because it creates a large lead-time between action (the "big design") and consequence (the built system). On the other hand, no design up front can also break down in cases where systemic consequences of early design decisions do not become visible until late in the construction process, which can give rise to issues that cannot be resolved cheaply or easily. In either case, the fundamental issue is that a decision about the correct sequence of software construction activities has cut off opportunities for determining the most natural way to build the system under development. And we get less maintainable systems as a result.

If, instead, we focus on design (in a broad sense, including requirements elicitation and coding and maintenance) as an opportunity to learn what the shape of our created system should be, then we are encouraged to ask the most significant questions first. (What do our customers want? What are the fundamental concepts in our problem space?) We are encouraged to test our ideas as soon as we can. (Following TDD to test the implementation. Building the Minimum Viable Product to test the market for our ideas.) We are encouraged to reflect on the outcome of having our ideas tested. (Refactoring towards greater insight [Evans, 2004]. Pivoting to serve a different market.) In other words, we are encouraged to train our focus where it will provide the greatest value.

In my opinion, there is no single correct answer to the question of "how much up-front design should we do?". Rather, ask "how can we learn the most about what our system should look like, as quickly as possible?". The answer to that question will be measurable in some quantity of design and implementation work, and that is where you should spend your focus. Software is operational knowledge. To produce novel software, we must gain knowledge. We must learn new things.