Wednesday, November 17, 2010

Pain Driven Development

One of the key things I learned from XP and the Agile movement in general is programming for today's requirements not tomorrow's predictions. And, every time you start writing implementation for requirements that may become valid in the future, the Agile folks would shout "YAGNI" (You Aren't Gonna Need It). Applied to code architecture and design, Ward Cunningham summarizes this philosophy nicely with his famous quote "What's the simplest thing that could possibly work?".

But, what happens when today's implementation no longer fulfills today's requirements? In other words, what happens when tomorrow becomes today and requirements grow or change? One example is when 100,000 more users are added to the system, making performance requirements much greater. Another example is when supporting one state is not enough anymore, and the business is now expanding nationally to cover all 50 states.

That is where awareness of pain comes into play. I wrote a blog post about sensitivity to pain a few years back that talks about pain and pleasure when it comes to writing and maintaining code. Developing that awareness of pain is highly important in detecting when to update today's implementation with a higher level of complexity that addresses today's requirements.

Though people have different levels of tolerance to pain, it is a gift that they can feel it as it is often what pushes them toward action. And, in the case of software development, it can point out when today's implementation no longer serves today's requirements and needs to be revised either with a higher level of complexity, or sometimes with a lower level when some requirements are no longer needed.

When I first heard of the YAGNI principle, I remember shuddering a bit and thinking "Isn't it kind of dumb to write code that I will revise in the future when I have to support more states when I could have added in multiple-state support to begin with?"

Well, unfortunately, my thinking was shallow in certain ways. While the argument is logical at one level since following flexible design practices seems to make it easier to handle some future needs, it is much less trivial if I dig a level deeper and include more variables such as whether these future needs ever materialize in the next 2 years, or how much stepping around I am doing while adding new features, mostly because of complexity in code implemented for predicted needs that are not yet valid.

And experience only confirms the concerns I raised above and shows that keeping the code as simple as possible, only addressing today's known business needs, seems to make it easiest to maintain the code and add more features as more needs come up. That is because the code always remains as simple as possible, yet adjusted in complexity only as pain is felt day-to-day.

One example of this that I recently encountered is writing a web feature that relied on data from a web service. At first, the simplest thing that could possibly work was to have it request data from the service synchronously as users hit the site. Later, as requests for data got more complex and time-consuming to fulfill from the service, the implementation became painful to deal with as far as performance, so background caching of service data was added. That is a very good example of what I like to call "Pain Driven Development" :)


Maarten Meijer said...

Al very true, but when does added complexity call for modularization. In other words shouldn't you do your modularization design right from the start, so that components (plug-ins or bundles) can evolve at different pace?

david_hatton said...

Loved reading your blog, I'm a regular follower. Beautifully writter. i spent a year living and working in america in chicago before returning to england. i loved it. keep up the good work

Think you might find my post and articles interesting, feel free to read them.

Andy Maleh said...

Maarten, it depends on the project.

If it is an open-source project that is meant to explore some idea (e.g. building domain-specific languages visually) and needs the help of the community in solving that problem, then the current need calls for having plugin support today.

If on the other hand, it is simply a business application that is not yet foreseeing much need for extension beyond the current user needs, then building plugin support into it that for example requires a lot of XML configuration may simply get in the way of quickly adding features important for today's needs.

Of course, one variable that needs to be factored in is the cost of modularization. With frameworks like Eclipse RCP, modularization comes almost at no cost as it is built right into the framework. However, there would still be a bit of cost in setting up extension points in your application, and that can be delayed till an actual need calls for it.