Organic/Evolutionary design

Design up-front is the most established and practiced design discipline in my experience. Most experienced developers/engineers assert that designing system architectures up-front is essential to ensuring that the finished overall application or suite of applications is architecturally sound, stable, inter-operable, extensible, secure, and efficient. The arguments for this design approach are fairly sound and straight-forward. I would argue however, that going this route is detrimental to each of the anticipated attributes listed above. In other words, design up-front ensures instability, quirky interfaces, difficult to extend object models, in-efficient system designs, and hackish end-results. Soundness is a product of all of these and security is a subjective exercise that requires less attention than it normally gets. Assuming that I am correct with this assertion that design up-front is bad for overall system design, what alternative exists that can better it, how does it accomplish this, and what benefits overall can be realized?

Organic or evolutionary software design bucks the trend to figure everything out up front. It follows, in identical fashion, the processes found in nature and in the improvement of all things that are ever built in any field. The idea is simple, build what you need, amend what you need, alter to fit what you need. Every component of new features and enhancements to existing features is based on current AND past needs. Foresight does not come into play. This is an extreme because there are certainly differing levels of foresight that go into building all the mini black boxes that compose a system, but overall, the governing architecture does not exist.

In nature, small incremental changes are introduced. Some of these changes result in no change to the success of the species, other times small improvements are noticeable, and then once every nth thousand iterations, a breakthrough appears that lends considerable advantage to that particular version of the species. The small improvements that lend no real current advantage eventually serve as the basis for this rare breakthrough. The activity that is occurring is called probing; dipping one’s toe in the water to check the temperature before committing oneself to the pool; constructing a prototype to illuminate the merits and detriments of the new design component. The success of mother nature cannot be argued. Evolutionary design is the best design approach available, bar none. Everywhere in the universe, this approach to design may be found. In human terms, our products continue to improve or get worse, but overall the progress is forward. Mother nature and software development learn from probing space and discovering the best form to create or to mutate to.

When developing software, the design up-front approach serves up a pair of hand-cuffs for the duration of the system’s existence, a barrier to becoming what it needs to become. The costs are undeniable to anyone who has ever maintained a legacy system. Business continually presents new needs to us and we continue to hack away solutions to satisfy their requests, never stopping to quantify the costs of not evolving the design three years ago to make adaptation simpler, faster, and more elegant. The costs of barely discernible logic to learn (guess at), modify (break), and enhance (hack) can and do become rediculous and unacceptable. Enter the system re-write in some popular (traditional), established (marketed), and widely used technology like Java or C#.NET.

Why is the rewrite necessary? Because the system never became what it needed to during it’s development. Instead, while the system worked, it became too difficult to amend, too expensive to improve, and too far behind current best practices, frameworks, protocols, and the like.

Test Driven Development, Test First Development, Prototyping, White-board Discussions, et al., are all probing activities that enable us to incrementally evolve our object designs to fit the current picture of the system. The activities we must employ to achieve success with this design paradigm include refactoring, spikes (prototyping), drawing pictures, carding sessions, and many more. The idea is to slowly discover the best design approach today, always understanding that contexts and requirements will alter that design going forward, in incremental fashion. The blockades of cost up-front must be removed. A case for this design approach must be communicated to the business as an eventual cost savings.

There are many benefits that naturally emit from such practices:

• happier developers
• better stability
• better interoperability
• better security
• easier to test systems
• looser couplings
• better adaptability
• better extensibility
• less code
• lower overall system cost
• et al

It’s not easy to convince most folks about this approach but I guarantee that following organic design practices will leave traditional design up-front systems far far behind.

A couple of organizational suggestions:

• Remove the position of programmer and architect. Everyone is titled as some level of engineer and each engineer is held responsible for the overall system architecture. Teach all developers how to incorporate good design practices, insist on good coding standards (including one style guid). Replace engineers who are unwilling to buy-in to this approach with engineers who are fully on-board (show some courage and make the best system you can).
• Practice pair-programming at least 50% of the time, to the extent that is possible. Cross-fertilize the organization with knowledge, ideas, approaches.
• Extend the evolutionary design principles to everything your business does, including improving or replacing tools, practices, and people.
• Trust your engineers and give them the opportunity to save you lots of money on down the line (let them refactor, insist upon it)
• Drive your development with tests. Make all your objects testable. Every object has a minimum of one test case with tests that exercise 100% of the class’s methods.
• Use Smalltalk, focus on business solutions, not syntax, typing, memory management, and performance. I find it interesting that (in my exposure) every C program needs memory management attention, every Java application needs performance attention, and every Smalltalk application “Just Works, and works fast”. Show me different, I suspect it doesn’t exist.

1 Comment