Agile Software Development - debate?

There has been a significant upsurge of discussions around the "agile" software development approach (as ill-defined as that may be). A backlash of sorts… Not surprising - I have yet to meet a software development approach that does not go through the typical cycle of discovery, dismissal, early adoption, enthusiastic adoption, boredom, discussion, rejection, obsolescence. The proponents of these approaches will happily quote Gandhi at various stages in the cycle: "First they ignore you, then they laugh at you, then they fight you, and then you win".

But we are talking about development approaches, which offer the interesting couple of challenges:

• the perfect solution costs nothing, presents no risk, and requires no revision
• the perfect solution solves the problems as stated, as understood by all stakeholders when stated and later, as discovered by new stakeholders after the delivery

And this results in the following challenge to any approach: what is the true measure of success it is being judged against?

I have had my share of software development responsibilities, as a developer, as an architect, as a development manager and even as a product manager. I will state the obvious - it's understanding and managing what you are measuring success against that counts, not how you get there. It's so easy to get bogged down into discussing the fine details of the processes and deliverables of a software development approach, and not understand what its motivation is, what it attempts to solve, and how to measure its effectiveness. I have seen more times than I care to count the adoption of a formal approach presented as the solution to problems not understood, invariably with dismal results, leading to the adoption of new approaches with the same lack of success.

Software development is difficult - it is not reducible to pure "mechanical" processes because the amount of creativity involved in it is such at this stage of its maturity that the "human" aspects play a huge role in its implementation. It's impossible to consider software developers as essentially interchangeable for any tasks of consequence. It's impossible to ignore human psychology when assigning tasks, problem solving, emergency triage, etc... It's impossible to set aside the fact that requirements do evolve, that customers do discover new needs, that they want them satisfied quickly or they move on, etc.

Recognizing this, and incorporating it in the approach, is what has made the "agile" approaches ("agile" is umbrella term for so many variations) valuable. There is nothing magical there, and nothing that was not done in many places before. I distill it down to two essential motivations:

• it's easy to lose focus of the true objectives, and the objectives vary frequently, leading to disasters: let's make the objectives those that ultimately matter, keep them transparent, manageable in size and in time, and let's make sure they are reviewed frequently
• it's easy to forget about human dynamics and the social aspects of software development, leading to disasters: let's have effective coordination, let's have common sense, participant-adapted processes and metrics, let's foster collaboration across developers and functions

Keep the focus on your transparent measurable and evolving objectives. Take into account the social aspects of software development. All the processes, deliverables, and recipes in the "agile" approaches are there to support these two key motivations.

Why do I say all this? Simply because the debates on the details of which "agile" method is better than others are noise. The key questions are: are these two motivations important to you? Do you recognize in them the purpose of removing problems you are facing? If so, any of the "agile" methods, adapted to your realities will fit the bill.

Decision Management is connected to all this. Its key goal is to infuse applications with the ability to respond in an agile way to regulatory, market and eco-system changes. The key question is then whether the approach to develop and manage the decisions is truly agile, and how you do assess that.

Carole-Ann has started writing about it, and will be soon presenting at RulesFest 2011 on this very subject.

Interesting stuff indeed.