Why IT Executives aren’t embracing Agile

Wille Faler has written an interesitng post Why IT Executives aren’t embracing Agile, referring in turn to another post on the same subject. Given my background, and my current role, I think I can comment on a technology executive’s opinion of agile processes.

Over the years I have worked on projects using a wide range of processes. Back in the eighties I worked on a team of very bright scientists, writing software primarily for their own use. We had almost no real development process (at best it was managed chaos). This was also one of the most successful software teams of which I have ever been a part. I do not think this is repeatable in most software development environments, because that particular environment had a number of unique characteristics:

1. Really, Really smart people, especially in the domain in which we were working (satellite control).  
2. Staff Retention: nobody ever left. When I joined the team, most of the people there had been there, working on the same code for 15+ years. Most had written the original versions of the code, and invented the algorithms.
3. The developers were the users of the software – eliminating alot of problems of capturing requirements  and expectations. It is pretty hard to have unfulfilled expectations when you are writing software your own software.

Shortly after that, I was was involved in a large military project (10 years, billions of dollars, hundreds of thousands of requirements). Needless to say, we had plenty of process. This was the epitome of the heavy process. Between the company I worked for, and the many subcontracting organizations, I was exposed to many flavours of software process (all of them heavy). I was also involved in ISO 9000 certificatiom programs, CMM assessments, 6-sigma programs and Design for Manufacturability programs (we did hardware, too). One of the things I learned in all of that was that you can have all the process in the world, and still fail. While having a strong software development process (whether it is heavy, agile, or otherwise) may vastly increase your chances of success, it by no mean guarantees it.

In the past 10 years, I have become a great proponent of “just enough process” – trying to take what I have learned from the heavy processes on the military projects, and apply what makes sense in a small, product-oriented environment, while leaving much of the “weight” behind. In the period from about 1998 through 2002 (the last time I directly managed development projects) I was greatly impressed with agile processes. While we never fanatically applied any of the agile methodologies, we did adopt many aspects, such as user stories, iterative incremental development, and test driven development. Some aspect just did not fit our environment (such as pair-programming). We had a fair amount of success using this approach, and many aspects of agile development are still in use.

Getting back to the topic at hand (why IT executives do not embrace Agile processes), from my perspective, agile processes are definitely viable and advantageous in certain contexts. Also, “heavy” processes certainly do not guarantee success. My feeling is that there is a time and place for both kinds of process. As in most things, it is important to have a number of tools at your disposal, and to have the knowledge of when it is appropriate to use these tools. Remember, an idea is a dangerous thing when it is the only one you have (didn’t I use that a couple of days ago?).

For example, I think it is entirely innappropriate to use “heavy” processes in small, commercial product development. Similarly, as an IT executive, I would be extremely hesitant to use an Agile process on a large, complex development project, because I have not seen sufficient evidence of the viability of the approach.

It all comes down to using the right tools in the right situations.