Addendum: In addition to the "general resistance to change" numbers that the surveys relate, it's instructive to put them in the context of the damage that's been done in the industry in just a short period of time. For reference, see this table from Robert Charette's 2005 article for IEEE Spectrum.
Agile software development company, VersionOne, has just released their second industry survey on the adoption of agile practices with some illuminating results on how much the project delivery landscape is changing. The survey is the only one of its kind, and with over 1,700 respondents participating from over 70 countries (an increase of over 120%), the State of Agile Survey is providing an increasingly statistically relevant sampling of how practices are changing - albeit slowly and shallowly.
Most interesting to me were the results behind two key questions:
a) What specific improvements have you actually realized from implementing agile practices;
In 2006, 87% of respondents said that increased productivity was their top result, followed by accelerated time-to-market (86%), reduced software defects (86%), and reduced costs (63%).
This year 90% of respondents said that increased productivity was their top result, followed by reduced software defects (85%), accelerated time to market (83%), and reduced costs (66%).
What's interesting here is that adopters are strongly indicating, year-over-year, that they are benefiting their companies through increased productivity while reducing the number of defects in their products. This is a significant indicator that agile practices, be they scrum, XP or other, are delivering against their promises - which to me, is hardly surprising, but to some may seem like we've just tumbled with Alice through the looking glass.
You know who you are.
b) What are the barriers to further adoption of agile in your current organization.
In 2006 about the same number of respondents (20% vs. 21%) indicated that finding qualified personnel and resistance to change were holding back agile penetration. This year, the numbers jumped significantly, with 36% responding that resistance to change was the number one factor, while qualified personnel was not far behind at 34%.
I find these results interesting for two reasons: First, there's still a shortage of agile-qualified people to help facilitate change and coach projects. This means there's a lot of work out there I can do!
Second, the undeniable effect that resistance to change is having on the industry. I've written at length on how waterfall/BDUF is ruining software, costing billions, risking lives and destroying careers. Now I have statistics that show this in black and white.
Other results that piqued my interest included the increased adoption of Scrum over XP (37% vs. 12%) which bodes well for adoption rates as it is easier to understand and practice, and that the key agile practices of iteration planning, unit testing and continuous integration are being taken up.
Overall, this year's survey is showing that the trends toward adoption are moving in the right direction, even if at a glacial pace. I'd be more optimistic about this, but to me it's a case of Nero fiddling while Rome burns - there's a lot at stake while we continue to play the How Long/How Much game and it is readily apparent that education about agile needs to occur at the executive level as well.
Download the 2006 survey here. (PDF)
Download the 2007 survey here. (PDF)