Getting Agile Part II: Introducing Scrum

Don't procrastinate; do something - no matter how small.
 - Ken Schwaber, Vienna 2004

For almost six years I've been an ardent and enthusiastic advocate of agile software development processes because of their simple, common-sense approach to managing the chaos, uncertainty and complexity that arise in software projects.  My first exposure to agile practices was with eXtremeProgramming (XP), which in time led to my discovery of Scrum a couple of years later.

Initially, I was not sure that Scrum would be that effective as it did not have the same “heft“ that XP did.  However, as I began to learn more about it from the writings of its creators, Ken Schwaber and Jeff Sutherland (in addition to many others), it became apparent that it represented a best-of-breed approach that could be extended to almost any project because of its simplicity.  Indeed, one of the key features of Scrum is that it can be taught to new teams in two days.

In today's post I'll be describing what Scrum is, what makes it tick, and how it represents a positive alternative to how projects are predominantly managed today under “big design up front“ waterfall processes.

So What is this Scrum?

Scrum traces its roots back almost 20 years to an article written by Ikujiro Nonaka and Hirotaka Takeuchi entitled The New New Product Development Game.  At the time, Japanese industrial engineers were beginning to revolutionize the auto manufacturing  sector with their radical approach to managing processes through techniques like “just-in-time delivery“ and empirical management of teams by directly observing them.  Collectively, these processes and techniques came to be known as lean manufacturing.

In the early 1990s Jeff Sutherland and Ken Schwaber began adapting lean concepts and philosophies into a lightweight set of rules and procedures for controlling software projects that revolved around the idea of delivering working software in regular 30 day intervals.  Instead of burdening teams with a mountain of requirements that were notoriously difficult to estimate, they took a cue from the lean principle of “just-in-time“ delivery to break work into smaller segments that could be more easily estimated and implemented.

They then went further to suggest that the best people to make estimates shouldn't be project managers or analysts, but the development teams themselves.  Scrum rules reason that developers are better qualified to make the call about what requirements they can deliver within a fixed period of time.  It eschews "command and control" structures that dictate what needs to be done in favour of surgical strike teams that determine on their own how they are going to meet the goals of the release.

In order to be this agile, Scrum projects are not prescriptive in the sense that there aren't a series of "one time only" exercises that happen at the beginning of the project before any development takes place (a.k.a. "big design up-front").  Requirements gathering, analysis and design become an ongoing activity that is embedded into every release.

Underlying each of these practices is the agile philosophy of active, ongoing collaboration with the customer or Product Owner.  In conjunction with the team, the Product Owner determines the features that they want for each release at the beginning of every sprint, and reviews this functionalty with the team at the end of every sprint.


How Does Scrum Work?

As described in the above section, Scrum is based on 30-day time boxes for delivering slices of complete, potentially shippable product functionality (also called "sashimi" after the Japanese delicacy).  Governing every sprint is a workflow that begins with a collaborative meeting between the Product Owner (a.k.a. client or sponsor), the Scrum Master (former project manager), development team and other involved parties called a "Sprint Planning Meeting". 

This typically takes one day, with the first four hours dedicated to working with the Product Owner to develop a prioritized list of features (called a Product Backlog) that they want delivered at the end of the sprint; after lunch, another four hours is dedicated to the team and the Scrum Master to turn the Product Backlog into a list of development tasks and estimates called a Sprint Backlog.

Once the Product Backlog and Sprint Backlog are estimated, prioritized and pared for the iteration (only as much functionality as the team feels they can deliver in 30 days), they are then used as the input for the development sprint.  For the next four weeks the development team will select tasks from the Sprint Backlog to develop as part of the Sprint Goal.


Self-Organizing and Cross-Functional Teams

When learning about Scrum, you'll often hear the phrase that Scrum development teams are "self-organizing and cross-functional".  This simply means that they are not dependent upon a higher authority for constant hand-holding and direction.  Much like a military unit or platoon, they are given an overall goal to achieve, but how they do it is entirely up to their discretion.  They divide the tasks up among themselves, and are responsible for helping each other out beyond their "specialties".  A Scrum team is collaborative and cooperative, and does not retreat behind job descriptions when work needs to be done.


The Daily Scrum

At the start of each day, the progress of the team is reviewed among the team and the Scrum Master during a brief 15-minute, face-to-face meeting called "the daily Scrum".  At this meeting, the Scrum Master asks three questions of each team member that they then answer to the group as a whole:

1. What did you accomplish yesterday?
2. What are you working on today?
3. What impediments are in your way?

The Scrum Master works with the team to help them help themselves.  He is not there to direct the team, but to guide it and  do whatever is necessary to keep it productive.  The objective of the daily Scrum is to have each team member become familiar with what others are working on, to make commitments to the team for their part of the project and to provide the Scrum Master with a daily window into how the team is working.

Scrum's Artifacts

Being an agile process, Scrum favours working software over comprehensive documentation; this means any design or project documentation must provide business value.  In a  Scrum project you'll generally find several types of artifacts:

• The Product Backlog - a prioritized list of features (usually an Excel spreadsheet) enumerating the features the customer wants in their product;
• The Release Backlog - a prioritized list of Product Backlog items that will comprise a release.
• The Sprint Backlog - a prioritized list of technical tasks created by the development team that describes what needs to be done to turn a selection of Product Backlog items into a functional release.
• The Product Backlog Burndown Chart - a simple graph illustrating the estimated total remaining hours of work for fully implementing the Product Backlog.  The vertical axis measures the total hours, while the horizontal axis defines each day in the Sprint.  This has a high “cool-quotient“ for customers and management as it provides a daily status on how the team is progressing, and can indicate when and where a team is encountering trouble.

Other documents, like UML diagrams or architecture diagrams can be accommodated as long as it can provide business value to the customer at the end of the sprint.  The time taken to produce the diagrams is usually scheduled as a Sprint Backlog item.


The Sprint Review

On the last day of the sprint, a three hour meeting called a "Sprint Review" is scheduled with the team, the Product Owner, the Scrum Master and any other stakeholders or managers.  At this meeting the team demonstrates the functionality that they have implemented and discusses the release with the Product Owner and stakeholders.  This critical part of the sprint allows the client to see exactly what's been developed and if it meets their expectations.  The reactions and impressions developed at the Sprint Review can then be used to drive the next iteration, where the entire process begins anew.


Scrum's Benefits...

As we have just seen, Scrum changes the entire notion of how software is designed, developed and released from a "command and control" model that tries (with some futility) to direct and control a project, to one that is empirical, self-organizing and adaptable.  Instead of releasing software at the end of a protracted development effort, Scrum enforces the agile principle of "release early, release often" with regular 30-day increments of potentially shippable software.

This frequent release rhythm has two benefits:  First, it dramatically reduces the stress normally associated with releasing software because of the subset of functionality it's delivering.  Second, it delivers real "return on investment" or ROI to the customer that cannot be matched under traditional waterfall, top-down management schemes.  As Ken Schwaber explains in his online videos about Scrum on the Conchango website (http://www.scrum-master.com), ROI has become a meaningless phrase because software projects have by and large been failing, or are implemented well after their scheduled release dates.  This has meant that customers are unlikely to recoup their investment for many months due to the heavy, up-front cost.  By limiting releases to 30 day increments, customers do not have to go into deep deficit before they see functionality or business value:

...and Challenges

As this post demonstrates, it is relatively quick and easy to explain Scrum.  Most teams can be instructed on the rules inside of two days.  The harder part comes in the challenges to the prevailing culture within an organization that Scrum necessitates.  Some of the top impediments to getting a team to run successfully under Scrum include:

1. The absence of direction from a project manager and the transition into self-organization;
2. Resisting outside interference by management while in the development sprint;
3. Adoption of new agile best-practice development techniques such as automated unit and acceptance testing, continuous integration, and daily builds;
4. Realizing that the process of developing functionality includes analysis, design, coding, testing and documentation within each task;
5. Understanding what it means to be "done":  Code built, compiled, tested and documented;
6. Fixed-price/date contracts.

For management, the most confusion about Scrum lies with the last point.  This will be a topic in an upcoming post, but I will allude for now that it only necessitates viewing the contract as a series of 30-day commitments (which just so happen to fit most billing cycles) which minimize risk for the client while providing them the security of knowing that they will have a subset of functionality provided on a regular basis.

Summary

In this post I've introduced, with very broad brushstrokes, the fundamentals of Scrum.  We've seen what it is and how, in a nutshell, it works. 

• We've learned about the major players in the Scrum pantheon:  The Product Owner, The Scrum Master and The Development Team. 
• We've learned what constitutes the "engine" of Scrum, the 30-day "sprint". 
• We've learned about how the Product Backlog and Sprint Backlog provide fuel to this engine, and how daily meetings or "Scrums" allow the project and team to be empirically guided toward completion.
• We've learned how Scrum teams differ from their waterfall counterparts by being self-organizing and cross-functional.
• We've also seen how Scrum helps deliver real Return on Investment (ROI) for clients with regular and frequent releases and reduces the overall stress that is the norm with other projects.

In my next post, we'll begin looking at Scrum processes in greater detail, including how the Product Backlog and Sprint Backlog are determined.

Sites about Scrum:

ControlChaos - http://www.controlchaos.com
Conchango - Experts in Agile and Scrum - http://scrum-master.com
Mountain Goat Software - http://www.mountaingoatsoftware.com
Scrum Alliance - http://www.scrumalliance.com
Agile Alliance - http://agilealliance.com
Scrum Development Yahoo! Group - http://groups.yahoo.com/group/scrumdevelopment/

Books about Scrum

Agile Software Development with Scrum - Schwaber/Beedle 2002
Agile Project Management with Scrum - Schwaber 2005