Skip To Content

Empirical Project Management: Agile estimation and being "Done" | Project Research Institute

Empirical Project Management: Agile estimation and being "Done"

"Theory without empirical evidence to test it, is simply a story" (Alberts, 2011, p. 36). Alberts said this in the context of people's ability to estimate the probability of rare events but the observation equally applies to estimating in general. Teams using Lean-Agile take an empirical approach; they test and retest their plans with evidence of progress so that corrective action can be taken as soon as possible.

In their 2002 book Agile software development with Scrum Ken Schwaber and Mike Beedle concluded that "defined" process control models were inappropriate for software development. They argued that the complexity and unpredictability inherent in software development made "empirical" process control models more suitable. The essence of an empirical approach is "frequent and first-hand inspection, followed by immediate adjustment" (Schwaber, & Beddle, 2002, p. 25).

I've discussed mechanisms for inspection and adaptation in my earlier posts on Governance and Complexity. In this post I want to drill down into two other parts of empirical control within Lean-Agile: estimation and measurement.

Most agile teams use variations on the estimating approach from Extreme Programming's Planning Game (see Agile Project Estimating; Agile Project Planning; Beck & Fowler, 2001; Cohn, 2006). In this approach the people who will do the work do the estimates, often as a group to provide a form of validation. The work items being estimated are meant to be small, vertical slices of functionality - typically called "User Stories". The estimates themselves, usually in "Story Points", reflect the difficulty of developing the User Story and consider both bulk and complexity. These estimates are relative measures so a User Story of two Story Points is twice the size of a one Story Point item. There is no absolute definition of how big a Story Point is so each team will have a different scale.

"Velocity" is the agile measure of productivity. It is the amount of functionality delivered each Timebox by the team. Velocity is measured at the end of each Timebox and is the sum of the the estimates for the work items completed in that Timebox.  Because it is derived from the estimates it uses the same unit of measure, i.e. Story Points.

Agile teams should continually update their plan to reflect the actual velocity reflecting the functionality being delivered. This measured velocity is used to predict the velocity for subsequent timeboxes in the Release Plan.  If the measured velocity equals or exceeds the hoped for velocity then the project is on track. If the measured velocity is insufficient to deliver all the scope then management has to intervene. Agile project managers will rarely try to Play Catch-up but will instead advocate reducing scope to fit within the agreed timeframes.

At the start of a project there is no empirical data on velocity so the velocity for the initial work is based on data from previous projects, if it exists, or is estimated. This estimation is prone to the normal causes of underestimation - technical, psychological, and political-economic (Terry Williams). The difference is that with the Agile approach underestimation is revealed quickly. After only a couple of timeboxes, say 2 to 4 weeks, an agile team will have evidence of whether their velocity is sufficient to deliver the essential functionality by the deadline. 

I mentioned that velocity is the sum of the estimates for work completed. Agilists usually refer to this as the work that is "Done". In fact Lean-Agile advocates are obsessed with being "Done". This interest derives from one of the Principles behind the Agile Manifesto which says "Working software is the primary measure of progress". Note that there is no mention of interim deliverables such as specifications and design documents. Value, in the context of software delivery, can only be realised when the software is available to the users. The focus on the working deliverable means Agilists subordinate the creation of interim deliverables to the final product. "Working software over comprehensive documentation" is one of the values of the Agile Manifesto. This subordination means that interim deliverables do not count when measuring progress. This is because the interim deliverables are only valuable to the team, not to the customer. The end goal is working software so that is what is used to track progress.

Lean-Agile is an "empirical" process control with frequent opportunities to inspect and adapt. "Velocity" is at the heart of agile planning and estimating. Velocity is the sum of the estimates for work that that is "Done" during a timebox. Velocity is the empirical evidence that prevents agile plans from being just a story and provides what Scott Ambler calls "fact based governance".

References and Further Reading

Agile Manifesto: Principles Behind the Agile Manifesto

Alberts, D. S. (2011). The Agility Advantage: A Survival Guide for Complex Enterprises and Endeavors. Command and Control Research Program (CCRP).

Ambler, S. (2007). Best practices for lean development governance, Part I: Principles and organization. IBM DeveloperWorks.

Beck, K. (2000). Extreme Programming Explained: Embrace Change. Reading, Massachusetts: Addison-Wesley.

Beck, K, & Fowler, M. (2001). Planning Extreme Programming. Boston: Addison-Wesley.

Cohn, M. (2006). Agile Estimating and Planning. Prentice Hall.

Schwaber, K., & Beddle, M. (2002). Agile software development with Scrum. NJ: Prentice Hall.

Thomas, S. (2008). Agile Project Estimating. It's a Delivery Thing.

Thomas, S. (2008). Agile Project Planning. It's a Delivery Thing.

Williams, T. (2011). The complicated world of under-estimation. PRI.

Williams, T. (2011). When we play catch-up on projects are we our own biggest project risk?. PRI.

The Institute

The newly created Project Management Research institute provides a focus on collaborative research excellence in project management. Read more..