“Errors using inadequate data are much less than those using no data at all.” – Charles Babbage, English mathematician, philosopher, inventor, mechanical engineer, invented the first mechanical computer (1791 – 1871)

“I always avoid prophesying before hand, because it is a much better policy to prophesy after the event has already taken place.” – Winston Churchill, British orator, author, and Prime Minister during WWII (1874-1965)

“Policies are many, Principles are few, Policies will change, Principles never do.” - John C. Maxwell, evangelical Christian pastor, speaker, and author of 60+ books primarily about leadership (1947 –   )

“We’re entering a new world in which data may be more important than software.” – Tim O’Reilly, founder of O’Reilly Media, supporter of the free software and open source movements (1954 –  ) 

This question below came up recently on our Agile Denver Kanban SIG (special interest group) LinkedIn discussion list.

“I am working for an organization that has ‘Lab Week’ six times a year. Lab Week encourages innovation by setting aside time for engineers to work on whatever they want. My current conundrum is, what do I do with the work items on my team’s Kanban board, specifically those in To Do or Doing, during Lab Week?

It feels wrong to put them back in the backlog, but keeping them in To Do or Doing will affect lead/cycle time. While this is reality, it is only reality six times a year. In the interest of predictability I think I would want to know what lead/cycle time is without the impact of Lab Week and then factor in Lab Week when we hit a Lab Week.”

I had two immediate responses to this interesting question. First, teams in a number of software development and IT/IS organizations are familiar with the basis for this question, and I’m guessing it comes up in a number of non-IT/IS or non-software development contexts as well. 

Second, there is already an upfront discomfort with the idea of moving workitems back to the “backlog” that have been “pulled” into the ToDo (or Ready, On Deck, etc.) workflow process states. I’m wondering what signal is this discomfort providing to us? I’m guessing the discomfort is even greater with the idea of moving workitems back to the backlog that have made it to some state of Doing in the workflow process. In essence, this would be “resetting the clock” for a number of workitems already considered work in progress for some period of time, and with expectations I’m sure they would be completed within some known SLA.

Just what I like, “real world” questions in particular when they come with some “pain points” clearly identified upfront. As I continued to think about this one more, it felt like a “perfect” opportunity to raise what I feel is a related question and discuss them some together in this post. When discussing metrics as part of measuring and managing flow, the question below is one I raise often with others who are applying the Kanban Method to their workflow processes.

“In your workflow process context: Does Data Shape Your Policy or Does Policy Shape Your Data?”

How might we in the case of a “Lab Week” produce the metrics we want and visualize this information without any “discomfort” of doing post collection adjustments or moving (resetting) workitems back into the backlog? What can we learn from thinking about and responding to this “Lab Week” question that applies beyond the original basis itself in how we develop policies for managing our workflow processes?

What is a Lab Week?

A number of organizations I know are fairly mature in their software development or IT/IS workflow processes and technical practices. It’s interesting to me too that a number of them do practice some form of activity typically called a “hack-a-thon, hack-fest, hack-day, code-fest”, and it looks too like they are called “Lab Week” by some. These events typically last anywhere from a day to a week and are essentially for a fairly similar purpose and consist of fairly similar activities. These include educational and social purposes, with an intent to create usable software, but often are more immediately focused on exploring new languages, APIs, operating systems, tools, platforms, or new product development and innovation efforts.

Should We Adjust Data to Account for a Lab Week Policy?

In these contexts I can see how it “feels wrong” to move workitems from ToDo or Doing back to the backlog. Yet, if there was a specific and immediate business need (a force, a problem, an opportunity) to be addressed by deriving information without the impact of a Lab Week, I see too how one might explore ways to do a post adjustment, a “correction”, of the lead times for these workitems by one week. However, for the moment, I want to consider doing something different than either of these.

From the original question, the initial concern appears that minimally seven days of “idle time” are being added to the lead times of any workitem that is in ToDo or Doing states of the workflow process when a Lab Week occurs. I wondered how might this concern, due to a Lab Week policy, appear in any of the metrics and charts that I use often? I’m also inferring a need to know something about these metrics of interest when the additional seven days due to a Lab Week are not a factor. So, how might these metrics influence policies that don’t require us to make “corrections” to collected data, while still showing us the reality of our system with Lab Weeks occurring six times a year?

A Run Chart and Scatter Plot Data Perspective 

Assume a team was using a run chart showing the number of workitems being completed each reporting interval and had this data plotted over several months now. Along the x-axis I’d see the reporting interval end dates plotted and vertically above each of these labels I’d see a point plotted at the appropriate height relative to the scale of increasing incremental counts plotted along the y-axis. Essentially a simple line chart of sorts.

In my experiences with real project data from several software development contexts, these points plotted at each reporting interval when connected created a line that oscillated up and down yet producing a fairly consistent pattern that trended neither upward or downward over time. This consistent alternating pattern gave me a fairly reliable indication of the number, that is a consistent range, of workitems I came to expect to be completed per reporting interval. For example, in one context this range was 7 to 17 “XP story sized” workitems completed per two-week reporting interval.

Assume a team was also using a scatter plot showing lead times for completed workitems over time. In similar fashion, the dates plotted along the x-axis were the end dates of the same two week reporting intervals used on the run chart. Plotted along the y-axis were increasing incremental number of days, in this case a scale of increasing incremental days (lead time) required to complete a workitem.

Again, based on my experience with real project data from several software development contexts, I’d see a large number of these points plotted densely at the lower end of the y-axis and then be less dense at the upper end of the y-axis. That is, this distribution of lead times gave me a fairly reliable indication that 70+ percent of my workitems would complete in seven “calendar” days or less. (Note: see my post here for examples of run chart and scatter plot; the last “bar chart” in this other post essentially is plotting similar info used to create a “run chart”.)

What might the effects of a Lab Week policy look like on my run charts? Well, with a two-week reporting interval one result may be no noticeable difference. That is, even if I didn’t have data well established before the Lab Week policy was implemented, the run chart might show reporting intervals at the low end of the range appear about as often where Lab Week occurs as where it doesn’t. You might be saying, “A two-week reporting interval that contains a Lab Week has to visually show some ‘dips’ on a run chart.” I wouldn’t be so sure.

That said, for now let us assume a run chart does show two-week reporting intervals containing a Lab Week are the ones consistently on the low end of the range. What problem does this cause for us? Looking at it from a delivery perspective, it certainly doesn’t make anyone want to have a workitem in progress during Lab Week. It probably also causes a desire by some to push on or for their workitems as a Lab Week approaches.

What might the effects of a Lab Week policy look like on my scatter plots? Well, with a two-week reporting interval, where most workitems complete in seven days or less, any workitem in ToDo or Doing when Lab Week occurs will likely see their lead time doubled. That is, there would be fewer points plotted in the reporting intervals with a Lab Week (fewer completed workitems) and more “scatter” of the points plotted in reporting intervals just after those with a Lab Week (due to the longer lead times for workitems “idled” during the prior Lab Week).      

What Would We Like Our Data to Show? 

Would it be preferable for our charts to show that Lab Week has little to no impact? How might we make this happen if we were in fact seeing the “dips” and “scatter” occuring as described above for the run charts and scatter plots respectively? Instead of “correcting” data, is it possible we could make some tweaks and adjustments (“corrections”) to our policies that would “shape” the data into what we’d prefer to see? 

A Lab Week six times per year works out to implementing one about every seven or so weeks. Staying with the two-week reporting interval for our discussion, every fourth reporting interval that contains a Lab Week, consider doing the following:

1. Implement the Lab Week as the second week of the reporting interval.
2. During the first week of this interval, if you complete a workitem, don’t immediately pull the next workitem from ToDo, instead first try to help on any other workitems in progress (Doing) to complete them if possible before theLab Week starts.
3. If you can’t do #2, pull a workitem from ToDo only if there is a high probablity you can complete it before Lab Week starts.
4. If you can’t do #3, start your Lab Week early, and end your Lab Week early.
5. If you finish Lab Week early, don’t immediately pull the next workitem from ToDo, instead first try to help complete, hopefully before the reporting interval ends, any of the workitems in progress (Doing) but idled due to Lab Week.
6. If you finish Lab Week early and can’t do #5, try to pull the workitems that have been the longest in ToDo (where typically these would be the larger workitems passed over just before Lab Week started in favor of completing a few more smaller workitems). 

Data Shapes Policy, Policy Shapes Data!

What do you think? Do the “dips” in a run chart guide us in shaping the “corrections” (tweaks) to policies that would in turn smooth the flow of workitems completing each reporting interval that contains a Lab Week? Could tweaks to these polices in turn shape the run chart and “smooth” the dips in reporting intervals that “contained” a Lab Week? 

Does seeing the “scatter” in the scatter plot guide us in shaping the “corrections” (tweaks) to policies that minimize the spread of workitem lead times completing just after each reporting interval with a Lab Week? Could tweaks to these polices in turn shape the scatter plot to reduce the the spread in reporting intervals that occur just after a reporitng interval containing a Lab Week? 

Insights from your data should be used to shape (guide changes to) your workflow process policies, and in turn tweaks to your policies should be used to help shape (create the change in) the workflow process data you wish to see. In particular, to smooth the flow (throughput) of completed workitems and minimize the lead time spread of completed workitems, with both of these contributing to the overall “predictability, sustainability, and quality” in your workflow process. 

Who Said Anything About Lowering WIP Limits?

Okay, now I’ll mention WIP limits. Were you surprised I hadn’t mentioned them until now? Well, what do you think, could lowering the upper end of your ToDo column WIP limits as you enter a reporting interval with a Lab Week help? Would this be a policy tweak worth considering?

You might think this is being a bit “manipulative” by choosing to “start the clock” on fewer workitems during these reporting intervals with a Lab Week. But isn’t this is a fairer expectation to visualize? That is, wouldn’t it be better to communicate “earlier” that fewer workitems will likely “enter” a work-in-progress state in the workflow process during the reporting intervals with a Lab Week?

Isn’t it more appropriate to leave a few more workitems in the “backlog” during these intervals than to consider moving them back after they reach the ToDo or Doing states of your workflow process, or adjusting their lead times after they complete? It might be we simply adjust the replenish policy to reduce the frequency, putting the ToDo column on a bit of a “diet” during these reporting intervals with a Lab Week. Or a combination of both. Either way the net effect is to lower “work in progress” as we head into a period of “lower capability.” Does this sound familiar?

Closing Thoughts

Near the top of this post I suggested that what we learn from our response to the original question could extend beyond the original basis. Lab Week represents a time where we know, or at least we assume, in advance that our team’s normal (capability) will be challenged. What other times do we know a similar challenge will occur?

What about when a two-day holiday occurs next to a weekend? Everyone on the team getting a “four-day weekend” can be a big hit to the workflow throughput and workitem lead times. I know these are also popular times for some to request an additional three days off and turn a two-day holiday into a full one-week vacation. What about times of the year when vacation requests are heaviest, like spring breaks for those who have children in schools, or the weeks between Thanksgiving and the New Year here in the U.S.? What about those times set aside for “annual or quarterly planning” or even “performance reviews” and the like?

Even if you don’t have “Lab Weeks” you could very well have something that causes a similar challenge. Think about these policy tweaks in these contexts also, consider how they might help, and more importantly think about why they might help. 

Take care,

Frank 

“Science and technology revolutionize our lives, but memory, tradition and myth frame our response.” - Arthur M. Schlesinger, Jr., U.S. historian. “The Challenge of Change,” New York Times Magazine (July 27, 1986)

 “A myth is an image in terms of which we try to make sense of the world.” - Alan Watts, British-born philosopher, writer, and speaker (1915 – 1973)

“What is the Kanban Method?”

• It’s not about replacing your current software development process.
• It’s not about changing or removing your team’s current titles and roles or adding new ones.
• It’s not about a process that is only for support, maintenance, or “dev-ops” teams.

Does any of the above surprise you?

If so, maybe it would be useful and helpful to take a closer look at the Kanban Method from a different perspective.

Kanban Leaders Retreat (KLRUS) 

In November 2012 I attended the Kanban Leaders Retreat (KLRUS) in San Diego. Over two days there I had an opportunity to take part in a number of break out sessions with several other leading coaches, trainers, and practitioners of the Kanban Method. In one of the sessions we focused on identifying some of the “surprising” myths and misconceptions we were hearing about the Kanban Method while coaching, training, and working with others. As summarized by some participating, “there really is a lot of surprising ‘stuff’ out there!” In the context of this post I’ll limit myself to providing a taste of our discussion there at KLRUS. In particular, focusing for the moment on a misconception I hear often, the Kanban Method: It’s just Scrum with tweaks.

If you’re just beginning to learn for yourself about the Kanban Method, I hope this post provides a useful foundational perspective to keep in mind as you continue learning, researching, and reading articles, posts, etc. For others more familiar with the Kanban Method already, I hope it provides a little insight into the misconceptions you’re likely to run into, if you haven’t already run into this one, and how some in the community are discussing them in ways we feel will help others see the Kanban Method from a more useful and different perspective. 

Yes, It Starts As Your Current Process with Tweaks, But There’s More!

It’s not all that uncommon for me when listening to others discuss the Kanban Method to hear some variant of “We’re thinking of switching to kanban because then we won’t have to do (fill-in-the-blank)” or “We’ve switched to kanban and now we don’t do (fill-in-the-blank) or “We don’t have (fill-in-the-blank), so we think switching to kanban will be a better fit.” When I hear these types of statements where the context is someone on a team that was using Scrum before applying the Kanban Method, I also often hear their conclusions and some of those “blanks” summarized as, kanban is “just Scrum with tweaks”, or often more specifically it is “just Scrum without iterations”, it is “just Scrum without time boxes”, or it is “just Scrum without estimation.”

Yes, the Kanban Method is about starting with your current process, in your current context, and continually improving it. So, if in a software development context one was using Scrum before applying the Kanban Method, I can see where they may have improved an aspect in their context of their current process by stopping something that is typically considered part of the Scrum process (or by starting something that is typically not considered part of the Scrum process). For example like not continuing to use the same uniformed fixed-length time-box iteration interval across all their process activities. Or, adding the use of lead time and throughput as measures of capability for their current workflow process, then, over time finding they no longer need to do as much upfront estimating using point sizing, ideal days, etc. 

When I began learning and applying lean-agile principles in late 2004 and more specifically the  Kanban Method in late 2007, my own personal experience, as an active member of teams delivering production software and meeting business delivery targets, included very similar experiences as those mentioned above (see other posts here and here). So, from someone’s off-the-cuff observer perspective of what our teams were doing, I can see how these “just Scrum with tweaks” like statements might come about and make some sense to those saying them.

However, when you learn the “Kanban Method” is not a software development process at all or is not only for IT/IS support, maintenance or dev-ops”, then from another perspective it also makes sense to see how a statement like “just Scrum with tweaks” is limiting in describing the use or reasons for applying the Kanban Method in your context as a way to improve workflow processes. Limiting in the sense of not fully leveraging its benefits. Probably worse though, sometimes, I think this limited perspective results as well in a stunted perspective and leads to misapplying the Kanban Method. There is more!

The Kanban Method – A Change Management Tool, a Process Improvement Lens 

When you look closely at the Kanban Method (see here), you’ll not see it specifically requiring or suggesting the use of iterations, uniformed fixed length timebox iteration interval, upfront estimating, specific roles (ex. scrum master, product owner, etc.), or specific technical practices (ex. automated unit testing, test-driven development, pairing, continuous integration, continuous deployment,  etc.). A lack of specifically mentioning or prescribing these should not be construed as an indication the Kanban Method precludes any use of them either.

In your context for a software development or other IT/IS related workflow process, many of these practices may and will contribute to improving your ability to predictably deliver valuable and quality services and products for the benefit of your end users, customers, and business stakeholders. However, you may also find some don’t, or over time as you continue to learn more about your context and workflow process, you may find it necessary to tweak some, extend some, innovate on some, drop some, and add some new ones to sustain (not fall back or degrade) and improve your capability.

Ask yourself what about your current workflow process hasn’t improved, that is, changed over the last year or two? While all change is not improvement, all improvement does require change. Could changing from using the same uniform fixed-length time-boxed iteration interval currently used across some set of your workflow process activities allow you to improve some aspect or address some issues in your current workflow process and context? Notice too though the Kanban Method isn’t specific to just IT/IS or software development workflow processes. It can be applied outside of IT/IS and software development workflow processes in your  organization as well, and most likely should be. So what is it?

Pull, Flow, and Improve 

I like to personally view the Kanban Method as a “lean and lightweight” lens, a tool for viewing a workflow process as it is today, then using this “different” perspective to learn how one might explore, discover, and identify “new” opportunities for improving the process through change, experiments and validation. While a bit more verbose, this personal view aligns and reflects how other leaders in the community describe it, the kanban method itself is not any specific (workflow) process, rather it is a change management tool, a catalyst for continuously improving any current (workflow) process. So, as of late, I’m working on frequently reminding myself and suggesting to others to think of it more as a tool for exploring, discovering, and identifying the most beneficial opportunities for continuously and incrementally changing (tweaking) and improving your current workflow process, starting from where it is today, and to think less in terms of “switching or replacing” any current workflow process.

Try not to let any initial low hanging fruit (tweak) type improvements that you might get simply by simulating (copying) process changes based on observing other teams using it, lure you into the trap of “limited” thinking about what the Kanban Method is and how it can be applied. As you go forward learning more about the Kanban Method yourself, and apply it to your specific context and workflow process,  go beyond observing just “what” tweaks (change) other teams are doing in their similar workflow processes (ex. decoupling the cadence or frequency of some set of activities so they don’t all always have to align to the same uniform fixed-length time-box iteration interval).

Get to “how” they came about making those tweaks (ex. the problem, the impediment, the cost, the delay, the quality issue, the pain point, etc. that motivated the change in their specific context and workflow process, and the tool they used to identify these). Get to “why” they thought the tweak would be an improvement, and the criteria and measures they developed and used in validating it (the tweak, the change).

Closing Thoughts 

I’ll close with a few more thoughts from elsewhere that I think reflect the comments above and contribute more to developing a helpful perspective and understanding of what the Kanban Method is and what it is not. I don’t recall exactly the source for this first one, but I read somewhere that Toyota, in times past, didn’t worry much about others including “competitors” coming in and “observing” their processes. For similar reasons, simulating or copying the process of a team that is using the Kanban Method is very different from learning how to use the Kanban Method to improve your workflow process. 

The source for these next ones are scattered excerpts from book “The High-Velocity Edge” by Steven J. Spear that I’ve captured below. I feel they also resonate with a similar helpful message as you consider, learn, and apply the Kanban Method.

“Though many firms had embraced various tools…they still never caught-up.”

“These firms had picked up the visible tools of high-velocity organizations…but they had not understood what these tools were for.”

“The most compelling theme is that when the solution to a problem is discovered, the discovery process itself must be conveyed along with the solution.” 

The Kanban Method  is not just Scrum with tweaks, though it might look like that to some, in particular to a casual observer. Again, take a close look at your current workflow process. Does it seem the easy and early improvements have been made and you feel you’ve now  plateaued some lately? Is it possible you’ve only simulated or copied what others have done but not figured out how to tweak it appropriately for your own context to get similar improvements that you see in their context? Did you only look closely at their solutions but not dig into the “tools” and the discovery process they used in getting to the tweak they made to their workflow process, be it Scrum or something else? If so, maybe it is time to take another look at the Kanban Method with a new perspective. Maybe it is time to go beyond being a casual observer of the Kanban Method, and if you do, I think and hope that you find, There is more!

Take care,

Frank

“Queues lie at the root of a large number of product development problems. They increase variability, risk, and cycle time. They decrease efficiency, quality, and motivation.”

“Large queues form when processes with variability are operated at high levels of capacity utilization. In reality, the misguided pursuit of efficiency creates enormous costs in the unmeasured invisible portion of the product development process, its queues.”

“Since high capacity utilization simultaneously raises efficiency and increases the cost of delay, we need to look at the combined impact of these two factors.”

“We have already pointed out that companies do not manage product development queues. What do they manage? Timelines...In contrast, when we emphasize flow, we focus on queues rather than timelines. Queues are a far better control variable than cycle time because, as you shall see, queues are leading indicators of future cycle-time problems. By controlling queue size, we automatically achieve control over timelines.”

- Don Reinertsen, 2009 – The Principle of Product Development Flow: Second Generation Lean Product Development

If you’re new to or rusty on CFDs, it would be helpful to review “Basics of Reading Cumulative Flow Diagrams” first. This earlier post covers basic definitions and the mechanics of reading WIP (work-in-progress), lead-time, and an average completion rate (throughput) from a CFD. It also includes links to external sources, one showing a basic example of how to create a CFD using MS-Excel, and another providing a quick overview of their use in the kanban method context. From this foundation I’d like to shift here toward a visual analysis perspective, exploring a few contexts where a bottleneck in our workflow is more or less present and see how it might appear on a CFD.

To be clear, I don’t think CFDs are the only way or “a best way” for spotting a bottleneck in your workflow. If you’re adequately visualizing your workflow on a board and diligently measuring and managing work items as they flow through your process, you’re well on the way to being able to identify where any bottleneck might be occurring. (Note: see my Kanban Board Design Primer under Resources/Nuggets here). That said, don’t under-estimate the value of using the CFD as well for learning about what happened, is happening, or will happen in your workflow. It can be a very effective tool for creating an easily accessible visual historical record and a trending perspective of a workflow in your context.

What is a Bottleneck?

Let us get a shared understanding of the term “bottleneck” by first looking at these selected Wikipedia definitions:

• Metaphorically a bottleneck is a section of a route with a carrying capacity substantially below that characterizing other sections of the same route. This is often a narrow part of a road, perhaps also with a smaller number of lanes, or a reduction of the number of tracks of a railway line.
• In engineering, a bottleneck is a phenomenon by which the performance or capacity of an entire system is severely limited by a single component.
• A bottleneck in project management is one process in a chain of processes, such that its limited capacity reduces the capacity of the whole chain.

As we continue our discussion, a key point from these definitions to keep in mind is the “singular” characteristic. Additionally, I’ll infer a “stationary” characteristic as well for the bottleneck throughout the time reflected by the example CFDs that follow.

A Bottleneck Without WIP Limits

Figure 1 – No WIP Limits Used

In the first example, the CFD is derived from a workflow that didn’t utilize WIP limits (see Figure 1, click image to enlarge, and note for this post I’ve chosen to run CFDs with latest date intervals on the right and earlier date intervals scrolling off to the left, which is more common, but again it’s a preference, not required). The workflow states shown include Design, Code, Test, Deploy, and Done. In this first example, do you clearly see work items are arriving into Design at a rate significantly greater than they are exiting from Deploy and entering the Done state?

What else stands out here? The width of the Design and Deploy states of the CFD remain relatively consistent over the reporting intervals, and they’re also narrow relative to the other states. In contrast, the Code and Test states get wider over time. What does this tell us? Recall, no WIP limits are being used here, so in this context, upstream of a “bottleneck” work items will back up, over time taking longer and longer to get through this point, and through the entire workflow end to end.

But which process state is the bottleneck? Clearly the Test state shows the point in the workflow where the greatest backing up is occurring, or more precisely appears to be the “single” component that is severely limiting the entire system. But what else do you see in this CFD?

Note the “flat-shelves” that occur on the CFD chart line between the Test and Deploy states (ex. look vertically straight up from Reporting Interval 57). What causes these to appear in this CFD? The “shelf” at Reporting Interval 57 shows a “purple” gap exists for the Deploy state, but there is no such gap (no purple) for the Deploy state at Reporting Interval 81 and 83. Is there any WIP for the Deploy state near Reporting Interval 81 and 83? Do you see any similar “shelves” on the CFD chart line between the Code and Test states (ex. Reporting Intervals 20 and 26)? Do shelves occur less frequently between the Code and Test states and do any show no WIP (no “green” gap) existing in the Test state?

When a shelf occurs, no work items are moving from the upstream workflow process state to the adjacent downstream state. When WIP limits are in place, it’s possible that an upstream state could be “blocked” from moving a work item downstream even when all the work for that upstream state is completed for one or more work items. But for this example, since there are no WIP limits being used, nothing is blocking the Test state. When no work items are passed from Test to Deploy during a reporting interval, it is due to none having their testing “completed”  during that reporting interval.

We also see the Deploy state is frequently “starved”, with no work items (no purple, no WIP) for several reporting intervals. When a process state frequently has no WIP, is this another characteristic to look for in a CFD when determining where a “significant” limiting factor might exist for the overall system?

Note: It would’ve been time consuming to collect real world project data for each CFD example, and no simple task to manually generate data for a useful number of work items in each example as well. Instead, I utilized an MS-Excel based kanban simulation tool (developed by Mark Robinson, available at Excelville.com). Using Mark’s tool I was able to easily and quickly model the processing of 100 work items through a fixed number of workflow process states (Design, Code, Test, Deploy, and Done) for a number of different runs.

The desired example models were created by adjusting the WIP limits and the level of effort limits in each process state of the workflow for the work items. These settings and summary statistics generated for individual runs were captured and placed above each CFD example. Since the output for each run required additional processing to create specific counts and reporting intervals needed to create a CFD, I used an MS-Excel companion workbook to code VBA routines to automate with a button click all the additional processing of the raw data, along with capturing the parameter settings, and generating the CFD.

By reviewing these settings for Team Size, WIP Limits, and Min, Max “level of efforts” for the various workflow states, and understanding the tool simulates rolling of dice to process work items, and being fairly confident it uses FIFO with no capacity allocated across functions or carried over reporting intervals, you should be able to get any necessary appreciation for how I used it to model the various examples.

Using a WIP Limit to Protect and Buffer Your Bottleneck

Figure 2 – Protecting & Buffering a Bottleneck

Picking up from our first CFD example above, I added a single WIP limit of 2 (hint: based on team size, min, max settings for test, and simulating dice rolls) to the Test workflow state (see Figure 2, click image to enlarge) . This was in essence to model the accomplishing of a couple of goals.

The first, was to protect the Test state from being overrun. Why? As per the quotes at the top of this post, in a real software development context, that ever growing Test backlog causes problems. Secondly, to provide a buffer for the Test state, against the inherent variability in work item size and arrivals, to ensure it always has work to pull from, since starving the “bottleneck” of a system can only lengthen the time it takes to get work items completed.

The resulting CFD shows a markedly reduced and uniform width for the Test state indicating the added WIP limit helped reduce the overall time a work item spends in this state. However, notice we still see shelves occurring between Test and Deploy, and where Deploy WIP goes to “zero” (starves). Why?

Applying a WIP limit and observing lead times decreasing for the Test state does not mean the number of work items per reporting interval (the rate)  that “complete” (exit) the Test state increases (automatically). In a real world software development context, this single WIP limit would likely help in a number of ways to complete work items in less overall time for the Test state. One obvious way is the WIP limit “controls” when the clock starts on tracking time in the Test state, therefore, overall work items spend less time physically “waiting” in this state before being “actively” worked on. But another possible real world effect might be less overhead (lost capacity) managing a large and mostly idle backlog. Another, might be less context switching (lost capacity) between now fewer “partially active” work items, which could likely contribute to less in-process mistakes (improving quality) and less rework (lost capacity). While the rate of work items completing Test might increase due to these real world benefits, for other real world reasons it may simply remain the same (or could even possibly decline). However, a large Test queue here isn’t without costs either, and an appropriate WIP limit did help to create predictability in the Test state lead times, which in many real world contexts is a step toward balancing the overall system costs.  

Note: The point above regarding the exit rate is a subject worth more discussion, but one I won’t dive into here. Still, understanding and being aware of this, is important in any real world context. It is also why we continue to see shelves between Test and Deploy in Figure 2. Once a work item reaches the Test state, the simulation tool simply processes the work items as it did before when there wasn’t any WIP limit controlling the entry. That is, its simple processing rules don’t model effects like increased capacity benefits from reduced context switching, or reduced overhead and potentially less rework from managing a much smaller backlog. Still, for the purposes of seeing examples of how a bottleneck might appear in a CFD, it is plenty useful.

The other “obvious” effect of this single added WIP limit is the Code state now looks like it could be a “bottleneck” in our workflow. But is it really a bottleneck? There are now quite a number of noticeable shelves between the Code and Test states in the CFD, much more than we saw in Figure 1. Why? By adding the WIP limit to the Test state, work items in the Code state now have to “wait” more often before proceeding to Test. Notice also there are no shelves where the Test WIP goes to “zero” waiting for work items from the Code state. Do both of these observations suggest the Code state is not the bottleneck of this workflow?

Applying WIP Limits Upstream of the Bottleneck

Figure 3 – Applying WIP Limits End to End

In this next CFD example, WIP limits are applied upstream of the Test state to the Code and Design states as well (see Figure 3, click image to enlarge). This models a “kanban pull system” from Design thru Test states in the workflow. What immediately stands out? First, just as we saw adding the single WIP limit earlier produced a predictable lead time for the Test state, we now see these additional WIP limits produce a very predictable lead time expectation for work items passing through the entire modeled workflow. Is this beneficial?

Recall, in the earlier examples, the overall WIP of the workflow (from entering Design to exiting Deploy) at any specific reporting interval increased in the subsequent reporting interval. We can infer the lead times for work items passing through the system was “increasing” too as we progressed through the reporting intervals. Adding the WIP limits upstream of the Test state, to the Design and Code states, produced a system where the overall WIP at any specific reporting interval, in particular after Reporting Interval 11, stabilized more or less, creating relative lead time predictability as we progressed through the reporting intervals. But where did the “visible backlog glut” go? 

The work items in this simulated example simply enter the workflow at a slower rate, over a longer period of time taking more reporting intervals to complete all the work items. But, again, this is only a model derived from a simple simulation tool. In a real world software development context, there are likely benefits from reducing and managing overly large queues that can help to increase the rate of completing (quality) work items as I mentioned earlier. While the predictability created is possibly a significant business value on its own (ex. helping to define meaningful lead time SLAs or system throughput measures), it’s also creates a foundation for meaningful process improvement efforts, a baseline, from which proposed process changes can be objectively measured and evaluated.

The other observation that stands out here is the “shelves” now appear frequently and on all the lines of the CFD chart. Why? Again, adding WIP limits causes work items in upstream states to “wait” more often before proceeding. Notice too, upstream of the Test state, there are no shelves where there the WIP goes to “zero”, these (starving) shelves still only appear for the Deploy state. Look at Reporting Intervals 57, 59, and 61, where the CFD shows clearly the Design and Code states are in a “holding pattern” and the Deploy state “starving” for work (WIP of zero), all waiting for the Test state to finish up work items. Clearly, the Test state has the characteristics of a bottleneck, right?

How Might Addressing the Bottleneck Change the CFD?

Figure 4A – First Test State Improvement Effort

Figure 4B – Second Test State Improvement Effort

Figure 4C – Third Test State Improvement Effort

In this next CFD example, I modeled three successive improvement efforts on the the Test state (see Figure 4A, 4B, and 4C, click each image to enlarge). Looking first at Figure 4A, what immediately stands out?

The first, is the shelves appear less frequent. Why? Obviously due to the ability to process work items more quickly through the Test state, the Code state is now blocked less often from passing work items downstream as soon as coding is completed. In turn, the Design state too is now blocked less often by the Code state being blocked.

Secondly, while there are still reporting intervals where the Deploy state is “starved”, these periods are now mostly shorter than in the earlier examples. Again, this is due to the improved performance of the Test state.

After two more runs that modeled further improvements to the Test state, the resulting CFDs respectively show even fewer shelves. Notice too the Test state band after the third improvement now becomes fairly narrow relative to the Code state. While there are still reporting intervals where the Deploy state is “starved”, they have become even less frequent now.

As a final observation, notice the “Average time to complete” metric goes from 12.4 down to 8.0 after the first modeled improvement effort to the Test state, then down to 7.1 after the second modeled improvement effort, then down to 6.3 after the third modeled improvement effort. Does this add even more support that the Test state was in fact the “bottleneck” for the modeled workflow?

Note: After reviewing these three example CFDs closely, I think Figure 4B (derived after the second modeled process improvement effort to the Test state) showed the “smoothest” overall workflow even though its “Average time to complete” metric is slightly higher than Figure 4C (derived after the third modeled process improvement to the Test state). Again, this is only a simulation, but this observation reminded me that focusing improvement on the bottleneck could “eventually shift” where it is in your workflow. After the third improvement, there are reporting intervals where the Test state WIP goes to “zero” and it appears to happen more frequently than for the Code state. Does this noticeable “turbulence” appearing in the CFD chart lines in Figure 4C relative to the “smoothness” in Figure 4B help indicate the third improvement effort on the Test state moved the system close to this shift?  What might this “turbulence” suggest about the “bottleneck” in this workflow example?

How Might Addressing a Non-Bottleneck Change the CFD?

I couldn’t help but run one last CFD example to “validate” my expectation about what happens if I had modeled an improvement effort on the Code state instead of the Test state. That is, picking up from the example that created Figure 3 above, if I model an improvement to the Code state, and left the Test state untouched, what might be your expectation? The result of this last example modeled is the CFD below (see Figure 5, but DON’T click the image to enlarge just yet).

Figure 5 – Improving a Non-bottleneck state

If the Test state is the bottleneck, would a modeled improvement to the Code state reduce the “Average time to complete” metric? Looking back at our earlier definitions for a bottleneck would suggest no, right? So, before you click to enlarge the Figure 5 image, one last time are you on-board with the Test state being the bottleneck?

Alright then, with your expectation now firmly in place, click on the image to enlarge and see for yourself what happened to the “Average time to complete” metric after I modeled an improvement to the Code state rather than the Test state.  

Well, was your expectation met? Look at the chart lines now more closely. How different in terms of the chart line characteristics does the CFD in Figure 5 appear from the CFD in Figure 3? How different are the CFD chart line characteristics in Figure 5 from the CFD in Figure 4A or 4B?

Now, ask yourself, “where are you making improvements in your workflow today?” Where in your workflow are these improvements having a desired impact? Are you seeing the benefits from these efforts that you were expecting?

Closing Thoughts 

If the modeled examples helped you better understand how bottlenecks might appear in a CFD, it would be helpful (and appreciated) to hear from you and learn how they helped, or also hear what is still puzzling.

The last thought I’ll close with is that a root cause analysis of any bottleneck should still be done carefully. In these examples, the Test state may be where the “bottleneck” appears in the CFD but an effective root cause analysis could lead you to something occurring elsewhere in the workflow. Maybe the Code state is producing very poor code. Could some of that “excess capacity” in the Code state go toward adding unit testing that would improve the quality and possibly see improvement in the Test state and the overall workflow lead time? Maybe the Design state is producing very hard to test solutions. Could using some of that “excess capacity” in the Design state to add some acceptance test driven development (ATDD) to the mix help reduce the testing effort further downstream in the Test state and improve the overall workflow lead time? Anything that directly or indirectly helps to reduce the efforts needed in the Test state in these examples would likely lead to an overall improvement to the workflow lead time, right?  

The CFD tool, like other tools, really only helps you see an issue exists and hints where to begin an effective root cause analysis. It can help you get to better questions faster, but as I like to say often regarding any tool, “Thinking is still required.”

Take care,

Frank

Additional helpful references:

David J. Anderson, see these two posts titled “Two Types of Bottlenecks” and “Detecting Bottlenecks in a Kanban System” (they are both at the same location, blog post, April, 2008, scroll down to see the second).

David J. Anderson, Kanban, (2010); foundational book on implementing concepts of limiting work-in-progress in software development context.

Donald G. Reinertsen, The Principles of Product Development Flow: Second Generation Lean Product Development, (2009); indispensable resource for learning about “the science” of managing and improving workflows.

“Here’s the bottom line: the number one driver for shipping products quicker is by focusing on the important ones and killing the unimportant ones.”

“You might be thinking: ‘True, but couldn’t we also increase the average completion rate’? You’re right, but the impact of doing that is much lower than reducing the TIP (things-in-process) — that is, influencing the average completion rate is rather difficult and is often a function of available resources, scope creep, market demands and changes, etc.”

- Pete Abilla, Nov 2006, Little’s Law for Product Development

A few weeks back Arne Roock (see his posts here), a fellow kanban/lean-agile practitioner, pinged me with a question related to Little’s Law and utilization. Paraphrasing, essentially it was “Queuing theory states that the speed of networks decreases dramatically (non-linearly) as utilization increases more than 80%. But according to Little‘s Law (given a stable system), Lead Time increases linearly if we increase WIP (which increases utilization). Why doesn’t Little’s Law show lead time going up exponentially from a certain point on (ex. past 80% utilization)?” This resulted in exchanging a couple emails discussing the use of Little’s Law, and why and how in the software development context an increase in work-in-progress could result in a non-linear increase in lead time. This post captures and reflects some of the thoughts we shared. My assumption is you’ve wondered too about similar questions. If so, I hope you’ll find this post interesting and helpful.

Little’s Law: In a Nutshell

First, starting out we’ll get on the same page with a quick review of Little’s Law.

Note: I see Little’s Law represented and labeled various ways when applied to different contexts and scenarios. The versions here are based primarily on my personal application of it in the software development context.

Little’s Law is typically defined as follows:

L = λ W

Where:

L = average number of work items in a queuing system over a period of time

λ = average number of work items arriving into the queuing system over some unit time

W = average time a work item spends overall in the queuing system

Applying some basic algebra, Little’s Law is often re-organized and re-labeled for use in the software development context as follows:

LT = WIP / Throughput

Where:

LT = average “lead time” of how long it takes a work item to move through the workflow from start to delivery

WIP = average # of work items you’ll see in progress at any time in the workflow over some period of time of interest

Throughput = average # of work items (exiting) the workflow per some unit time

Observe the changes include substituting Throughput, an “average completion rate” (ACR), for an average arrival rate in the original form. In the software development context the work items are typically things like a new user story, a change request, or a bug fix. 

Note: See my earlier post here for a bit more discussion of Lead time versus Cycle time in a software development context and how it might impact a discussion on duration or effort tracking. 

Little’s Law: A Quick Test Drive

Now let’s take the second form of Little’s Law above for a quick “classroom context” test drive. It’s a simple ratio, so by holding Throughput (the denominator) constant, it’s easy to see when WIP (the numerator) increases, then LT(the quotient) will increase in a proportional manner. Nothing difficult about this classroom context and scenario, right?

But does the assumption that we can hold Throughput constant as WIP increases hold up often?We’ll revisit this question in a bit, but let’s also get on the same page with assumptions that Little’s Law is based on first.

Little’s Law: Assumptions

There are only a few, and here’s my short version: all three parameters in Little’s Law have to be in the “same units”, and each must be a “long running” average in a “stable” system. The first assumption is easy. If you’re specifying Throughput in work items per day then WIP needs to be measured in “equivalent” work items. For example, in a software development context, if you measure an average rate for Throughput using task level work items completed per day don’t use an average WIP measured in story level work items. 

However, the last two assumptions are a bit more complicated because there are nuances specific to different contexts and scenarios. Being aware nuances exist and understanding them is important, but discussing them in detail is beyond the scope of this post. For now, I’ll point you to this reference for more information on these nuances (see Ch. 5 – Little’s Law, by Dr. Stephen C. Graves, Professor of Mechanical Engineering and Engineering Systems at M.I.T.). Still, I’ll include just-enough-detail here to show why and how they are important. 

Little’s Law: Long Running Averages and a Stable System

Time to get our feet wet with Little’s Law and these assumptions using a simple real world context and scenario. My sketch of two water tanks (click on image to enlarge) is inspired by an example of Little’s Law found here. 

In the left tank, water arrives and exits at the same rate (2 gal./hr.), so over time, Throughput, the water exiting remains constant. Also, the water level in the tank, WIP, remains constant over time (5 gal). Since no water gets stuck to the side of the tank, over time the average time, LT, it takes for any one molecule (or a gal.) to pass through the tank is constant. Again, as long as we’re consistent for both Throughput and WIP, we can use units of molecules, or gals., or whatever is best for our context.  

Applying Little’s Law we get 5 gal. / 2 gal. per hr. to yield a LT of 2.5 hrs. Let’s step through this too. If water stopped flowing into the tank, all the water would pass through in 2.5 hours based on the exit rate. Replacing the water at the same rate it exits simply keeps the water level at 5 gal. while water entering passes through the tank on average in 2.5 hours. 

Now we’ll utilize more of the water tank, increasing the flow in until it reaches the new water mark (20 gal.), then back off the flow going in to a rate again equal to the water exiting the tank (2 gal. / hr.). We keep the Throughput, water exiting the tank, constant during this time, so the same number of molecules (or gals.) are exiting the tank as before the water began arriving at a faster rate.

Note: I’m focusing on the case as we increase WIP, since this is really the heart of the original question regarding Little’s Law and a system’s utilization as it increases. I acknowledge there is another side related to what happens when we lower WIP beyond a certain point, that I’m not addressing at all in this post.

While water arrives faster than it is exiting, WIP is not a long running average, but is increasing over this time, right? Is the system in a stable state during this time? If we’re increasing the WIP of water in the tank, by Little’s Law, we’d expect an increasing average LT for our water molecules (or gals.), right? But, is it appropriate to use Little’s Law during this time that water is arriving faster than exiting? 

Throughput was constant as WIP increased during this time. Still, in our simple water tank example the system becomes “stable” again only when the flow of water arriving in the tank returns a rate equal to the water exiting. Once the system is stable, then Little’s Law yields a new long running average LT at this higher WIP. We see 20 gal. / 2 gal. per hr. yields a new long running average LT of 10 hrs. The increase in LT is again proportional with the increase in WIP which is now four times greater.  Note though there’s no mention of utilization levels in the assumptions that Little’s Law is based on. 

Little’s Law: Utilization Effect

So, is Little’s Law a “linear” relationship such that an increase in WIP always produces a proportional increase in LT? Let’s revisit that earlier question, can we assume we’ll hold Throughput constant in our second form of Little’s Law as we increase WIP? 

The water tank example is a context and scenario for using Little’s Law that is pretty simple, as it models a fairly “continuous and uniform workflow.” The value of walking through this simple example for me is that it helps see the importance of understanding the assumptions. The why and how regarding using same units is pretty evident. Hopefully now, so is the why and how related to having long running averages and a stable system before using Little’s Law.

Note: For me, another critical point to catch here is the importance of a WIP limit. Even setting just one at the overall system level to start, even if it appears “impractically high” at first, a WIP limit helps create and maintain a stable system enabling the (effective) use of this helpful tool (Little’s Law), for learning, understanding, managing, and more effectively validating if changes improved your workflow.

When someone says they’re not using “explicit” work limits, and claim they’re “successfully” delivering for the most part on-time, and there exists a clear sizeable backlog of pending work,  after “digging” a little deeper I commonly find one or both of these things.

The first, someone (usually an experienced, skillful project manager) is employing “implicit” work limits, by effectively deferring some work items often through reducing scope or fidelity of features, and making efforts to manage expectations and mitigating visible risks. This is “good project management” in my opinion, but I don’t see this often.  The second is, quality really could be much better and there is a significant amount of “hero effort” needed. 

The disadvantage of an implicit way that I observe is the lack or poor visibility contributes yet even more to a process being subjective, often more political, enables more circumventing, encourages more context switching, often requires more “unplanned” over-time, often produces more bugs and  hacks, etc. Yes, I see explicit work limits ignored too with similar results. However, having worked both ways my experience still leads me to believe determining and visualizing WIP limits that closely reflect current capabilities of the environment, then creating behaviors through polices to manage your workflow with respect to these limits, results in a more effective (less subjective) context for surfacing root causes of delay and poor quality, and making and validating helpful changes.

But will workflows in the computer network or software development contexts and scenarios be as uniform or continuous as water? If not, how might this change things with using Little’s Law? In a software development context, I immediately thought about how context switching, if not managed (with workflow policies) as WIP increased, could make it difficult to keep Throughput constant. 

For example, looking back at the second form of the equation again:

LT = WIP / Throughput

if context switching began occurring heavily as WIP increased causing Throughput to decrease, then we’d see an increasing numerator over a decreasing denominator. In this case, when (if) the system became stable once again, plugging into Little’s Law the new higher long running average WIP along with the new lower long running average Throughput, then the quotient, LT will increase non-linearly, not proportionally, relative to this new long running average WIP. Again, when Throughput doesn’t remain constant but instead decreases as WIP increases we’ll see a non-proportional increase in LT relative to the increase in WIP. 

Summary

The key points from my email discussion with Arne can be summarized as follows: 1) it is important to know and understand the assumptions that Little’s Law is based on including the nuances for your context and scenario; 2) depending on your context and scenario, Little’s Law can yield a result for LT that is proportional to an increase in WIP as well as one that may be non-linear relative to an increase in WIP.

Speaking more specifically for the moment about the second form presented above, we need to make sure our system is stable with long running averages for WIP and Throughput before any calculation of a long running average for LT can be made. If Throughput remained constant as WIP was increased to a new level, then the new value for LT will be an increase proportional to the increase in WIP. However, when an increase in WIP is accompanied with a decrease in Throughput (ex. context switching in a software development context and scenario), the new value for LT will be a non-proportional increase relative to the new increased WIP level.

Over the next few days though I realized there is more to Arne’s initial question. I felt we didn’t dig into the first part of the question enough. Yes, while these key points summarized above related to Little’s Law were helpful to discuss, they don’t touch on the core of why in a computer network context a non-linear decrease in speed occurs as utilization increased above 80%. In retrospect, maybe this wasn’t really at the core of Arne’s question to me either. Still, over the next few days I wondered more about why and how the non-linear decrease of speed in computer networks occurs at higher utilization. More importantly could this be helpful to me in creating and shaping the polices that manage software development workflows at higher WIP levels? As you might guess by now, I’ve dived into this question too a bit already and learned some interesting  things I think would be helpful to capture and share, but this definitely will have to wait for another post.

Take care,

Frank

“Knowledge is an unending adventure at the edge of Uncertainty.” - Jacob Bronowski, British mathematician, biologist, historian of science, poet, inventor, author of “The Ascent of Man”

“Maturity of mind is the capacity to endure Uncertainty.” – John Huston Finley, 1938 – New York Times editor-in-chief

“Many high performers would rather do the wrong thing well than do the right thing poorly.” – Thomas J. Delong & Sara DeLong, “Managing Yourself: The Paradox of Excellence”, June 2011 – Harvard Business Review

It’s common to see an organization (the people in them) focus on building systems with as many features as possible and targeting delivery by a specific due date. Yet, often the result is missing the date while also ignoring other important goals demanded by the businesses such as high levels of product quality, development productivity, planning reliability, employee satisfaction, and customer loyalty. Retrospectives, if done after such an occurrence, surface the dissatisfaction concerning missed dates, poor quality, technical debt, and more, still frequently this pattern repeats. Does this scenario sound familiar to you? If so, why do you think it is so? In a past or maybe your current organization, I’m guessing you’ve heard or thought, “We need our business and technology people on the same page.” How might “being on the same page” look in your organization? Does your current software development methodology, its principles, processes, and practices, contribute effectively to this objective? Is getting on the “same page” with objectives and goals enough? 

Over the last three years Richard Hensley, AVP Process at McKesson Health Solutions, has worked to address these challenges within the three business units of the division he works in at McKesson. I’ve been able to catch up with him on several occasions and discuss his efforts over these years. This post is a brief “fly-over” of his experience, capturing some of the key thoughts Richard developed over this time, and shared during our conversations.

No Silver Bullets Here!

Anyone reading a blog post on getting business and technology people on the same page probably knows there is no “silver bullet” to this challenge. Chances are good you’ve already been a part of such an effort, right? If so, you know it is not a simple task. Or even if you saw them start out on the same page in your organization, did issues appear that caused this working relationship to return to its earlier challenging state?

In short, Richard suggests, “you won’t find quick fixes for this issue” and his experience indicates it takes a serious effort from both sides. It also requires a continuous commitment over time because “getting everyone on the same page is a good start but not sufficient to sustain the business.” Yes, getting on the same page upfront, by itself, can be a positive thing as it helps everyone feel good, knowing where others are at and being in the know. If you’re familiar with the practice of daily stand-ups, you can draw a comparison, getting on the same page only is like a stand-up where team members simply provide “status” of their work. It is a good start, but alone it is ineffective for helping to solve real issues. In a similar fashion, getting on the same page with objectives and goals, if that is all you do, doesn’t help anyone address how things might be done more quickly or more effectively. It is a good start, but more is needed!

First, A Little History for Context (L&K 2009, LSSC, and LSS)

In May 2009 the Lean & Kanban Conference was held in Miami, Florida. It was the first ever conference focused completely on applying lean-agile and kanban principles and concepts to the software development field, put on then by the recently formed Lean Software and Systems Consortium (LSSC). The next year this conference was renamed to LSSC 2010 Atlanta, then followed by LSSC 2011 Long Beach, and most recently LSSC 2012 Boston. Though Richard and I both live in the Denver-metro area, the first time we ever met and began discussing his efforts at McKesson was actually at the L&K 2009 conference in Miami.  

Richard Hensley (Brickell Key Award Winner – LSSC 2011) has worked twenty-five years in the healthcare technology industry, building systems supporting pharmacies, prescription insurance claims, clinical laboratories, and many more. At McKesson Health Solutions, he leads the lean business and product development transformation, working with business and product development leaders to “pull the quality lever” in a meaningful way. Over the last three years they have worked to transform the development organization responsible for products supporting four major lines of business that contribute significantly to the financial success of McKesson.  

Employing the Kanban approach for change management, McKesson implemented new tools selected from RUP, XP, Scrum, and lean—daily focused planning, stand-up meetings, retrospectives, TDD, information radiators, user stories, etc. They automated anything they could and measured everything possible. These efforts have delivered 103 production releases with no significant defects, fulfilled sixteen multi-million dollar contracts, maintained high employee morale, and trained 5,000 users. Most importantly, though, they developed a culture that puts quality and continuous improvement at the forefront.

How Might Business and Technology “Peacefully” Co-creating Look?

To describe what business and technology “peacefully” co-creating in his organization might look like, Richard suggests you think in terms of “Getting in the same boat!” That is, it doesn’t look like a large boat dragging a dingy. It is clearly the organization’s business people and technology people getting in “one” boat”, the “same boat”, and then everyone paddling in the “same direction.”

How does this look on the organization floor? At the highest perspective, Richard says business and technology peacefully co-creating looks like this:

1. Co-making decisions
2. Co-facing the outcomes and consequences
3. Developing and using a common language to communicate with each other

 
It is about co-making decisions, with both business people and technology people engaged in identifying and creating a shared understanding of these decisions. This includes identifying key principles and having a common belief in them as a foundation for guiding these shared decisions.

Then, it is co-facing the outcomes and consequences of these decisions, especially during necessary and important learning phases when some of these decisions won’t be perfect and the water gets a little choppy. Without shared creation and shared accountability it’s too easy to slip back to “separate boats” thinking and actions. Again, it’s about being in one boat, paddling in the same direction in the good and easy times and in the tough (learning) and rough times. Richard’s experience indicates you really have to start here with these first two keys to visualize how business and technology “peacefully” co-creating would look in your organization.

A big part of making these first two keys work is the third key that he suggests which is developing and using a shared nomenclature across your organization’s levels. The nomenclature you develop and use needs to be meaningful in your context and it needs to become the primary tool that enables good communication between your organization’s levels. However, it’s important to also allow each level in the organization to develop and have their meaningful part of the nomenclature. That said, a level’s nomenclature must still be understandable and have meaning across the other levels in the organization. We’ll talk more about organization levels in the next section of this post as well.

As LSSC 2011 Long Beach came to a close, several groups gathered for open space sessions. Several of these sessions soon converged on discussing the challenge of helping higher business levels in the organization learn about the lean-agile principles many of us there were successfully employing at the technical levels within our organizations. Richard began sharing several of the thoughts captured here now in this post. His notion of developing upfront a common nomenclature caught my attention for a couple of reasons. The first is I think this explicitly states a key point (perhaps missing or not explicit enough) related to the ideas I’ve read about elsewhere for creating a Lean-agile PMO in an organization (see “The Lean-Agile PMO” by Sanjiv Augustine).

But what really hit me about this suggestion was the parallel to the idea of developing a “ubiquitous domain language” related to developing software architecture models used to aid builders (developers) of software with effectively communicating with users and business stakeholders (see “Domain Driven Design” by Eric Evans). In essence, Richard’s thoughts on developing and using a common nomenclature from an “organization architect’s” perspective addresses a similar challenge of facilitating effective communication across the business and technology levels of an organization. 

How Does Business and Technology “Peacefully” Co-creating Occur?

So you’re in the same boat now, and paddling in the same direction. You’ve identified some key principles to guide your processes and practices, you’re co-making decisions, and you’re co-facing the outcomes together, both the good ones and the bad ones. And you’ve developed and are using a common nomenclature to talk about the things important to both the business and technology sides of the house and through the levels in the organization. What’s next?  

Think back to “tough times” you’ve had when working with others. What do you remember most about these? Was it “frustration” with not being able to help someone “see” your perspective? A large part of keeping things moving in the same direction is handling these tough times constructively. How? A key to addressing this Richard suggests is to “focus on helping others learn by doing (not by teaching).” Why?

It is tempting to think you can “teach” the other side what they don’t know when differences arise. However, teaching doesn’t guarantee learning occurs (that is, learning by those being taught, even though teaching others about something does often help the “teacher” learn it more deeply). Learning models indicate learning occurs more deeply and effectively by doing (hands-on) in the context of what you are attempting to learn.^{1, 2}

Richard’s experience leads him to believe the “the necessary education comes after the learning, learning by doing, not before doing, and not by trying to teach it up front.” In fact, he thinks “teaching up front would have likely constrained the solutions (their early efforts) prematurely due to yet incomplete knowledge of problems (their system’s yet undiscovered and not well understood issues).”

Again, what stands out for me is here regarding Richard’s thoughts, is how they parallel and adapt from what I’ve read about elsewhere. In this case the impact of focusing on learning upfront before specifying or designing an initial solution is a concept reflected in Michael Kennedy’s work studying the keys of the Toyota Production System³ and also reflected in ideas found in David Snowden’s work on complex adaptive systems⁴.

As you go through this process of learning together and, as mentioned earlier, develop your common nomenclature, Richard suggests you continue to ”recognize the following different organizational levels and roles.” At McKesson, for his division, he’s identified the following:

1. Strategic Level – Business Leaders
2. Tactical Level – Product Owners, Business Analysts
3. Execution Level – Developers/Engineers, SMEs (subject matter experts) QA & Testers, Software Architects

 
Business Leaders are responsible for defining “Why” via “initiatives” needed to achieve the strategic business objectives and goals. In Richard’s context at McKesson these initiatives, this term itself and the language used to describe them, are part of the common nomenclature developed to communicate them to the other levels of the organization.

Product Owners and Business Analysts are responsible for defining “What” via product “features” needed to achieve the business initiatives. In essence, this represents the tactical aspects of achieving the desired strategy developed by the business leaders. These features, this term itself and the language used to describe them, also are part of the common nomenclature developed in Richard’s McKesson division.

Developers/Engineers, QA & Testers, and Software Architects are responsible for defining “how” via technical processes and practices needed to build the specified product features. At this third level, the notion of “stories”, the term itself and the language used to describe them, become part of the common nomenclature, as do many other terms related to processes and practices at this level. For example, terms like “sprint” or “expected lead time”, “flow rate or throughput”, “backlog”, “t-shirt size”, etc. need to be well understood and shared across the necessary levels in the organization too.

One last and very key point Richard makes is related to the terms responsible, accountable, and influence. We’ve already discussed “responsible” above for each level. But Richard feels it is critical that “no level should define for another level anything that it has no direct ability to truly influence.” For example, how long it takes to complete a requested feature, after the tactical level has specified it, is solely the domain of the execution level of the organization. The tactical level should not attempt to specify both what (the feature), and also how long it will take to develop (executing the building of the feature).

Next Steps

In closing, the intent of this post was to be a “fly-over” of Richard’s experience, capturing some of the key thoughts he’s shared with me during our conversations about his efforts over the last three years at McKesson Health Solutions and in his role as AVP Process. In particular, at a high level we’ve talked about how business and technology “peacefully” co-creating might look, identifying three key characteristics, co-making decisions, co-facing the outcomes and consequences, and developing a common nomenclature for communicating between the levels in the organization. We then suggested that business and technology people focus on helping each other learn the other’s differing perspectives in the context of doing and working together on real issues, not by trying to “teach” the other upfront with discussion or debate and why this was important based on learning models. Lastly, we identified three common levels in an organization and highlighted the differing perspective and focus of each and the importance of respecting these differences of responsibility and influence.

Going forward, ideally we’d like to hear from you any specific questions or comments on the concepts and ideas above. Our plan would be to possibly address and include those in a follow-up post maybe even two based on interest and the feedback received. There is certainly more to share about Richard’s experience including a deeper dive into the nomenclature he’s seen developed at McKesson, as well as looking more at how they have visualized their organizational level workflows and the key interactions between these levels. Depending too on interest, another area to explore further might be the key metrics they are using to help identify issues and monitor their improvements. We hope you share your thoughts too and look forward to hearing from you!

Take care,

Frank

References:

¹ For a reasonably short but still usefully in-depth paper as a starting point on understanding how teaching, learning, and learning to learn, might differ, check out “Learning to Learn” by Karl R. Wirth and Dexter Perkins.

² ”Strategies for Learning from Failure” by Amy C. Edmondson is a reasonably short but still useful paper on how an organization’s attitude toward failure can inhibit or promote learning, not just learning, but rapidly learning, and provides insights for strategies in different work settings where learning can occur effectively.

³ See the book “Ready, Set, Dominate” by Michael Kennedy; also his earlier work “Product Development for the Lean Enterprise” for insights into concepts like set-based design (if you think SBD is only about simultaneously considering multiple potential solutions to a problem and converging to one, you’re missing a significant part of Kennedy’s message).

⁴ Dave Snowden, see Cynefin Framework (YouTube video) and Cognitive Edge web site.

“The possession of tools, techniques, and technology is not the competitive advantage…it’s the learning you wrap around them before anyone else.”
– Steven J. Spear, Senior Lecturer – MIT Sloan School of Management & Engineering Systems Division – May, LSSC 2012, Boston

“Specifications should come more toward the end of the project not the start…capture knowledge for future use – both successes and failures.”
– Michael Kennedy, author of “Ready, Set, Dominate”, retired Senior Member, Technical Staff after 31 years at Texas Instruments Defense Electronics – May, LSSC 2012, Boston

“I’ll probably annoy a few system thinkers as well as proponents for self-organizing teams, but that means I’m doing my job…extremes of centralization and de-centralization are sub-optimal.”
– Don Reinertsen, educator at California Institute of Technology , author of “Product Development Flow” – May, LSSC 2012, Boston

“Today, I’d like to be provocative, not obnoxious because I think that is the place to be…pushing people to work faster is not as effective as working on the interfaces (the handoffs).”
- Gregory A. Howell, educator, author, and co-founder & managing director of the Lean Construction Institute (LCI) – May, LSSC 2012, Boston

 “Experience is not the same as knowledge (remember from which you are answering a question).”
– Yochai Benkler, Professor of Entrepreneurial Legal Studies at Harvard, and faculty co-director of the Berkman Center for Internet and Society – May, LSSC 2012, Boston

If you missed the Lean Software and Systems Consortium 2012 (LSSC 2012) conference in Boston and you’re looking for an overview of the keys from the keynotes, highlights from the presentations, or a taste of the pre and post conference happenings, I can point you to several posts that do that very well here, here, here, and here. (Note too, going forward the LSSC is now the LSS, the Lean Systems Society, which you can find more about here).

Here’s My Take on the Lean Software Systems Consortium 2012 Conference

An alternative perspective of LSSC 2012 that I’ll offer you orbits more narrowly around my single biggest takeaway. It wasn’t a single keynote talk or individual track presentation, it wasn’t some succinct brilliant message appearing in the titles for any of the keynote talks or track presentations, nor was it overtly offered as “the main point” for any of the keynote talks or the track presentations that I saw. It was, for me, an unmistakable message noticeably and repeatedly threaded, as the underlying (core) lesson, into the themes of the keynote talks and several of the presentations I attended. “Focus on Learning to Learn” emerged as my “un-official theme” for the conference and was the key takeaway from LSSC 2012.

On the surface that sounds a bit obvious doesn’t it? Learning is the focus or purpose of most any conference, right? Okay, I do wonder about some that occur year after year in the same resort places, but that’s for another time and place. Yes, of course, “learning” is the focus of most any conference. To be clear though, I’m specifically saying, “learning to learn” here, not just “learning.” What’s the difference? ¹

My Early Days – Some Context and Framing

I attended few conferences for professional development early in my career because I found them more often great for breadth but not much in the way of depth. Instead I mainly attended class-type seminars or courses and focused my learning on IT/IS technologies, technical software development practices, software architectures, and the current business domain I was in at that time. That’s plenty, but there were still other problems to solve! I also observed these problems could often marginalize the best efforts put forth from hard-earned learning and lessons in implementing technologies or technical practices.

In 2004 these types of problems led me to learning and applying lean-agile principles and XP practices in my software development environments. They also helped me begin seeing the need to learn even more about the processes governing the environments I worked in as well. This led me to learning and applying more explicit principles of flow, queueing theory, and pull/kanban methods starting in late 2007. It was then I became even further interested in learning more about processes in the software development context and in the concepts broadly related to learning.

Another Try: The First Lean & Kanban Conference (L&K 2009)

These new interests in processes and learning, in particular related to the software development context, led me some three years ago to attend the first ever Lean & Kanban conference in Miami (Lean & Kanban 2009, before the name change to LSSC). I followed that by attending LSSC 2010 in Atlanta, LSSC 2011 in Long Beach, and most recently LSSC 2012 in Boston. In this time I’ve seen this conference grow in number, but more importantly it has grown in quality and learning value. While I think Miami in 2009 will remain my most unique LSSC conference experience, Boston in 2012 is now the high watermark for me personally. In large part I think it is because of the type of thinkers I’m able to meet and talk with in the LSSC community and the message I hear from them, Learning is great, but “learning to learn” is a much more powerful ability to pursue and possess.

Think about this for a second. The attendees of the LSSC 2012 conference are principally from the software development context, yet the three keynote speakers and two of the “featured” 90-minute presenters (see quotes above) are all distinguished individuals outside of the software development field. Still, all of their messages were quite valuable, applicable, and useful to me (and I believe most who attended as well). Why?

For me, it’s this focus on learning to learn! It’s this focus that has made the LSSC conference and this community more valuable to me over this time, and in particular LSSC 2012!

Where can you literally turn around in your seat while your waiting for a keynote to start and find Don Reinertsen ready and willing to chat with you one-to-one for 15 minutes about your questions related to what you think are misconceptions in the use of CFDs in the software development context?

Where can you find the likes of Greg Howell (co-founder of the Lean Construction Institute) standing a “safe” distance from the pre-dinner reception chatter for the Brickell Key award willing to answer your questions one-to-one for 30 minutes about his project management experiences in the construction field and how what he learned could help you in a software development context?

Where could you attend a conference, then catch “the T” to M.I.T. 10 minutes away, and discuss one-to-one for 45 minutes with Dr. Stephen C. Graves (author of Ch. 5 Little’s Law, and Professor of Mechanical Engineering and Engineering Systems at M.I.T.) some about his work in operations management, and his work with John D. Little?

The answer for me to all of these questions was LSSC 2012 Boston!

Connections -Big K Kanban and little k kanban (meta-process)

This focus on learning to learn I believe too is why over the last few year many in the LSSC community now more often refer to the Kanban Method as a meta-process, what is often called “big K” to distinguish it from the “small k” when talking more specifically about a “pull” system. That is, a meta-process layered on top of your current software development workflow process, not something to “replace” your current process. This meta-process is really more about identifying properties that focus on learning to learn about your underlying process, not prescriptive steps for a process. It is this focus on learning to learn that contributes to what is often described as the “catalyst for change” to your process. Yes, as you learn to learn more about your underlying process, it should result in changes, real and effective changes based on validated learning, to your underlying process.

Looking Forward

I see this focus on learning to learn too at the core of several other practices found in our software development context. For example, in the Test-driven Development (TDD) cycle, in the different forms of set-based planning, set-based design, and set-based engineering, and in pull-scheduling systems. So, as I continue to reflect on my LSSC 2012 experience, I’m pretty sure there is a post or two coming from me in the near future that will be inspired by what I learned at this conference. I can tell from notes I captured during the keynote talks and presentations for any one of those listed above, along with several follow-up conversations I had with some of them since the conference (and not to mention from reading, and re-reading now some of their books), the greater challenge will be narrowing it down to a post or two.

My last two months were pretty full with helping organize and put on our local Mile High Agile 2012 Conference, developing and delivering several training tutorials on the road, and attending LSSC 2012 in Boston as well as co-presenting a tutorial there. So, now, I’m working to get a little slack back into my personal kanban board, including time to revisit my “blog post seed material” folder. I hope you’ll stay tuned!

Take care,

Frank

References:

¹ For a reasonably short but still usefully in-depth paper as a starting point on understanding how teaching, learning, and learning to learn, might differ, check out “Learning to Learn” by Karl R. Wirth and Dexter Perkins.

“The ability to take data – to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it, it’s going to be a hugely important skill in the next decades, not only at the professional level but even at the educational level for elementary school kids, for high school kids, for college kids. Now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it.”

“I think statisticians are part of it, but it’s just a part. You also want to be able to visualize the data, communicate the data, and utilize it effectively. But I do think those skills – of being able to access, understand, and communicate the insights you get from data analysis – are going to be extremely important. Managers need to be able to access and understand the data themselves.”

 - Hal Varian, Google’s Chief Economist, Jan 2009 – The McKinsey Quarterly

My previous post discussed how some of my earlier teams used T-shirt sizes for story level work items in their software development planning processes. But T-shirt sizes were only a part of what helped us get effectively predictable. The emphasis on just-in-time (JIT) story creation and story analysis, along with just-enough story and portfolio level backlogs (limiting work-in-progress or WIP), were also significant contributing factors. Two other key factors were de-emphasizing upfront estimating of level-of-effort and duration, and instead placing a greater emphasis on lightweight tracking of real (lead and cycle) times to complete and deliver story level work items. (See my earlier posts here and here for a bit more context and background on push vs. pull scheduling systems.)

I also discussed how some basic analysis of this lead time data and T-Shirt sizing helped us develop an internal service level of agreement (SLA) for completing story level work items. But this information also guided and shaped the policies we developed to influence the team’s interactions and specific responses (pulled from our toolbox) in a JIT manner as information unfolded about a story’s level of effort and duration. My observation is that all this contributed to us becoming predictable in a context where we never had been before using heavier upfront planning strategies. Based on my study of scheduling systems this combination reflected key pull scheduling characteristics, where the role of our software development workflow management changed from determining all operation activities upfront, to one focused much more on setting the rules for interactions (in turn influencing our work environment structures).

There’s lots more analysis (mathematical and statistical) you can do using the minimal and easily collected data that produced the Basic Story (Lead Time) Metrics table and T-shirt sizes from my earlier post and that will have to wait for another time. For this post I want to focus on visualizing the information in this simple table to see if we might extract a bit more value from the modest analysis investment already expended.

I’ll also build on this initial visualization, using a bit more (quick) low hanging fruit type analysis of the full raw data set represented by the earlier spreadsheet snippet, to produce a basic temporal (time) perspective of story lead times. Can adding a basic temporal perspective  provide a number of other useful insights into understanding the nature of our workflow’s story level work item lead times? (Both the earlier table and spreadsheet snippet are included here; click images to enlarge). To this basic temporal perspective, with a bit more new analysis on the existing raw data spreadsheet, I’ll then add new information extracted to give a work item type perspective as well, and then overlay the T-shirt size information to this mix.

Finally, again with just a bit more analysis on the existing raw data, I’ll visualize two other separate temporal perspectives using both  percentages and frequency counts. Afterwards, let me know what you think. Do one or two of these other ways of visualizing the information in the earlier table and spreadsheet snippets help you and others access, understand, and communicate insights that leads to a more effective predictable workflow in your software development context?

Visualizing T-Shirt Size Frequency and Percentiles (Histogram and Pareto Charts)

The charts in this section don’t reflect the “textbook” definitions of a Histogram and Pareto Chart respectively. Still, I’ll refer to them as such for now and leave it to your curious side to learn why they’re “pseudo” versions. Enhanced descriptions of the bins appear as the x-axis labels on the histogram, and the counts are pulled directly from the data in the Basic Story (Lead Time) Metrics table above and appear as labels on the individual bars with a frequency scale plotted along the y-axis.

I used color too, in the same way as done in the earlier table, with green shades indicating the more desirable T-shirt sizes for the story work level items in our context. What do you think? Does this visualization of the same data in the earlier table provide improved access, understanding, or better communicate insights into the expected lead times of story level work items? Note: these story level work item lead times are calendar days that include weekends, holidays, any sick days, or vacations, etc.

Next, the individual percentages of each T-Shirt size bin from the table were added to this histogram, which compliments the visual representation of the color bars and gives additional meaning to the absolute frequency counts for each T-shirt size bin and helps in comparisons. Finally, the cumulative percentage values were used to add the “pseudo Pareto line” which clearly shows story work level items with a lead time of greater than two weeks really are less common relative to the others. All the information in this pseudo Pareto chart exists in the earlier table, there is no new information here. So, does this visualization give better insights, access, or understanding about potential SLAs, or outliers?

Temporal Perspectives of Story Lead Times (Scatter Plot Charts)

For the limited effort required, the histogram and Pareto chart give us a number of useful insights into the distribution of story level work item lead times and have helped us define some meaningful T-shirt sizes (derived from real times and produced from our existing software development workflow process). But, we have no insights into the stability of how our story level work item lead times, or how specific actions along the way impacted them (or didn’t impact them).

To add a temporal (time) perspective we’ll use a simple scatter plot of the underlying raw data to visualize story level work item lead times over our reporting intervals. As mentioned earlier, I did a bit more analysis of the raw data spreadsheet to partition lead time data by functional story level work items and infrastructure story level work items. This was quick to do as several existing fields in the underlying raw data spreadsheet distinguished between these two work item types. (Note: I scroll time along the x-axis with most recent information on the left as older information “falls” off at the right end, which matches my preference for creating CFDs as well. Again, this is a “preference”, and you should flip the direction as needed to fit your context and audience).

What insights about our story level work items does the new temporal perspective in this scatter plot provide us? How does distinguishing between functional and non-functional story level work items help? Do you think the value might be greater or less depending on the context of your software development workflow? Overall we see a number of infrastructure story level work items distributed throughout the full timeline. We also see a higher density of story level work items at the latest end of this time line. What might this mean? It’s also pretty clear we hit a spot in the overall timeline where our story level work item lead times were quite long in our context. What happened here? Can we look back at our story notes (or reporting interval notes, as we kept them) and explain these cases? Clearly this is where our larger T-Shirt sizes came from, right? Why?

Now, I’ll add the T-shirt sizes from the earlier table to this new temporal perspective as solid and dashed colored horizontal lines, in effect adding Pareto type information to the scatter plot. Does this help make it more clear just how few story level work items were taking greater than two weeks lead time to complete? Again, this is lead time, not cycle time, so it includes time starting from when the story was placed into the ToDo process state (and in our context included weekends, holidays, etc.). How about helping to identify outliers? In retrospect, we know from the earlier table there are just under 600 lead times (blue and red dots) plotted on this scatter plot and we can hand count the eleven that have a lead time well above 28 days. This is no more than 3% and we can see for the most part they were concentrated in one area along our 16 months of data. Is that helpful insight in setting SLAs or communicating expectations with others?

Two More Charts for Fun (Stacked Bar Charts)

The temporal perspective of the scatter plots above do provide some useful insights. Again, the overall effort required is fairly limited too. However, along the way I wondered if visualizing both the concrete “counts” of story level work items being completed and the “percentages” of their lead times over time might potentially add some valuable insights. So, I did a bit more analysis on the raw data spreadsheet to partition the months of story level lead time data into four-week intervals (no special reason I picked four weeks, could have been two or six, depending on context, or one might start with two and expand the intervals as more data is collected). 

This first stacked bar chart shows percentages for each T-shirt size bin for a number of reporting intervals displayed on the x-axis. (Again, I’m scrolling latest date on left and earlier date “falling” off to the right.) Notice the reporting intervals labeled Wk 25-28, Wk 29-32, and Wk 33-36. What insights do these three stacked bars provide you? There are some larger percentages of story level work items with longer lead times during these twelve weeks. How does this time frame match up with our scatter plots above? Do you see the alignment with the distribution of dots on the scatter plot being higher on the chart? What new insights or value might the temporal perspective of this stacked bar chart of T-shirt sizes provide you? You certainly see less green for these three reporting intervals. 

On the second stacked bar chart I show the counts for each T-shirt bin for a number of the same reporting intervals again displayed along the x-axis. So, what stands out here? Obviously the number of story level work items completed per four week interval is fairly consistent until the last two reporting intervals. What happened in Wks 69-72 and Wks 73-76? I actually have notes on what was happening in our environment but this is a whole other story for another time. The point here is the stacked bar chart certainly provides some new insights and reflects the context of what was happening in our environment at this time.

Look at reporting intervals Wk 25-28, Wk 29-32, and Wk 33-36 again on this chart. While we don’t have specific percentages, it’ still pretty clear there is less green and more orange and red here indicating longer story lead times, and story counts took a slight dip before rebounding up a bit. Any guesses on what happened? Again, match it up with the scatter plot and notice the time of year. These new insights reflect some of what was happening in our context according to our notes during this time beyond just the usual impact of holidays. 

Closing Thoughts

While reading this post, it might appear at times the analysis and visualization discussed above was done only in retrospect after all the data was collected. I admit while writing this post looking back myself it was hard not to give that impression or perspective. Yet, to be clear, the analysis and visualization of the results in these charts above occurred and evolved over time, as more and more of the full 16 months of data was collected on this project, and as we learned and observed along the way. Also, I want to be clear that as we collected actual story level work item lead times, among other data too like story cycle times, stories completed per reporting interval and per feature, feature level work item lead and cycle times, and analyzed this data in similar ways as above, the models, the information, and the insights were very valuable in helping us get effectively predictable and confident in what we could commit to delivering in a specific time.

A second point I want to be clear on was we didn’t use this data or analysis as part of individual or team performance evaluations. The focus and purpose of this data and analysis was primarily on helping to understand and learn about the nature of our software development workflow related to completing and delivering quality story and feature level work items. As mentioned earlier the information was used primarily to help us guide and shape the policies that influenced the team’s interactions and specific responses in a just-in-time manner as information unfolded about a work item’s (story, feature, epic) level of effort and duration. The role of our software development workflow management became focused much more on setting the rules for these interactions and in turn influencing our broader work environment structures and interactions, and on shaping (conditioning) the workflow inputs and managing them through the well-known constraints and measured capabilities of our software development workflow process.

Take care,

Frank

“We do estimation very badly in the software field. Most of our estimates are more like wishes than realistic targets.”
– Robert L. Glass, Nov 2002 – Facts and Fallacies of Software Engineering

“The best way to get a project done faster is to start sooner.”
– Jim Highsmith

In earlier posts (here, here, and here), I commented briefly about moving away from upfront estimating and toward tracking and measuring “actual” metrics to complete work items, and using this data as a foundation for predictable scheduling and forecasting. Am I suggesting your software development team can’t achieve effective “predictable” scheduling using upfront estimating? No. But first, to be clear, I’m delineating upfront estimating activities from upfront analysis activities. The former is associated with predicting the number of calendar days (duration) to complete a work item and the actual hours (effort) expended on a work item. The latter is associated more with partitioning larger work items (epic, feature) into smaller ones (story, task) and determining relationships and dependencies between them to assess complexity or risk.¹ The significance for decoupling these becomes apparent shortly. So, in your context, if teams haven’t achieved effective predictable scheduling using upfront estimating, I am suggesting it might be beneficial to approach project planning from what is likely a different perspective. I’m also suggesting that over time this different perspective may reduce planning effort.

As an example, I’ll focus on one of the “actual” measurements my past teams collected, in this case Lead times for “story” work items, and discuss how we used this data to derive a very predictable range of “T-shirt” sizes. This range helped us predictably manage story level work items thru our workflow, and enabled us to identify issues earlier as they emerged for those needing alternative (special) processing or management. Before moving to how and what we did and the benefits obtained, I’ll briefly provide context for why we shifted perspective, which may reveal some insights for benefit in your environment.

Earlier Times in a Nutshell

Our development teams were well into implementing agile processes, and demonstrated visible improvements in several areas related to technical practices and our ability to deliver more frequently. However, we continually missed “commitments” for the number of stories to be completed in a sprint whether it was two weeks or 30 days. We were unsuccessful at effectively predicting duration or effort for completing stories using upfront estimating techniques that we had tried including variations of planning poker with ideal days, points, hours, Fibonacci sequences, etc. I won’t go into consequences of these estimating efforts here, but will highlight a key observation made during efforts to mitigate these costs that led to shifting our planning perspective.

As the mid-sprint approached and we saw our commitment at risk, at one point a team decided to assertively reduce (“de-commit”) work items (stories in this case) in their “Doing” state, moving them back to ToDo or to the story backlog. Again and again, when we did this in a sprint, stories remaining in the Doing state would get completed quickly, and then we’d “whip back” the other way and re-pull stories before the sprint end. After of a few of these “herky-jerky” sprints, we did something different and shifted to a “pull-scheduling” perspective.

Push vs Pull Scheduling?

This image is my summary derived from sources contrasting push and pull scheduling systems.² Push scheduling starts with obtaining commitments to upfront estimates of duration and effort, then develops a schedule based on these estimates and seeks predictable outcomes by attempting to adhere to the developed schedule. Pull scheduling starts with obtaining predictability of upfront (measured) capabilities and known constraints, then develops a schedule based on these capabilities and constraints and seeks to achieve commitments to the developed schedule. Is this distinction a bit too intangible for you?

Seeing Pull Scheduling in Action

The changes included some made upstream related to conditioning the feature level work item backlog in the software development workflow along with establishing some work-in-progress limits.³ Downstream of the feature backlog, “policy” changes focused upfront planning on the analysis activities of partitioning a “feature” into story level work items. While the Product Owner still created some key stories upfront, now most were discovered just-in-time (JIT) with the full software development team after the feature was pulled to work on.

The initial analysis activity was time-boxed to either identify the first “batch” of 15 to 20 core stories or end after two hours. As a story was created, we did make a good faith attempt to “size” it roughly as a two to four day effort. But this “sizing effort” was mostly a simple “head-bob” exercise. We simply asked, “Does this look like two to four days?” If too few heads bobbed “yes”, a “spike” story was attached to this questionable large story, both were placed in ToDo, and any more sizing analysis was deferred until the story was pulled to be worked on. When this combination was pulled from ToDo, a deeper analysis spike activity of a day or two was performed to break down the questionable large story into a number of “more likely” two to four day stories. We clearly expected “new” stories to be discovered after work started on any of the feature’s initial “core” 15 to 20 stories. Again, the upfront activities now were focused on analysis to more fully scope, partition, and understand the feature. They were not to estimate upfront duration or effort, though I’ll touch on these two again just a bit later.

As the team worked on stories, basic metrics were collected (see spreadsheet “snippet” of actual project data, parent MMF/epic identifiers were grayed out intentionally, click image to enlarge). From these basic “actual” measurements we were able to (quickly) begin developing a number of useful models including a “T-shirt” model for expected durations to complete a story level work item. Over time, as more data was collected, this model became very stable. These images show the model at 12-weeks, 24-weeks, and then after nearly 16 months of data (see the tables below labeled Basic Story Lead Time Metrics, click images to enlarge).

From these models we learned early that nearly 50% of our stories could be expected to be completed in 1 to 4 days, nearly 20%+/- in 5 to 7 days, nearly 20%+/- in 8 to 14 days, and nearly 10%+/- over 14 days. While over time we improved a bit on these performance metrics on the lower end of the duration lengths, the significant point here is early on we learned key capabilities and constraints of our software development system in our context. We were now able to effectively predict durations for completing stories and with much less effort.

Note: I’m not discussing Cycle time here but a similar distribution data table was also created showing how long it was expected to take for a work item to reach Done after it had arrived in the Doing process state of the software development workflow (development effort). However, see my earlier post here for a fuller discussion of Lead time versus Cycle time and how it might impact a discussion on duration or effort tracking. Also, the question whether to include weekends, holidays, vacations, etc. in Lead or Cycle time calculations is a different discussion in my opinion and one I wont’ get into here either.

Walking Thru the T-shirt Sizing Table

How did these tables help? First, we now began to expect and communicate to others the “typical” story (50%+/-) was a small size work item and should be done, after entering the ToDo process state in the workflow, in four calendar days. This became the team’s internal service level of agreement (SLA) and it included weekends, holidays, vacations, sick days, staff meetings, HR trainings, etc. for members of the team. The basic metrics we collected (see above spreadsheet) included all these “variables” (numerous small variations) as part of “the system” measured and reflected them in these models. Second, as a story’s duration approached 4 days, it garnered additional attention (analysis) from others on the team and in our daily stand-ups. What is slowing it down? Who might be able to help? Can it be partitioned with some portion being completed and still providing value, while the other pieces gets de-scoped, deferred, or completed later (see concept of fidelity)? ⁴ This 20%+/- of the “typical” stories we encountered were considered medium size work items completed typically in 5 to 7 calendar days after entering the ToDo workflow process state.

When a story reached eight days days (orange row on the table), it was considered a “large” work item and our model showed we’d expect to have 20%+/- of the stories reach this size level. This is when we ramped the analysis up yet another notch and applied other tactics beyond the standard process and practices. For example, we might have multiple pairs swarm on the story, reconsider the initial strategy or solution for the story, consider applying a set-based design or engineering approach, re-evaluate its value and priority in the light of more information about the story, defer it, or even decide to simply continue working on it “normally” understanding it more fully and accepting the “cost”, if any, of the delay. Finally, when a story reached two-plus weeks (red row on the table), it was considered an “extra-large” work item and our model showed we’d expect 10%+/- of the stories to reach this size level. These stories typically were those that had dependencies outside of the cross-functional software development team. For example, stories with dependencies on marketing, legal, external engineering or IT departments, 3^rd-party tools or services, or needing approvals for purchases of tools or 3^rd-party services, etc. In many cases these stories required more than additional analysis and often were issues related to systemic or organizational challenges requiring higher level management involvement to address. ⁵

Bringing It to a Close

As mentioned earlier, the same basic data collected above was analyzed in various ways to develop additional models (ex. other data distributions, scatter plots, run (trend) charts, etc.) that helped us visualize and understand the capabilities and constraints of the software development workflow (system) in our context. For this post I only focused on the story level work item and used actual story Lead time data in the context of T-shirt sizing as one example of applying a pull scheduling perspective. I showed how we were able to get effective predictability and reduce overall upfront planning activities using an alternative to upfront estimating that was not providing a commensurate level of value (predictability) returned for the effort expended. However, several other similar and useful distribution models (T-shirt sizing style) contributed to our ability to create more effective schedules as well. For example, models for the expected number of stories completed per specified time interval of interest (flow) such as per day, per week, two-weeks, whatever, or models for the expected number of stories completed per feature.

We in fact used these additional models to establish similar effective predictable expectations for these metrics as well. In the process, we were able to eliminate the need for a “sprint planning meeting” every two weeks. Instead, we only needed JIT planning sessions approximately every 29 to 32  calendar days, which was the measured typical duration for completing one work item from our feature level backlog. Just as we were now able to identify emerging large and extra-large stories and respond earlier to the associated issues, in a similar fashion a distribution model of story counts per feature allowed us to more effectively determine expected number of stories per feature level work item and recognize an emerging “large” feature and respond earlier to associated issues. In essence, this is pull scheduling. We had established predictability upfront based on identifying capabilities and constraints derived from “actual” metrics (measurements) of our software development workflow (system). We developed schedules (expectations) based on these capabilities and constraints. Then we worked to gain commitments to these expectations and to meet them by also using the models to help identify and enable us to respond more quickly as issues emerged rather than continue to try with little success to get better at upfront estimating (predicting the future).

Take care,

Frank

References:

¹ David J. Anderson, “Estimation versus Analysis” (blog post, December 18, 2005).

² Mary and Tom Poppendieck, Lean Software Development, (2003); Implementing Lean Software Development, (2006); Leading Lean Software Development, (2009); see push scheduling, pull scheduling, push systems, or pull systems in the index of these three books (see their web site here).

³ David J. Anderson, Kanban, (2010); foundational book on implementing concepts of limiting work-in-progress in software development context.

⁴ Dean Leffingwell, Scaling Software Agility, (2007); introduces and discusses the concepts of “fidelity” levels (least imaginable, minimum credible, moderate, and best) when delivering software development solutions, and “committing to the signature of a feature’s scope”, not the implementation of the feature. Both are key tools in being able to deliver software predictably in challenging contexts (see Dean’s web site here).

⁵ Dave Snowden, see Cynefin Framework (YouTube video) and Cognitive Edge web site; in our context, the Small, Medium, Large, and Extra-Large T-shirt “sizes” for story level work items were less likely to infer amounts of “tasks” to do (more dirt to shovel) but more likely to be indicative of increasing difficulty in partitioning a story or attaining functional solution desired (simple, complicated, or complex) and as “duration” increased our efforts went from standard practices to much more iterative, learning focused, and emergent ones.

“The paradox is that trying too hard to create predictability creates the opposite effect.”
- Mary Poppendieck, Aug 2003 – Lean Development and the Predictability Paradox

“If you want to go in one direction, the best route may involve going in the other. Paradoxical as it sounds, goals are more likely to be achieved when pursued indirectly.”
- John Kay, Jan 2004 – Obliquity, January (article)

“Rationality is not defined by good processes, irrationality lies in persisting with methods and actions that plainly do not work – including the methods and actions that commonly masquerade as rationality.”
- Johh Kay, Mar 2010 – Think oblique: How our goals are best reached indirectly

If you’ve worked in the software development field for any length of time, I’m sure you’ve been asked, “When will the project be finished?” I’m also sure you’ve found this question challenging to answer more often than not. Why is this? Estimating, forecasting, projecting, call it what you will, is not easy to do well, or even close to effective, especially as the project increases in size, complexity, or both. In a single post, I’m definitely not diving deeply into how one might estimate, forecast, or project how long a software development project might take to complete. However, keeping in mind the “estimating challenges” we’ve encountered in software development provides a real world context for discussing what I think are a couple of related and intriguing concepts.

The Predictability Paradox

In 2003, Mary Poppendieck published a paper titled “Lean Development and the Predictability Paradox.” The lean principles found in this paper appear in a refined and expanded form in Mary and Tom Poppendieck’s book “Lean Software Development: An Agile Toolkit” (2003). However, unless I missed it, the Predictability Paradox, neither as a term or concept, appears explicitly in this book. But, the concept definitely resurfaced in their following book, “Implementing Lean Software Development: From Concept to Cash” (2006), although relabeled as a chapter sub-heading “Myth: Predictions Create Predictability.”

So, what exactly is the Predictability Paradox? The paradox, as explained by Mary and Tom Poppendieck in this second book is that “in our zeal to improve the predictability of software development, we have institutionalized practices that have had the opposite effect. We create a plan, and then act on that plan as if it embodies an accurate prediction of the future.” The key point for me here is, developing a plan itself isn’t the issue, but failing to recognize that we act as if this plan could never be wrong is an issue. You might say, “Now wait a minute, if the plan is ever found to be wrong, shouldn’t we then work harder upfront to get better at making sure the plan is more often right?” I’d say, “Sure!” But then, I’d ask, “How?”

In discussing the Predictability Paradox further, the Poppendiecks list four factors they feel contribute to making the effort of accurately “predicting” the future a difficult one, that is, when what we are trying to predict is one or more of the following:

1. Complex
2. Detailed
3. To occur in the distant future
4. Occurring in an uncertain (changing) environment

 
While I don’t consider this “a complete list”, it’s definitely sufficient to illustrate sources of the challenges we face in any effort to ensure upfront the plan is more often right. But, what if we instead approached a goal of attaining predictable outcomes from another perspective? If we start with the premise that our plans, and in particular our initial and early plans, will not be highly accurate or correct (a high probability in many cases actually), what would we do differently? What if we put more effort, especially early on in the project, into increasing the available response options we’d have if or when we learn the plan is not right?  Could this be a more effective approach to creating predictable outcomes, than applying additional upfront planning or extra efforts needed to adhere to the “original version” of our plan?

The Poppendiecks conclude this section in their book with this thought “an organization that has a well-developed ability to wait for events to occur and then [is able] to respond quickly and correctly will deliver far more predictable outcomes than an organization that attempts to predict the future.” Is this line of thinking something you can or will buy? I’m assuming the first response for some reading this is “This sounds nice in theory, but in my real everyday world I don’t have the luxury of waiting until all hard-to-predict events occur.” In the broader context of the reading I’ve done on lean, and in particular software development, and even in the broader context of the Poppendieck’s book, I understand their message, but that doesn’t mean it is easy to do, and getting to this ability that they speak of doesn’t happen overnight either, does it? Still, thinking about how to increase your options to respond when plans turn out wrong and working toward increasing this ability is a desirable goal and one that you can move toward even if just incrementally (and iteratively) to start. What downsides exist for you that would prohibit considering or taking small steps in this direction?

Economic Success, Economic Choices

In his book “The Principles of Product Development Flow”, Don Reinertsen suggests that pursuing economic success by focusing on efficiency metrics, such as maximizing capacity utilization, or by focusing on conformance to plan, is fundamentally wrong. Instead, he says economic success is more likely to be achieved by making good economic choices, based on the latest most reliable information available, and acknowledging as information changes, the best economic choice may change as well.

Earlier I suggested a key point for me was seeing that planning is not the issue, but it is acting as if the plan could never be wrong. Here, I think the key point again suggests that planning itself is not the issue, but that failing to change the plan when we have later and more reliable information is the issue. It is not just working on our ability to keep options open longer or to more quickly respond when a glitch occurs, it is also shifting to an economic perspective to guide our choices in how we plan, how much we plan, how we respond to glitches when they occur, and continuing to ask along the way if conforming to the earlier plan still makes sense.

Just thinking about economic models is probably enough to dissuade some, if not most, that adding an economic perspective to planning is not a viable option. I’ve actually met Don on several occasions and have followed his work for several years now. Again, I understand the message here too, but that doesn’t mean it’s easy, does it? Still, at the very least, as you plan, is it possible to start asking where in your planning efforts does it “feel” like you’re not getting a good return on your investment of time, and if possible roughly quantify it? For example, begin with looking for where in your planning today the effort to plan or estimate upfront how long something will take to complete is easily observable to be a significant amount of time relative to the time you think it would take to simply complete the work “that you’re planning.” From an economic perspective, is this trade-off in time for value returned at least worth questioning? As an experiment, for a period of time do no upfront estimating of these types of work items at all. Instead, simply track the actual times, plot the data over time, and see if this trend information provides all that you need for “future planning” of these types of work items. Experiment with other work item types too and identify others that forecast well using this lightweight form of planning.

Obliquity

John Kay, a leading British economist, and author of several books, began publishing writings about the concept of Obliquity as early as 2004 (but indicates the term came about earlier). Kay’s emphasis early on was in microeconomics and in particular economics related to businesses. He was particularly interested in answering questions similar to this one, “If two firms face the same basic conditions of supply and demand, and operate within the same market structure, why does one firm do better than another?” In the research he did to answer this question, he studied various businesses and some public agencies, and found the most successful organizations were those that achieved their goals with an indirect approach rather than a direct one. This was even more the case when the systems and the environments were complex and “unpredictable.” You can find examples that he cites in several posts online as well as in his recent book titled “Obliquity: Why our goals are best achieved indirectly” (2010).

Here is another interesting example that demonstrates a twist on obliquity thinking that comes from a study cited in the popular book by Tom DeMarco and Timothy Lister “Peopleware: Productive Products and Teams” (1999). DeMarco and Lister reference a study done in 1985 by two researchers, Michael Lawrence and Ross Jeffrey, at the University of New South Wales. The focus of the study was to see how productive programmers were on projects based on who developed the upfront estimates. There is much more interesting information they reference in this study but I’ll limit my comments related to our discussion on obliquity for now. In short, the study showed that programmers were more productive when they developed the estimates for the work they were asked to do, than if their managers developed the estimates. This is probably not a surprise. However, this study also showed the programmer’s productivity was highest when there were no estimates requested at all. In the desire for achieving greater productivity, a direct approach was taken, based on the belief that asking programmers to estimate their work upfront and then have them work toward their estimates would ensure high productivity (by avoiding gold plating or an assumed to be true Parkinson’s Law type effect). Yet, this study showed an indirect approach, asking for no estimates upfront, actually resulted in higher programmer productivity.

How might taking an oblique (indirect) approach change your planning in the context of “estimating challenges” and a desire to produce predictable outcomes? Do you accept that for some problems the full picture and challenge is revealed only as you actually get into the work to solve them? If so, would predictable solutions be possible earlier, or more economically obtained, using an upfront approach with a focus on learning (experimentation and discovery) rather than one emphasizing upfront planning or estimating (guessing)? How might you communicate when a project will be done with less upfront estimating efforts?

One alternative to the upfront (push) estimating planning approach you might consider is move toward a pull scheduling system. Rather than “predicting” (anticipating demand or delivery), a pull system approach reduces the emphasis on upfront planning or estimating by a shift of emphasis to one on collecting and tracking actual throughput capability of a system over time, and then using this historical (production data, not predictions or guesses) to project (forecast) expected delivery going forward. As more data is collected, trends become more pronounced, and normal delivery capacity becomes better known and documented. This information becomes a solid foundation for predictable outcomes as well as a dependable baseline for assessing the impact of any improvement efforts to the software development process in place.

Closing Thoughts

So, in closing, do you think the ideas of the Predictability Paradox and Obliquity have some relevance in helping you improve your ability to answer the challenging question “When will the project be finished?” Do these concepts give you ideas for making changes to your planning approach or estimating, forecasting, and projecting practices? Do you think taking an economic perspective will result in you making some different choices in how much upfront planning or estimating you’ll do or cause you to be more responsive and open to changing your plans as new information becomes available? If so, I hope you’ll share some thoughts or insights from your experiences.

Take care,

Frank

“Don’t manage the unknown and unpredictable the same way you manage the known and predictable.”
– Doug DeCarlo, Oct 2004 – eXtreme Project Management

“Managers must be flexible enough to adopt the right approaches at the right time…to find the balance between planning and learning.”
– Arnoud De Meyer, Christopher H. Loch, and Michael T. Pich – Managing Project Uncertainty: From Variation to Chaos

Reflecting on the last ten years of my own experience working on projects, early on, I saw mostly “traditional project management.” That is, project management that places greater emphasis on planning upfront, and as a whole, a greater proportion of the total project planning is done very early in the project, and seeks to provide predictability of activities over longer planning horizons. Other terms I’ve heard for this project management style are “predictive” or “sequential (gateway) delivery.” In more recent years, however, my experience has been to see “iterative and incremental” approaches. Again, that is, project management that places less emphasis on planning upfront, and as a whole, a greater proportion of the total project planning is done in small amounts iteratively across the project, and seeks to provide predictability of activities only over shorter planning horizons. Other terms I’ve heard for this project management style are “emergent” and “incremental delivery.”

While this is a minimal level of distinction, for our purposes it will be sufficient. This won’t be a post on “good vs. evil” styles of project management. In fact, I’m advising, that might be a “limiting” perspective. Rather, the “focus” is to view projects from a perspective of uncertainty (as a model) to gain insights into these two project management styles that may help us more effectively manage our projects.

 Are All Projects the Same?

In your experience, how often have you seen projects managed differently from one another? Is there “evidence” to suggest, based on how you manage or see projects managed, that all projects are the same?

Take a look at this list of a few common project management practices. Do any sound familiar to you as some of the “sure ways” to successfully manage projects that you’ve used or seen along the way?

1. Identifying and tracking milestones
2. Identifying potential issues and developing risk management plans to reduce and mitigate them
3. Utilizing iterations
4. Conducting intermittent project plan reviews

 
Why would you choose to use “iterations”, and what makes them “different” than simply a “short” version of a traditional project management approach, what is often referred to as “mini-waterfalls”? Do your planning horizons (or your iteration lengths) ever vary or are they the same length on each project and for all parts of the project? Is there a lack of clarity in the differences of project management styles and reasons for certain practices that results in us more or less using them, or observing others use them, in the same way on any project we work on, and consequently making them less effective at times? We’ll come back to this question, but first, we’ll consider projects themselves a little more.

Well-Known Projects and Not-Well-Known Projects

What would a “well-known project” look like to you? For starters, it would be a project where many of the requirements and tasks appear very familiar to you already, where you have previous and extensive experience with projects that are very similar. It could be a project with a relatively small set of requirements and tasks, making it fairly straightforward to identify it as a part of a number of larger and similar projects you’ve managed successfully already. Do you think a well-known project would have a small number of assumptions? It is reasonable to expect that a well-known project would in fact have few assumptions, and any, would be ones that could be validated easily or predictably.

Would a well-known project be one that has a limited number of highly predictable outcomes? A well-known project would likely have a small finite number of dependencies such that the possible outcomes are easily enumerable and highly predictable in terms of some (quantifiable) probability. What else would you add to this list?

Now, consider what a “not-well-known project” might look like to you. For starters, it would be a project with requirements and tasks that you have no or little previous experience managing. It may also be a project with a large number of assumptions, and even if the number of assumptions were small, a number of them might be difficult to validate. It would be a project with a high number of dependencies such that the outcomes are not easily enumerable or a number of the possible outcomes are not highly predictable in terms of some (quantifiable) probability.

But these are characteristics often associated with “innovative or break-through” projects, those “game-changing” projects that create “separation” between you and your competitors. They are projects that seek to accomplish something “never-done” before or to accomplish something in a competitively improved way (technology or process). Then, what about the environments we’re managing our projects in? Consider a project managed in the context of a “merger” or in the context of a just released new competitor offering. These factors as well may put any project into the not-well-known context. Again, what else would you add to this list?

Another Frame (Lens)

No surprise, in the context of being well-known or being not-well-known, it’s clear we can see projects are not all the same. So, here’s my earlier question restated in other ways. When was the last time you didn’t “iterate” on a project or managed some part of your project using a “predictive” approach over an “emergent” one? Maybe it is time for a “fresh” look! What makes a “series of mini-waterfall deliveries” different from an “incremental and iterative delivery”, or is it really different? Maybe there is a “new” perspective and different distinction to look for here!

We’ll take another look now at that earlier list of practices, using a different frame (lens). I created this table below as a model for myself, heavily derived from the article “Managing Project Uncertainty: From Variation to Chaos” by Arnoud De Meyer, Christopher H. Loch, and Michael T. Pich. The uncertainty types listed in the middle column are extensively discussed in this article, and while no table exists in the article itself, I feel mine below conveys what I took away from the article. We’ll now go through each row in more detail.

Milestones and Variation

Milestones, in my experience, are intended to represent a necessary outcome for the project success, and it’s understood (or assumed) there are decidedly negative consequences if you miss one. Is your experience similar? If so, I would base a “milestone” then only on something well known to be achievable, and highly predictable, even on a longer planning horizon. It doesn’t mean achievable 100% of the time, but it is more than “a wish” for something to be achieved. That is, when established, the milestone is based on information (evidence) supporting my opinion (judgment) that it is achievable and highly predictable, otherwise “as a label” it is of little value.

This brings us to the first type of uncertainty listed in our table, variation. De Meyer et al., describe one form of uncertainty that exists in projects resulting from combining numerous small influences that individually would be costly to plan for or monitor (due to expense or time). However, many times it is possible to measure this “activity” as a range of values (an overall variation) that in turn can be used to cost effectively plan (manage, not control) with a high level of predictability even at longer planning horizons and also monitor this activity throughout a project. Is variation another way to represent a necessary outcome or something similar and as useful?

Here are a couple examples from real world projects. The first is from direct experience as part of a software development team several years ago. We were using an iterative and incremental development (IID) approach. As part of this IID approach, we’d attempt to “estimate” the work that we “expected” to complete in the upcoming short and uniformly fixed-length time period; this was done attempting to (analyze) and estimate each of the small numerous individual work items. These “estimates” were then used to establish and communicate an overall project release plan, which included “milestones” along the way. However, in our context, time and again these particular  “estimates” proved not to be accurate (training, or effort put into estimating wasn’t the issue).

We tried another approach and began simply tracking the “lead times” for work items (observing how long they took to complete once they were selected for being worked on). In (weeks), we established measures of variation for how long it took to complete individual work items, and how many work items we’d complete per day (per iteration); over (months) we also established a measure of variation for how many work items we’d complete for any project.

We found these measures of variation proved to be very accurate for us and required significantly less time to acquire from the overall team. To be clear, we continued the software development work itself using an IID (emergent) way of working. However, “milestones” for release planning (forecasting) purposes were now based on well-known, measures of variation in this case, and gave us high predictability over longer planning horizons.

The second example comes from my collaboration with a colleague who is working with a client that is using a similar approach in their project planning efforts. However, in their context, his client has been able to establish lead time measures of variation at the project level as well (lead time variations of projects, not just for work items needed to complete a project). They moved away from estimating individual work items 18 months earlier. Now with well known, project level variation measures in this case, they are in a position to further reduce efforts needed for planning “milestones” for forecasting purposes. In fact, they are at a level of predictability to consider providing their business stakeholders something akin to an SLA (service level of agreement), well known, achievable and highly predictable, for completing any project in their organization.

Both show that a project manager can choose to use iterative and incremental delivery (emergent) practices, but it doesn’t exclude managing some aspects using non-iterative (predictive) practices. Manage what is “predictable” in your project (well-known or can easily and quickly become well-known) differently from what is not predictable! Measures of variance can effectively aggregate numerous “hard or costly to predict” small influences into something “visible”, well-known, and predictable over longer planning horizons.

Risk Management and Foreseen Uncertainty

Risk management, as practiced in traditional project management, is not often something I see explicitly performed on iterative and incremental projects I’ve worked on recently. I believe the “uncertainty” typically addressed by risk management as practiced in traditional methods is, in large part, implicitly diminished by several software development technical practices commonly found in iterative and incrementally managed projects.

This brings us to the second uncertainty type listed in our table, foreseen uncertainty. De Meyer et al. describe foreseen uncertainty by contrasting it with variation. Unlike variation, which is a measure, a range, of the combined influence of numerous small influences, foreseen uncertainty influences are considered “singly” identifiable (distinct) from each other. Here’s another way to contrast these two uncertainty types: 1) view variation, as we’ve described earlier, as inherent in your system, expected to occur (to exist), and for this reason it is often referred to as “common cause variation” (or common chance variation or quantifiable variation); 2) view foreseen uncertainty as not necessarily inherent in your system, but if it is, expect the numbers of these influences to be smaller, and more significantly, the probability of expected occurrence for each would be low, and for this reason it is often referred to “assignable cause variation” (or special cause variation or non-quantifiable variation). In general, foreseen uncertainty are events that you’ve seen happen in your projects or other projects, so even though rare, you can “foresee” them happening, and therefore plan for them using risk management practices.

When I’ve seen risk management used explicitly to address foreseen uncertainty, the focus is on identifying, classifying, evaluating (quantifying), and then planning accordingly to reduce (avoid) potential for occurring or to reduce (mitigate) the negative effects. This is typical of how you would use “insurance” for example to mitigate the effects of a car accident, a home fire, illness, or an early death, along with of course working to reduce these occurrences through performing regular car maintenance, following safe driving practices, monitoring fire hazards in you your home, having a smoke/CO detector, eating right, and exercising.

In a similar fashion, many (technical) practices are “done implicitly” for the most part in “iterative and incremental” managed projects to address foreseen uncertainty. Some examples include: nightly backups, tested backups, software configuration management, continuous integration, automated unit-testing, automated integration testing, automated acceptance testing, and continuous code refactoring. In fact, the iterations themselves are intended to be a form of risk management. However, iterations address even more the uncertainty type we’ll discuss next.

For now, recognize “iterative and incremental” project management practices can be used to provide risk management implicitly for a number of foreseen uncertainty influences, but explicit risk management is still necessary for some foreseen uncertainty influences on your projects. In particular, look for possible events (rare, but foreseeable influences) in the less-well-known (or not-well-known) parts of your project that are not addressed by the common or similar technical practices listed earlier. You are using a number of those above listed technical practices, right?

Iterations and Unforeseen Uncertainty

Iterations, we’ve talked about from the beginning, in particular distinguishing between “iterations” as a series of short “sequential” (mini-water fall) deliveries and “iterations” as a series of “iterative and incremental” deliveries. Since we’re exploring project management with a new frame (lens) here, experimenting a bit, we’ll move forward assuming for the moment we don’t know there is a difference, don’t see one for ourselves, or don’t see a significance (this may be “reality” for most of us). My take, for the moment, from the article by De Meyer et al. is that from a perspective of types of uncertainty, what is key is that a series of “sequential iterations” can be a useful project management practice for addressing the next type of uncertainty we’ll look at in our projects.

This brings us to the third uncertainty type listed in our table, unforeseen uncertainty. De Meyer et al. describe unforeseen uncertainty as influences that can’t be identified in project planning, or if “foreseeable”, they are considered so unlikely you don’t bother with contingencies, or even risk management. This sounds a little odd at first, but contrast this with our earlier context of foreseen uncertainty and “insurance.” You don’t buy insurance for every “conceivable” negative influence remotely possible in your life (in some cases “insurance” doesn’t help you anyway).

These influences, remotely possible, and in some cases never imagined, we’ve seen experienced by others or ourselves, but only faced, when they occurred! During these times focus gets narrowed and acute, perspectives get shifted or re-oriented, and “planning horizons” often get shortened or very short (one day or one week at a time). There is still stability in terms of some assumptions and goals in these instances, but we are in a mode of coping with new situations or information, learning of still more new challenges, and required to explore new and other paths, while forgetting about pursuing others we thought were only recently still possible.

Unforeseen uncertainty results also from an unanticipated combination or interaction of a number of “foreseeable influences.” This is the case that most likely occurs on the not-well-known projects talked about earlier. When unforeseen uncertainty is encountered as we take on a not-well-known project, just as we’d similarly do in our personal lives, our project management practices too need to change to a more narrow focus or “bite-size” approach, and this is where “sequential iterations” come into play more significantly.

The more not-well-known the project, the shorter the planning horizons need to be, and the more frequent they must be revised. The project management perspective and role also “shifts away” from planning and monitoring progress of a few large milestones, and is “re-oriented toward” an emphasis on learning, adapting, exploring new paths or solutions, and creating along the way (more just-in-time, and incremental) new and smaller milestones (targets), and to a greater monitoring of assumptions.

To be clear here, with unforeseen uncertainty you go beyond using predictive or risk management as you see in traditional project management approaches; you incorporate sequential iterations, making them shorter and more frequent as unforeseen uncertainty influences increase, and you shift your perspective from planning and monitoring long range, to an emphasis of adapting, learning, exploring, monitoring assumptions, and reaching frequent incremental smaller milestones (targets).

In the software development contexts I’ve most recently worked in over the last seven years, sequential iterations are the predominant project management practice used in particular because much of software development is related to implementing “never-done-before or new-to-us” features and often using “never-used-before or new-to-us” technologies as well. However, in these contexts, we continued to experiment, using variable (not uniform fixed-length) iteration lengths, and have also decoupled various project aspects (planning, development, incremental deliveries, retrospectives, etc.) from each other, as we discovered more effective iteration frequencies (natural “cadences”) for each aspect.

Project (Assumptions, Goals) Reviews and Chaos

Project reviews, as often described, are likely to be viewed as a status update on the project or an incremental product review. But in our context of discussing uncertainty types, I’m referring to a “review of the overall projects assumptions and goals.” That is, “re-thinking” the project assumptions or goals in whole, not just a  “refinement” of the project assumptions or goals.

This brings us to the fourth uncertainty type in our table, chaos. At first glance there may not appear to be any value talking about chaos in projects. Again, contrasting often helps us improve our understanding. De Meyer et al. describe chaos as “extreme uncertainty” and contrast it to unforeseen uncertainty by distinguishing along the lines of the project’s assumptions and project goals. Projects with extreme uncertainty have unstable overall assumptions and goals; projects with unknown uncertainty, while they can approach an appearance of being “chaotic” at times, for the most part maintain a degree of stability along key project assumptions and key project goals.

Projects with “extreme uncertainty” are also often referred to as research and development (R&D) projects. In these types of projects, extreme uncertainty is fully understood and expected. Project reviews in this case have little or nothing to do with obtaining status or “refining” stable project assumptions and goals, but rather much more about “re-thinking” and “re-defining” overall project assumptions and goals based on the latest information. Often, from R&D projects the original “objectives and hoped-for results” are not the end product from these efforts.

I don’t have experience working in a full R&D environment. However, I’ve been a part of team that worked on a project where the original goal was something that had not been done in the business domain. Over the course of the project the main overall goal remained stable, but many project assumptions and sub-goals along the way were unstable, and in fact for a time we were working in an R&D (very chaotic) mode. The team delivered on the project’s primary goal, reasonably near (within months, not years) the company’s desired timeframes, and it was in fact a “first ever” type of product release in this business domain.

While the end result was a successful product release, it was also definitely a narrower and “much refined” version from many of the product targets conceived along the way. It too was the most exciting (fun), challenging (demanding) project I’ve worked on, and as you might imagine provided me a great opportunity to learn a tremendous amount about managing (not controlling) and leading a project with a great deal of unforeseen and, at times, extreme uncertainty.

Closing Thoughts

Look for “differences” between the projects you manage and be open to considering that even a single project may not be “entirely” well-known or not-well-known. Experiment with using measures of variance to “aggregate” small numerous influences inherent in your project, and eventually across projects, and see if they enable you to create, over time with less effort, more predictable “milestones” over longer planning horizons.

View iterations, and in particular the technical practices they enable and that you’d implement, as implicit risk management to address foreseen uncertainties (negative influences that are rare but you see happen); but, also recognize where explicit “traditional” risk management practices are still needed in your project for these types of uncertainties (insurance where needed). When a project has the characteristics of a not-well-known project we discussed, experiment with iterations, if you’re not already, and shorten planning horizons to a point where you’re more regularly meeting “milestones.” [My first “iterative” effort divided a traditional managed project with a twelve month planning horizon into three iterations, each four months long; in time we did evolve to delivering incrementally in the range of weeks].

But more importantly alter your role (and other’s perspective) to being more focused on adapting, learning, and exploring alternate paths or solutions. Lastly, as you manage your changing projects, continue experimenting and tweaking your practices while keeping the quotes at the top of the post in mind!

Take care,

Frank