The video from my Journey to Systemic Improvement talk at the UK Lean eXchange conference is now available.

I was originally told I had 50 mins to present plus questions, then was told just before going on I had 40 mins, so zipped through it!!!

A video of Benjamin Mitchell and I running the Red Bead Experiment in the openspace session towards the end of the day is also available. If you missed our previous “In the brain of session” where we ran the Red Bead Experiment, you can find this here (note the sound quality for this first experiment is much better).

Finally Benjamin and I interviewed each other after the Lean eXchange event, which can be viewed here.

Its always interesting when presenting to get instant feedback from those in the audience using twitter. Here is a selection of the tweets.

worldofchris @dpjoyce Red Bead Exp. was great as was your preso. #leankanbanx

christianralph “over 50% of work is from failure demand (rework, bugs, complaints) – Vanguard” @dpjoyce #leankanbanx

lunivore “If current work uses IT then leave it, work with it or treat it as a constraint.” @dpjoyce #leankanbanx

skirk @dpjoyce mentions failure demand at #leankanbanx, after @agilemanager talking about “failure modes”. There’s clearly something in the water.

lunivore “Understand, improve, then ask if IT can further improve. Large gains from > thinking around design & mgt of work.” @dpjoyce #leankanbanx

christianralph “understand and improve, then ask if IT can improve further. ” @dpjoyce #leankanbanx

skirk From @dpjoyce at #leankanbanx “Since IT can, should it?” – good question!

lunivore “Outside-in: Sales, Mkting, Finance, HR, IT” – interesting use of the BDD term from @dpjoyce #leankanbanx

lunivore Results from @dpjoyce’s #kanban available here: http://is.gd/58UkV Also really like @agilemanager’s feedback on same page. #leankanbanx

Jazzatola This is what I was getting at. You need “…a team willing to reflect, adapt and improve.” @dpjoyce #leankanbanx

marcjohnson RT @lunivore: “<Change> based on a set of principles – better practice, not best practice.” @dpjoyce #leankanbanx

christianralph “BBC introduced a kaizen board, to collect improvements that can be played during downtime” @dpjoyce #leankanbanx

lunivore “Phase 1, no limited WIP but bugs, blockers, anything which stops you hitting the deadline visible.” @dpjoyce #leankanbanx

lunivore “Standups – 20 people in < 15 mins – they’re enumerating the work, not the people.” @dpjoyce #leankanbanx

scottcowan “chargeable and non-chargeable queues, where you need a minimum number of chargeable cards in progress” @dpjoyce #leankanbanx

lunivore “The kanban ‘flu’ soon spreads to other teams.” @dpjoyce #leankanbanx

christianralph “BBC upgraded blockers to 1st class citizens on the kanban board” @dpjoyce #leankanbanx

worldofchris “Stop Starting, Start Finishing” @dpjoyce #leankanbanx

christianralph “planning sessions act like a handbrake to teams momentum and end of each iteration” @dpjoyce #leankanbanx

benjaminm “Iterations were like pulling a handbrake on the team every fortnight” fighting talk on iterations by @dpjoyce #leankanbanx

joakimsunden Time for @dpjoyce “A Journey to Systemic Improvement”. #leankanbanx

Today I gave a talk at the UK Lean eXchange entitled Journey to Systemic Improvement.

My slides can be found here.

Note it is a media rich presentation so the PDF is almost 50MB!!!

A video recording of the presentation and our second running of the Red Bead Experiment will soon be available.

Benjamin Mitchell and I recently ran the Red Bead Experiment at Skills Matter.

We changed the experiment slightly to revolve around a software project rather than a factory:

• Product Owner and a Project Manager jointly played the role of Foreman
• A burndown of white beads was referred to
• Discussion afterwards focused in part on software based projects

A video of the session can be found here.

If you would like to run the session yourself then you may find our script of use, along with the F-Test, motivational posters and lessons. If you would like the spreadsheet that creates the control chart and burndown then please mail me as WordPress wont allow me to upload. You can buy the Red Bead Experiment kit here. Note we also supplied our own plastic box for decanting the beads, a tray to catch spillages, clipboards and pens for the inspectors.

The Transformation Forum run a Red Bead Experiment masterclass for those looking to learn how to run the experiment and meet with others who have done so over many years.

I have previously blogged results from our largest Kanban team.

Now its the turn of another team who had been using Scrum up until April 2009. They were recording data (Lead Time, Engineering Time, Development Time, Live Defects) during their time with Scrum and then from their time using Kanban. Its interesting to see the difference in stats since the switch, which the SPC charts below depict. The charts are split between pre and post April 2009.

The major changes in process the team went through during the transition from Scrum to Kanban were:

• Whilst using Scrum the team were carrying a hefty Product Backlog, with lots of time spent defining acceptance criteria and estimation (in points) up front, only for these work items to sit in the backlog for long periods of time. When using Kanban the team moved to a just in time model, which resulted in the team being able to spend more time on development, and less time in upfront analysis and less time in planning and estimation sessions.
• Items were batched (during sprints) for testing when using Scrum, the first week and a half the team would focus on development with items batching in QA. Work would be branched into a release candidate towards the end of a sprint, then tested, and in process bugs fixed. Releases were also batched. After moving to Kanban the team moved to a develop, test and release on demand model. Significant efforts were made to reduce and automate the release process once the sprint shackles were freed, in addition a suite of automated regression tests were written to add confidence in a release on demand model. This added agility to respond to customer demand. The QA became more involved througout rather than just towards the end of the sprint, with the developers on the team also testing work items to alleviate bottlenecks in QA due to their upstream development stations becoming blocked.
• Lengthy planning and estimation sessions (every 2 weeks) were dropped. Instead planning and estimation happened on demand.
• 2 weekly retrospectives were kept but with the addition of the team talking about Lead Times and Cycle Times using SPC charts from data being captured.
• Blockers were elevated to first class items. Unfortunately the number of blockers and their length were not recorded by this team.
• Rigid 2 weekly Sprint demos were dropped, instead demos were completed on demand.
• Work in process was limited. As per my last post, once again Little’s law came into affect which can be seen in the Development Time SPC chart below.
• Focus became on finishing work in progress rather than starting new work. Following the mantra; Stop starting and start finishing!
• Stories were grouped under MMFs, with the number of MMFs in progress at one time also limited. MMFs in progress were made visible on the board.
• The Kanban board went through several evolutions to make both the work and the workflow visible.
• The team started using the board during standups, discussing the work, rather than standing in a circle near the board doing the normal what I did yesterday, doing today, round robin.
• The team felt they no longer were tied to arbitrary deadlines, which reduced stress and the feeling of shoehorning the last remaining items in before the sprint completed.

Its also worth noting that this team have lost 3 developers since March 2009 (who left the company). The product consists of a large code base with much of the front end considered difficult to change. Yet the performance of the team shown below shows a consistent improvement. In addition this team is one of the few in our organisation who has integrated both UI creative/design and development into their process. Both of these require specialisms which were mapped on to the board, with hand offs between each.

Lead Time

Lead time has reduced from a mean of 25 working days to 14 working days over the past year. There is a large drop in the spread of variation, with the upper control limit dropping from 82 to 36, over a 50% drop in spread. There is a consistent downward trend with the majority of the most recent items under the mean.

6 of the outliers in 2008 were technical debt items with a lower class of service, the rest were waiting a long time in the Product Backlog. In 2009 3 of the outliers were technical debt items with a lower class of service, the others were items batched (at the request of the customer) into one significant release.

Engineering Time

Engineering time has reduced from a mean of 10 days to 4 days over the past year. There is a large drop in the spread of variation, with the upper control limit dropping from 33 to 12, over a 60% drop in spread. There is a consistent downward trend with the majority of the most recent items under the mean.

The majority of outliers in 2008 were technical debt items with a lower class of service or items that were de-prioritised. In 2009 4 of the outliers were technical debt items with a lower class of service, the others required root cause analysis.

Development Time

Development time has reduced from a mean of 5 days to 2 days over the past year. There is a large drop in the spread of variation, with the upper control limit dropping from 16 to 7, once again over a 50% drop in spread. This portion of the value stream was directly under the team’s control and not subject to delays from 3rd parties or upstream or downstream parties. The major factor in reducing development time has been to limit work in process.

The majority of outliers in 2008 were technical debt items with a lower class of service or items that were de-prioritised. In 2009 4 of the outliers were technical debt items with a lower class of service, the others required root cause analysis of which 2 were found to be special cause.

Live defects per week

We need to ensure that the reduction in lead and cycle times are not at the expense of quality. The chart below shows that the number of live bugs is within statistical control, and since April we are actually seeing a reduction. A large factor in this was due to investing in automated testing and paying down technical debt.

Following on from my recent Kanban results post more information is provided below in regards to the teams focus on handling blockers, which is a vital part of improving flow.

In Scrum we are taught to keep an impediment list:

Anything around a Scrum project that impedes its productivity and quality is an impediment. It is the responsibility of the ScrumMaster to remove any impediment that is stopping the team from producing production quality code. The impediment list is simply a set of tasks that the ScrumMaster uses to track the impediments that need to be solved.

David Anderson however has another take:

Another aspect of Agile methods we can look at is how they handle impediments or blocking issues. The early Agile literature didn’t have a lot to say on this but pointed out that impediments should be raised at a morning standup meeting, or scrum. Very few teams treat impediments or issues as first class work items. They do not focus on tracking of the issue behind the block or for any escalation path to be implemented or tracked. Agile teams encountering an impediment would generally mark an item as blocked and go on to another one.

This is not the behavior you would expect on a Kanban team truly limiting WIP. Because the WIP limit will have been reached, an impediment causes idle time. The only course of action available to maintain flow is to pursue the impediment, track down its cause and resolve it. In a truly WIP limited process impediment removal is paramount.

As I mentioned in the previous post, the team changed their daily standup to talk about blockers first, and actively assigned, tracked, escalated and removed these blockers. Evidence of this is shown below, once again in statistical process control charts.

Number of working days blocked

The number of working days items were blocked has reduced from a mean of 26 days to 5 days over the past year. There is a consistent downward trend with the majority of the most recent items under the mean. The outlier in 2008 was a result of waiting for a 3rd party to complete their work (a special cause), powerful data to use when discussing performance with those 3rd parties in our quarterly meetings. As in the previous post the periods on the charts have been split from 2008 until our financial year end (April 2009), and from July 2009 until October 2009.

Blockers per week

The chart below shows that the number of blockers is within statistical control. We are in fact observing an increase in the number of blockers raised, however as the chart above shows these are being removed at a faster rate by the team.

Once again this data was used in our retrospectives and quarterly reviews. Reoccurring blockers were investigated and route cause analysis performed. Outliers were discussed as were those the team deemed worthy of discussion.

Great feedback below from David Anderson on my recent Kanban results post.

It is really interesting to hear this feedback, as to us in the team these changes have become almost natural and the norm. For example we regularly use SPC chart data in our retrospectives and quarterly reviews, the team often suggest changes to help improve the process, starting new work is often rejected with the focus on completing work in progress first. More recently the team are starting to record touch time to see where they can further improve and drive out waste, and to understand and highlight dependencies between work items.

These improvement have obviously required a team willing to try something different (from Scrum which they were doing before) and want to continuously improve. We also have an experienced BA in the team that works very closely with those upstream and whom has an extremely detailed knowledge of the lines of business that we deliver MMFs to. A focus for improvement has also been with those downstream who release MMFs into the live environment. Finally to complement the improvements in process the team has strived hard in the last year to improve their engineering practices; pairing, Test Driven Development, continuous integration, Automated Acceptance tests, improved configuration management, improvements in coupling and cohesion, to a point where they can release on demand with very low transaction and coordination costs for each release.

There are similar improvements from our other team who are using Kanban, and I will be blogging about these over the next month or so.

A further note is that Dr Peter Middleton is writing an academic paper on our results which will be published shortly.

From David Anderson…

After Lean & Kanban Miami in May, Corey Ladas tweeted that he’d like to see fewer cumulative flow diagrams and more statistical process control charts from teams doing Kanban. At the time, I commented that we had to be patient and that teams would mature to the point where they felt the need for statistically defensible methods such as SPC.

We are beginning to see that maturity develop in the community and it is mostly happening in London. Benjamin Mitchell has been carrying around some SPC charts from his team at BNP Parisbas and showing them to anyone who’ll listen at Ltd WIP Society and XTC meetings. Now David Joyce from BBC Worldwide has blogged results from 12 months of Kanban introduction. He chose to use SPC charts to show the improvements in lead time and defect rates.

I’ve posted this response on my own blog

I did so because I want people to see beyond the basic numbers. While the improvements are definitely worthwhile and I am sure the reduction in lead times, increased number of releases and improved quality is significantly improving business agility and customer satisfaction, David’s data contains a lot more information. It contains further empirical evidence that teams using Kanban achieve high levels of maturity and do so within time frames hitherto unreported in the literature.

I have some thoughts on why this is so. I talked about them at SEPG in San Jose in March. For example, Kanban sets an expectation of quantitative management through its transparency and availability of data. It sets an expectation of kaizen culture (continuous improvement driven from the shop floor) by providing models like bottleneck management, waste reduction and variability reduction that enable teams to visualize and implement improvements. It sets an expectation of flow and the maintenance of flow through swift issue escalation and resolution. And it encourages root cause analysis and resolution to maintain flow and improve predictability.

From the beginning Kanban sets an expectation of high maturity behaviours. Behaviours that appear in levels 4 and 5 of the CMMI. It appears that by doing so from the outset, this really does accelerate improvement and achievement of higher maturity levels.

Our community needs more stories like this. If you have one, please submit it when the call for papers is issued for Lean Software & Systems 2010 in Atlanta April 21-23. We’d love to have you present at the conference.

Meanwhile, a cap doff to David and his team and the remarkable changes that are taking place at the BBC’s commercial arm.

David

http://www.channelkanban.com/

Over the past year our Kanban teams have been striving to reduce the following:

• Lead Time – the time it takes from a customer request to when it is delivered
• Development time – the time it takes from entering the Ready For Development queue to when it is handed off to QA
• Engineering time – the time it takes from entering the Ready For Engineering queue to when it has passed QA, left Engineering, and is ready for UAT

Through various means; working on the system, talking about blockers first in the standup, actively assigning, escalating and removing blockers, recognising and reducing bottlenecks, retrospectives, improving our process by separating common cause problems from special cause problems, using MMFs and component stories and tasks, implementing Kaizen, implementing classes of service, highlighting items that have been on the board for too long, to name but a few, we have seen improved results which are depicted below in Statistical Process Control charts using data taken from our largest Kanban team.

Note the links in the above paragraph link to other areas of this blog that describe in more detail how each of these have been achieved. You can click on each of the charts below to see a larger version.

Lead Time

Lead time has reduced from a mean of 22 days to 14 days over the past year. There is a consistent downward trend with the majority of the most recent items under the mean. Each of the outliers were proved to be special cause. The periods on the charts have been split from 2008 until our financial year end (April 2009), and from July 2009 until October 2009.

Development Time

Development time has reduced from a mean of 9 days to 3 days over the past year.There is a consistent downward trend. This portion of the value stream was directly under the team’s control and not subject to delays from 3rd parties or upstream or downstream parties. The major factor in reducing development time has been to limit work in process. The periods on the charts have been split from 2008 until our financial year end (April 2009), and from July 2009 until October 2009.

Engineering Time

Engineering time has reduced from a mean of 11 days to 8 days over the past year. Once again there is a downward trend. However there are more outliers that required investigation. Some of the outliers were proved to be special cause, but the majority were down to waiting for 3rd parties to complete their development and QA, something the team actively worked on to reduce. The periods on the charts have been split from 2008 until our financial year end (April 2009), and from July 2009 until October 2009.

Throughput

We class throughput as the number of items released, and would expect an upward trend as the code base is decoupled, work items broken into MMFs, and cycle time reduces. The chart below shows this upward trend in the number of releases per month. Note that we are subject to release freezes hence the drop from December to February where the release freezes were imposed.

Bugs Per Week

We need to ensure that the reduction in lead and cycle times, and increase in throughput are not at the expense of quality. The chart below shows that the number of live bugs is within statistical control, and since July we are actually seeing a reduction.

There are now several follow on posts from this original post

http://leanandkanban.wordpress.com/2009/10/25/kanban-results-feedback/

http://leanandkanban.wordpress.com/2009/10/26/kanban-results-part-2/

I was asked by my COO to create a one pager on Systems Thinking, Real Options and Agile principles, for discussion on how they could be used in our organisation. I thought I would share the results.

Making Measurement work for you workshop – Jeremy Cox

The final part of the day was a workshop ran by Jeremy Cox who is a Vanguard consultant.

He started of by explaining there are 3 types of waste in a system

Type 1 – you can just stop (if you have the authority) for example appraisals
Type 2 – you need to design out of the flow
Type 3 – you need to do to survive

Vanguard have found that IT Systems that we depend on so much are high wired into current business silo targets (not measures), this then makes removing the targets harder to do (and more costly). One of the audience, an IT Systems Manager mentioned that he has seen targets around page views and impressions for web sites. This resulted in the sites being made more complicated to use, which meant people were clicking around more and the overall stats went up! If simplifications were made to the site they would suffer as their stats had gone down!
In Vanguard’s experience organisations spend a fortune on IT and have no knowledge on how it is affecting performance.

In the Vangaurd literature there is much talk of the “customer”. Jeremy explained there may be more than one customer and if this is the case in your organisation to not try to create a universal customer. The word customer may be wrong in your context too, it could be consumer or patient for example. By taking an outside in approach you may find the “customer” is different per area or touchpoint.

Measures

The key question to be asked is “have you got measures that enable you to learn and improve?” if not then they are arbitrary.

Measures need to be developed with the staff, use this guiding principle; who are your customers and what do they want?

During the session Jeremy used a framework that he has found useful when applying Systems Thinking; Principles, practice, issues.

Measures principle 1 – Purpose -> Measures -> Method

We brainstormed current targets in our organisations. It is usual for them to be set at Method i.e. bottom up. You are told it will be done like x and a measure comes out of it. Other times we start at Measures and come up with a target. Both of these result in a defacto purpose.
The advice is to start with Purpose, this helps you develop measures and liberates method. ALL of your measures should be firmly routed in purpose.

Remember removing targets doesn’t mean  “no measures”.  We arent suggesting to remove measures, but to create new measures derived from a customers point of view.

Changing measures requires senior management buy in. The suggested practice is to add more useful measures to the existing targets you have to report now, then over time reduce the old targets as people realise they are becoming less and less useful. Its important to note that between the worker and senior management there are various levels of performance managers, they are likely to cling to old targets and become perturbed by the change to measures. They need to experience the normative loop and discover themselves why the new measures are better for the customer and your organisation’s service.

What does good service currently look like in your organisation? When going out into the work ask the following questions “how often are we able to say yes?”

How do you study purpose? Go out into the system and study demand, what matters to your customer. Look at Value Demand vs Failure Demand, go and talk to your customers.

When setting measures you may not get it right first time, review your data regularly, is Failure Demand dropping? are end to end times reducing? are your customers happier?

What do you measure and how do you do it?

By design target based measures prevent learning and improvement. Regularly ask the following question “do the measures that are currently in use lead you to learn and improve?”
A leader should follow this up with the following questions

1. Show me your methods
2. What are the problems you have discovered? What have you learnt?
3. What is your plan to deal with these problems?
4. What help do you need from me?
5. What decisions do people make with the data?

Asking the above will enable you to easily see if the current measures are useful.

What do good measures look like? a good answer to the above questions would be for example “we have studied demand over the last week and have found a new Failure Demand that is predictable and preventable. This is how we plan to reduce it”

Its common when you ask these questions you find there are no measures in place, or there are too many measures, or people are wary and you get a negative reaction as they suspect you might be inspecting them. You should ask both staff and managers, you will more than likely get different answers. Never take people only at their word, they may say something and do something different, watch the work as well to see what is truly happening. You have to roll up your sleeves and go out into where the work is done.

Jeremy advised to stick to the principles but not rigidly to the Vanguard method, you will have to compromise during the journey.

Change Thinking

This is intervention theory, this is the really hard stuff!

You can read, Check, Plan, Do as

• Get Knowledge
• Build the desire and confidence to try an alternative
• Make it normal

The Check stage in the Vanguard method is an “unlearning” process for staff, people will unlearn at different rates, or in some cases be unwilling.

Some managers will jump straight from Check to Do and set new targets. One of the first reactions to seeing true end to end times following Check is for managers to set new targets to reduce them.

Remember, are the measures defined and used in the work? do the people who do the job work with their managers to set measures? They have to be set in conjunction with those that are located in the work.

Measures, work design and job roles are all interlinked, they are mutually enforcing. With any intervention you always change these 3.

One technique that works is to put measures up on a board near the team, they all review regularly, use them to improve, update them and modify the work and the roles accordingly. Use carefully and gain buy in from the team. Dont just put targets on a board and point to them!

Cost/Benefit where are the boundaries

The business case should contain Revenue – Waste costs = Savings. Its important to do Check (at least a pilot) first to help with a business case. You never know the boundaries of your system until you start studying it. You will find that bits of the system will create demand for other bits of the system, so follow the work. Agree a boundary for change with the sponsor.

Don’t spend enormous amounts of time trying to get *exact* Failure demand figures for the business case.

The question that is always asked is “how much are we going to save?” You cant quantify this up front, change is emergent. Engage with the sponsor and agree on what they want out of it and focus on this. A pilot Check will give enough indication to present to a sponsor to see if we should proceed.

A fundamental part of Check is engaging with senior leaders and agreeing scope. Don’t start too large, the faster you go after waste the sooner you will know how long it will be to take it out.