Kanban Results – Part 3 – From Scrum to Kanban
I have previously blogged results from our largest Kanban team.
Now its the turn of another team who had been using Scrum up until April 2009. They were recording data (Lead Time, Engineering Time, Development Time, Live Defects) during their time with Scrum and then from their time using Kanban. Its interesting to see the difference in stats since the switch, which the SPC charts below depict. The charts are split between pre and post April 2009.
The major changes in process the team went through during the transition from Scrum to Kanban were:
- Whilst using Scrum the team were carrying a hefty Product Backlog, with lots of time spent defining acceptance criteria and estimation (in points) up front, only for these work items to sit in the backlog for long periods of time. When using Kanban the team moved to a just in time model, which resulted in the team being able to spend more time on development, and less time in upfront analysis and less time in planning and estimation sessions.
- Items were batched (during sprints) for testing when using Scrum, the first week and a half the team would focus on development with items batching in QA. Work would be branched into a release candidate towards the end of a sprint, then tested, and in process bugs fixed. Releases were also batched. After moving to Kanban the team moved to a develop, test and release on demand model. Significant efforts were made to reduce and automate the release process once the sprint shackles were freed, in addition a suite of automated regression tests were written to add confidence in a release on demand model. This added agility to respond to customer demand. The QA became more involved througout rather than just towards the end of the sprint, with the developers on the team also testing work items to alleviate bottlenecks in QA due to their upstream development stations becoming blocked.
- Lengthy planning and estimation sessions (every 2 weeks) were dropped. Instead planning and estimation happened on demand.
- 2 weekly retrospectives were kept but with the addition of the team talking about Lead Times and Cycle Times using SPC charts from data being captured.
- Blockers were elevated to first class items. Unfortunately the number of blockers and their length were not recorded by this team.
- Rigid 2 weekly Sprint demos were dropped, instead demos were completed on demand.
- Work in process was limited. As per my last post, once again Little’s law came into affect which can be seen in the Development Time SPC chart below.
- Focus became on finishing work in progress rather than starting new work. Following the mantra; Stop starting and start finishing!
- Stories were grouped under MMFs, with the number of MMFs in progress at one time also limited. MMFs in progress were made visible on the board.
- The Kanban board went through several evolutions to make both the work and the workflow visible.
- The team started using the board during standups, discussing the work, rather than standing in a circle near the board doing the normal what I did yesterday, doing today, round robin.
- The team felt they no longer were tied to arbitrary deadlines, which reduced stress and the feeling of shoehorning the last remaining items in before the sprint completed.
Its also worth noting that this team have lost 3 developers since March 2009 (who left the company). The product consists of a large code base with much of the front end considered difficult to change. Yet the performance of the team shown below shows a consistent improvement. In addition this team is one of the few in our organisation who has integrated both UI creative/design and development into their process. Both of these require specialisms which were mapped on to the board, with hand offs between each.
Lead Time
Lead time has reduced from a mean of 25 working days to 14 working days over the past year. There is a large drop in the spread of variation, with the upper control limit dropping from 82 to 36, over a 50% drop in spread. There is a consistent downward trend with the majority of the most recent items under the mean.
6 of the outliers in 2008 were technical debt items with a lower class of service, the rest were waiting a long time in the Product Backlog. In 2009 3 of the outliers were technical debt items with a lower class of service, the others were items batched (at the request of the customer) into one significant release.
Engineering Time
Engineering time has reduced from a mean of 10 days to 4 days over the past year. There is a large drop in the spread of variation, with the upper control limit dropping from 33 to 12, over a 60% drop in spread. There is a consistent downward trend with the majority of the most recent items under the mean.
The majority of outliers in 2008 were technical debt items with a lower class of service or items that were de-prioritised. In 2009 4 of the outliers were technical debt items with a lower class of service, the others required root cause analysis.
Development Time
Development time has reduced from a mean of 5 days to 2 days over the past year. There is a large drop in the spread of variation, with the upper control limit dropping from 16 to 7, once again over a 50% drop in spread. This portion of the value stream was directly under the team’s control and not subject to delays from 3rd parties or upstream or downstream parties. The major factor in reducing development time has been to limit work in process.
The majority of outliers in 2008 were technical debt items with a lower class of service or items that were de-prioritised. In 2009 4 of the outliers were technical debt items with a lower class of service, the others required root cause analysis of which 2 were found to be special cause.
Live defects per week
We need to ensure that the reduction in lead and cycle times are not at the expense of quality. The chart below shows that the number of live bugs is within statistical control, and since April we are actually seeing a reduction. A large factor in this was due to investing in automated testing and paying down technical debt.
David,
thanks for sharing this. It’s really useful to see how the change in development processes impacted a team in an example.
I’ve been mentioning your recent posts about results to a few people recently and someone asked me about how the team members have been perceiving this approach. Have you been tracking any morale or satisfaction indicators?
Not specifically no. The teams regularly suggest improvements themselves which you could argue is a signal of buy in. Morale in the second team suffered somewhat due to external pressures being applied to that team, resulting in 3 of the developers leaving over a short space of time. These were not as a result of Kanban however. That said the same team also delivered regularly and received praise from customers using major new functionality.
Great stuff David. Thanks for publicizing the data!