Making Measurement work for you workshop – Jeremy Cox
The final part of the day was a workshop ran by Jeremy Cox who is a Vanguard consultant.
He started of by explaining there are 3 types of waste in a system
Type 1 – you can just stop (if you have the authority) for example appraisals
Type 2 – you need to design out of the flow
Type 3 – you need to do to survive
Vanguard have found that IT Systems that we depend on so much are high wired into current business silo targets (not measures), this then makes removing the targets harder to do (and more costly). One of the audience, an IT Systems Manager mentioned that he has seen targets around page views and impressions for web sites. This resulted in the sites being made more complicated to use, which meant people were clicking around more and the overall stats went up! If simplifications were made to the site they would suffer as their stats had gone down!
In Vanguard’s experience organisations spend a fortune on IT and have no knowledge on how it is affecting performance.
In the Vangaurd literature there is much talk of the “customer”. Jeremy explained there may be more than one customer and if this is the case in your organisation to not try to create a universal customer. The word customer may be wrong in your context too, it could be consumer or patient for example. By taking an outside in approach you may find the “customer” is different per area or touchpoint.
The key question to be asked is “have you got measures that enable you to learn and improve?” if not then they are arbitrary.
Measures need to be developed with the staff, use this guiding principle; who are your customers and what do they want?
During the session Jeremy used a framework that he has found useful when applying Systems Thinking; Principles, practice, issues.
Measures principle 1 – Purpose -> Measures -> Method
We brainstormed current targets in our organisations. It is usual for them to be set at Method i.e. bottom up. You are told it will be done like x and a measure comes out of it. Other times we start at Measures and come up with a target. Both of these result in a defacto purpose.
The advice is to start with Purpose, this helps you develop measures and liberates method. ALL of your measures should be firmly routed in purpose.
Remember removing targets doesn’t mean “no measures”. We arent suggesting to remove measures, but to create new measures derived from a customers point of view.
Changing measures requires senior management buy in. The suggested practice is to add more useful measures to the existing targets you have to report now, then over time reduce the old targets as people realise they are becoming less and less useful. Its important to note that between the worker and senior management there are various levels of performance managers, they are likely to cling to old targets and become perturbed by the change to measures. They need to experience the normative loop and discover themselves why the new measures are better for the customer and your organisation’s service.
What does good service currently look like in your organisation? When going out into the work ask the following questions “how often are we able to say yes?”
How do you study purpose? Go out into the system and study demand, what matters to your customer. Look at Value Demand vs Failure Demand, go and talk to your customers.
When setting measures you may not get it right first time, review your data regularly, is Failure Demand dropping? are end to end times reducing? are your customers happier?
What do you measure and how do you do it?
By design target based measures prevent learning and improvement. Regularly ask the following question “do the measures that are currently in use lead you to learn and improve?”
A leader should follow this up with the following questions
1. Show me your methods
2. What are the problems you have discovered? What have you learnt?
3. What is your plan to deal with these problems?
4. What help do you need from me?
5. What decisions do people make with the data?
Asking the above will enable you to easily see if the current measures are useful.
What do good measures look like? a good answer to the above questions would be for example “we have studied demand over the last week and have found a new Failure Demand that is predictable and preventable. This is how we plan to reduce it”
Its common when you ask these questions you find there are no measures in place, or there are too many measures, or people are wary and you get a negative reaction as they suspect you might be inspecting them. You should ask both staff and managers, you will more than likely get different answers. Never take people only at their word, they may say something and do something different, watch the work as well to see what is truly happening. You have to roll up your sleeves and go out into where the work is done.
Jeremy advised to stick to the principles but not rigidly to the Vanguard method, you will have to compromise during the journey.
This is intervention theory, this is the really hard stuff!
You can read, Check, Plan, Do as
- Get Knowledge
- Build the desire and confidence to try an alternative
- Make it normal
The Check stage in the Vanguard method is an “unlearning” process for staff, people will unlearn at different rates, or in some cases be unwilling.
Some managers will jump straight from Check to Do and set new targets. One of the first reactions to seeing true end to end times following Check is for managers to set new targets to reduce them.
Remember, are the measures defined and used in the work? do the people who do the job work with their managers to set measures? They have to be set in conjunction with those that are located in the work.
Measures, work design and job roles are all interlinked, they are mutually enforcing. With any intervention you always change these 3.
One technique that works is to put measures up on a board near the team, they all review regularly, use them to improve, update them and modify the work and the roles accordingly. Use carefully and gain buy in from the team. Dont just put targets on a board and point to them!
Cost/Benefit where are the boundaries
The business case should contain Revenue – Waste costs = Savings. Its important to do Check (at least a pilot) first to help with a business case. You never know the boundaries of your system until you start studying it. You will find that bits of the system will create demand for other bits of the system, so follow the work. Agree a boundary for change with the sponsor.
Don’t spend enormous amounts of time trying to get *exact* Failure demand figures for the business case.
The question that is always asked is “how much are we going to save?” You cant quantify this up front, change is emergent. Engage with the sponsor and agree on what they want out of it and focus on this. A pilot Check will give enough indication to present to a sponsor to see if we should proceed.
A fundamental part of Check is engaging with senior leaders and agreeing scope. Don’t start too large, the faster you go after waste the sooner you will know how long it will be to take it out.