“If you can’t measure it, you can’t improve it” so goes the saying. This has been the stalwart response which has inhibited many an attempt to improve Joy in Work. And as we know, without Joy In Work, attempts to improve patient safety and cost effectiveness with be limited in their impact.
We hope to use this blogpost to challenge your thinking and offer responses to those who ask you “how do you measure 15s30m?”
The mainstay of 15s30m is that it is accessible for any member of staff. As soon as we start showing graphs, SPC charts, it inhibits their confidence to go and make a change.….so we’ve taken a different approach.
Do we measure everything we do?
Did you give your child or niece or nephew a birthday present when they turned 1? And again another present on their second, third, fourth birthdays….presumably you quantitavely measured the joy the initial gift gave, and used this data as confirmation to proceed in subsequent years, analysing the mean Joy quotient, comparing the effect of different size, quantity and expenditure on each cycle of giving…. No? But you give a gift anyway don’t you…..
What’s in your basket?
QSIR recommend using a basket of measures to capture a range of data across outcome, process and structure.
Here are the 15s30m measures we’ve used:
Structure : Staff awareness, Twitter followers
Outcome : Frustration score before and after a mission
Pre and Post workshop questionnaire
Process : Number of workshops, number of missions
How did we get on?
Are you a movement if no one is following you?
Early on we were warned about ‘vanity metrics’ of social media: how many likes, retweets and followers you have does not tell you much about your impact. It did however give us a way of assessing what sort of posts resonated with our community (they like a mix of inspiring quotes, practical advice and storytelling examples). Using Twitter analytics, we could also tell what time of day they are most likely to see them and adjust our twitter strategy accordingly. This was our first set of PDSA cycles – you see, we can do ‘proper’ QI!
We also knew that most of our target audience weren’t going to be on twitter; we needed to find out how many of our staff knew about 15s30m. For this we went back to the tried and tested clipboard-and-pen methodology, standing on the hospital concourse at lunchtime and stopping people for a few seconds. We also recruited most of our workshop participants off the back of these exercises so it was time well spent. We learnt that the prime-time was 1145-1230, and over 4 sessions staff awareness increased from 0% to 64%.
How do you measure frustration: you only need to ask
Having researched options for measuring the opposite of joy, frustration, we fell upon this helpful comment on a blog “if you want to know how an emotion is being felt, now, in the moment, then the most reliable way is to ask just that, just then”. So we did: for the Clinic set-up boxes Mission we asked ten nurses immediately after they had set up clinic “How frustrating do you find this task, out of 5”. We then introduced the new clinic boxes, and a few days later, asked the same question, again immediately after they had set up the clinic.
This metric is simple, transparent, requires no analysis and produces exactly the evidence you need to sustain a change in process.
Does attending a 15s30m workshop change your approach to #JoyInWork?
We are working with @ImproveWellUK and @RCHTWeCare to validate a short pre and post workshop questionnaire which is completed by staff who attend. In general, the empowerment and the realisation that THEY are the person who can change the way they feel at work increases their personal joy. The questionnaire is available under the workshop tab on the website www.15s30m.co.uk – give it a try and do share your results with us so we can build a wider picture.
Does 15s30m impact #JoyInWork for an organisation?
This was our initial starting point for measurement: could we prove that 15s30m has the impact we aspire it to? Large scale, £100k funded studies on measuring Joy in Work have been unable to demonstrate significant change even when direct changes to environment, pay staffing levels and systems have been introduced. It’s for the same reason that using the NHS Staff Survey is unlikely to pick up the scale of impact we are having. Quite simply, there are so many extrinsic factors which affect individual answers to these questions: a recent parking ticket; a difficult day; disappointment in promotion hopes on any given day.
But what we can easily measure is how many staff have attended; how many missions have been suggested; how many people have adopted a 15s30m approach without having attended a workshop.
Plus, there’s evidence that the scale and spread of 15s30m outstrips every other QI initiative that trusts have tried
A different approach: qualitative measures are the future if we want to engage staff.
We haven’t found a lack of quantitative data a barrier to trusts wanting to adopt 15s30m. It’s the qualitative measurement – the stories which capture people’s hearts and minds: the tea lady whose face lit up when she talked of having the freedom to offer relatives a cuppa; the clinic nurse who proclaimed “I no longer drive into work dreading the start of the day”; the buzz around a department when we post a cog out to a hero whose idea has spread – that’s the value of 15s30m – helping people feel valued in themselves.
So no, its not easy to measure the impact of 15s30: but we don’t think that is a good enough reason not to do it.
There is no single validated measure of joy in work. IHI signposts many of these here http://www.ihi.org/communities/blogs/measure-joy-in-work.