When to give people their report – revisited April 29, 2009
Posted by talentinnovations in 360 degree feedback, 360 feedback, Facilitation, Report.1 comment so far
In an earlier post, we described why we believe it is best practice to give people their 360 feedback report a day or two before a feedback coaching session.
Since 2007 we’ve been working with part of a large government department. They are using 360 feedback as part of a development programme, and they give all individuals a personal coaching session on the 360 as well as some other tools (eg. MBTI).
They had previously been in the habit of giving people their 360 report just before the session, then talking them through it. This is, after all, recommended practice for hard-to-interpret psychometrics like OPQ and the like. However, at our recommendation they tried switching to sending the 360 report to the individual a day or two beforehand. At the end of the year they did a rigorous review of their programme, asking individuals how they felt everything went. Here’s the results from their section on this question:
“I found it helpful to receive the 360 report in advance of the feedback session:”
Strongly Agreed: 56%
Agreed: 42%
Disagreed: 0%
Strongly Disagreed: 2%
Pretty conclusive, I think! Here are some quotes from the individuals:
”Although much of the insight for me came during the feedback, getting the report before allowed me to prepare properly.”
“Being able to analyse it and prepare questions or responses was excellent and also meant there was no fear of the unknown.”
“Not to have had time to examine the report and digest its content would have limited the effectiveness of the feedback session.”
“Value would be severely diminished without the opportunity to read and reflect upon the charts and specific comments.”
And here was their final conclusion from their ‘experiment’:
The vast majority agreed here, with some going as far to say it was essential to have the report before the feedback session, to allow sufficient time for preparation.
So if you’re considering doing any coaching of 360 feedback, we would urge you to send the report out beforehand, and hopefully the comments above will give you the confidence to do it!
Mark.
How long is a piece of string? September 24, 2008
Posted by talentinnovations in Uncategorized.Tags: 360 feedback, psychometrics, time
6 comments
A key decision in any 360° feedback project is how long to make your survey. Here I show how smaller isn’t always better.
The obvious argument for making a 360° survey smaller is to save people time, but this can come at the expense of depth, breadth and clarity of feedback. And we often see customers try to ‘squeeze together’ behaviours to reduce the number of questions. Here’s a real example we’ve seen from the standard 360° questionnaire (a mere 24 questions long) used by one of the biggest retailers in the UK:
“Gains commitment to achieving results through simple communication, actively listening and by adapting leadership style to the needs of others”.
I spot at least 4 different behaviours in there! A manager could easily be good at some but poor at others. This complexity makes it difficult for reviewers to understand and score the question, and hard for the recipient of the feedback to interpret the results. Both practically and psychometrically it is virtually useless!
Our approach is to work with our clients to develop a set of items that are simple and easy to understand, measure observable behaviours, and have the breadth to cover all of the important competencies. This can make for a longer survey, so what’s the time impact of all those extra questions?
We’ve been tracking how long people take to do their questionnaires, and have found some interesting results. The time taken to complete different aspects are as follows:
- Each rating on a 1-to-5 scale: 7 seconds
- Each word of written feedback: 7 seconds
It’s the written feedback that makes the biggest difference. Here are two real example projects we recently implemented:
| Project | Rated Items | Total written feedback (average) | Average time taken |
|---|---|---|---|
| HR Consultancy | 44 | 177 words | 19 mins |
| Engineering Company | 104 | 42 words | 14 mins |
Despite having less than half as many items, the HR consultants took longer to complete their surveys than the engineers!
So fewer questions does not necessarily mean less time, but as seen above, fewer questions can result in unclear, incomplete and hence less insightful feedback.
We are committed to enabling truly powerful feedback. Demands for “quick and dirty” solutions are best met with the question “what do you really want to achieve?”
Mark
Performance Management Tension April 14, 2008
Posted by talentinnovations in Performance Management.Tags: Annual Review, Development, Performance Management
1 comment so far
For a while now we’ve had a performance management module in our software. We think it’s pretty good – it links the person’s annual review with their 360 scores by competency, and allows HR managers to monitor everything easily. We developed it for one particular client, and they’ve recently had a bit of trouble with their ‘annual review politics’.
The basic problem is this: there is an inherent tension in doing performance reviews between consistency and openness. There are basically two ways of doing it:
- Have the review reflect an honest conversation between the manager and the individual about the individual’s performance over the year
- Have the review reflect a ‘calibrated’ measure of performance that fits correctly with everyone else’s performance reviews (and which can therefore be used to feed into bonuses / raises / etc.)
If you go for option 1 then the obvious thing to do is for the manager to only decide on the review score at the end of the review meeting. Whereas if you go for option 2 then it is important that the review score is agreed with other people (e.g. among the department leadership team) before the review meeting even happens.
Our client’s managers were using option 1, but then they all got together and realised that some managers had been far too generous, and so they had to ‘recalibrate’ everyone’s reviews, which was obviously very disappointing for some individuals who thought they were in line for fantastic bonuses! In other words, the reality (that performance reviews have to be consistent and agreed among many stakeholders) undermined the ideal (that performance reviews reflect the conversation between the manager and the individual).
So anyway, we’ve been thinking we’d like to redesign our performance management module more closely around a new sort of process that would try to overcome this problem. We would actually have 2 separate review scores – one which is ‘what the manager thinks’, and another for ‘what the company thinks’. The two scores would deliberately be on different scales (e.g. the first on a 4-point scale and the second on a 6-point scale) in order to avoid one devaluing the other. We’d be making explicit something which is usually hidden – that a person’s final review score arises from their manager making their case to the other people who control the finances.
We think it might be a great idea, but can’t help wondering whether people really want to be made so aware of the truth?
Mark.
Taking security seriously April 23, 2007
Posted by talentinnovations in 360 degree feedback, Security.add a comment
One of our current clients is one of the big UK high street banks, through our partnership with Cranfield School of Management. It was a very interesting experience to work with them on implementing their 360 feedback project, mainly thanks to the insight it gave into how culturally different big banks are from many other organisations.
One of the key differences is their attitude to security – they take it very seriously indeed. This is entirely as it should be – you don’t want hackers to be able to get your bank details through their web-site, do you? (Just think of TK Maxx!) So they insisted that we pass a number of very stringent security checks (relating to business continuity, backups, login credentials, etc.), as well as pass what’s called a penetration test, or ‘pen test’. This involves a professional hacker spending a week trying to break into your system! Thankfully we passed the test (hooray!), but it’s amazing how many tricks they try. (If you haven’t heard of hacking techniques like cross-site scripting, session spoofing, brute-force attacks or SQL insertion, these guys sure had!) This was quite an inconvenience, not least when they sent 5,000 emails to our marketing department from the ‘contact us’ form on our web-site!
In the process we learned a lot about how paranoid the IT people in big banks have to be to stay a step ahead of the hackers…
Mark.
When to give people their report? December 2, 2006
Posted by talentinnovations in 360 degree feedback, 360 feedback, Coaching, Development, Facilitation, Report.1 comment so far
Earlier this week I had the unusual experience of talking in depth to a consultant from a competitor. Strangely this Consultant used to work for me at SHL so it was very odd and at the same time exciting to be working with her again.
One of the issues that came up was about when you should give someone their 360 report. Should it be before they get feedback or should they get first sight of it from their facilitator/coach? She said that “best practice” was the latter approach, where the facilitator gives them the report on the spot. I would like to challenge this view. I used to take that line and became very used to seeing the range of strong reactions people experience – some have been angry, stunned, shocked, upset, and some have simply denied the relevance of the data. A few have even queried the truth of the “ratings”.
Over the last 3 years, I have had the pleasure of working with an organization truly committed to leadership development. Together we designed the process to get the maximum development and behavioural change out of the 360s and the programme. As a result I have coached in excess of 75 people in the following manner:
- We have emailed the 360 report to the participant
- 1 or 2 days later we have the feedback/coaching meeting.
I have achieved amazing results in the 2 hour sessions with these people. Some had studied their data and got stuck in a few areas but we soon got clear on the real impact of the data. Most were genuinely curious and eager to be supported through the process of working it all out, willing to look at the underlying themes and patterns they usually hadn’t spotted on their own. They had had time to start the emotional reactions and were ready to truly explore the meaning and implications of them. They were ready to take on that the data was a result of the way they had behaved with their colleagues, and ready to see that they could generate different conversations and have a different impact should they choose to do so. In the 2 hours we nearly always got clear on the top 2 to 4 priority areas for focus. They finish committed to moving these areas forward and they had some ideas for next steps.
The people I coached found it helpful and inspiring. How do I know? Well, each participant was always invited to complete an anonymous review after the session so I got feedback (only once did I regret that idea of mine!). It worked really well. Some participants did not get their report in time to have a look at it (due to late reviews usually). Every time that happened I observed that we achieved less in our session.
Overall, giving out 360 reports before a feedback session has lots of benefits:
- it gives people time to reflect,
- it allows different learning styles to adjust and deal with it in a way that suits them,
- it respects their privacy,
- it allows them space to be upset in their own way
- it actually treats them like an adult, trusting them to deal with it maturely.
In essence, it hands over responsibility to where it belongs: the participant.
It would be great to hear others’ views on this…
Elva
Visual displays November 29, 2006
Posted by talentinnovations in 360 feedback, Report, Statistics, visual display.add a comment
I always think that part of what makes for an effective 360 degree feedback report is the way the information is displayed. And that’s not straightforward either, because the information can be quite complex – lots of different items in different competencies, shown for all the different people with their different relationships.
Personally, I think the key is to show all the information but still to keep it simple – a difficult balance to strike. A recent client wanted to be able to have a chart that showed the averages for each competency but with self contrasted with everyone else. Here, as always, you want the visual display to do the job of turning the data into insights that can guide action. An obvious way would have been to use a scatterplot. Here’s a quick sample I knocked up in Excel (it’s from real data on a 360 I did on myself).
However, I think scatterplots can be quite intimidating for people – I find that less analytically-minded people find them confusing. The insights don’t jump out at you. It’s also difficult to fit all the words on. What I came up with was just a simple bar chart but with the self scores ‘overlaid’ as diamonds. Elva suggested putting an ‘S’ in the diamonds as a subtle cue to make it easy to remember that it means ’self’.
I think it works really well. The difference in opinion on the ‘Resilience and Drive’ competency jumps out at you doesn’t it? And, more importantly, the client really liked it too. Now I just have to understand how it is that I’m so deluded about how resilient and driven I am!
Mark
