Evaluation - Gold or a Chimera?

In the film Top Gun, a rather fanciful account of the US Navy Fighter Weapons School, the Chief Instructor (callsign Viper) tells the hero who comes to him with a crisis of confidence:

 A good pilot is compelled to always evaluate what has happened so he can apply what he has learned. 

The resonance with organisational development is clear.

Although there are many models of evaluation the dominant one for learning and development interventions and equally applicable to coaching is Kirkpatrick’s.  He uses four levels of evaluation:

1.      reaction of client - what he or she thought and felt about the coaching

2.      learning - the resulting increase in knowledge or capability

3.      behaviour - extent of behaviour and capability improvement and implementation/application

4.      results - the effects on the business or environment resulting from the client’s performance

Sadly in too many instances Kirkpatrick is honoured in the breach rather than the observance.  Having been Head of Training in DFID for 8 years I can attest to this absolutely.  I designed evaluation protocols and models, carried out evaluations, and presented to the Cabinet Office on the politics of evaluation.  Not once were we able to get any further than level three and mostly never beyond level two.  In the main this was due to a number of factors:

• A non-trivial issue of attribution and causality, methodologically impossible to demonstrate without using control groups.
• A reluctance to use time series by the line: “That was six months ago – I am interested in other things now”.
• A concentration on the present and the future rather than the past in an organisation that had many policy changes and initiatives.
• A general scepticism as to whether evaluation could be in any way useful.

As a result we were never able to apply Viper’s Dictum in its entirety.  It was rather like “….evaluate what has happened so he [we]…can forget about it…”!  And the picture in the other organisations where I have been head of training was just as dismal for the same reasons. 

Coaching is no less worthy of evaluation than any other people strategy.  Indeed it can be argued that it is even more important to evaluate coaching because of its high relative cost and extreme individualisation.  Evaluation should be both normative, to provide information on techniques and coach effectiveness and summative to provide both the coach and the client (and his or her organisation) an estimate of the value of the process.

For coaches this means giving serious consideration to the appropriate evaluation mechanisms before coaching starts and even as far as being fairly specific during the tendering process.  To some extent the objectives of the coaching will be pre-agreed or in some intances established during the contracting process between the coach and the client and, as required, the sponsor.

For if we don’t evaluate we won’t know whether we have achieved what we set out to. 

Do 90% of Organisations Really Use Coaching?

The CIPD is featuring research that purports to show that a whopping 90% of companies polled use coaching.  Now without knowing what the methodology was (the final paper has not been released yet) it is hard to evaluate the findings, but the claim does strain credibility.  Are we really to believe that so many companies use either internal or external qualified, even EMCC accredited, coaches?  Or is it a case of managers when asked saying  "oh yes we coach our staff", when in fact they have no proper training at all?

Here at Diomedea Executive Coaching we only use properly qualified coaches who have undergone rigorous training to EMCC standards.  It is not enough to simply say you are a coach.  You need to be properly qualified as well.  Only then can you help your clients to find their wings and fly.

Top Ten Tips for Managers

It can be very daunting when you are a new manager; lots of things to think about, problems to solve and your team to develop and motivate.  So here are some tips to make your life easier:

1. Management is an honour and a privilege.   Behave accordingly. 

2. Be fair at all times. 

3. Look after your staff before looking after yourself. 

4. Be visible and accessible.  Make time for your staff when it’s needed.  Not when it suits you. 

5. Get to know your staff as people – their lives, hopes and fears.  Develop their talent.  Help them fulfil their dreams. 

6. Take your staff into your confidence and share your own hopes, dreams and expectations. 

7. Be true.  Tell the truth even if it hurts you or your staff.  Give proper feedback. 

8. Assume the best about your staff not the worst, but don’t be disappointed if they fail to live up to your expectations. 

9. Always keep your word.  No exceptions 

10. Be loyal at all times.  Defend your staff when they are attacked.  Take the rap for them when needed.

Remember that staff management is about creating and maintaining good interpersonal relationships.  If you doubt this try managing a chair.

Why Does the Client Want an External Coach?

For large organisations, both in the private and public sectors, the question of whether to use internal staff, suitably trained or external consultants for coaching services looms large.  Ultimately the answer will involve procurement issues and will thus be complex and based on a number of often competing priorities.  It is important for a coach to be aware of, and understand these issues when bidding for work.

Generally speaking large organisations instinctively lean towards external providers for those services not seen as core to the mission of the organisation.  My experience over 30 years of dealing with the selection, engagement and management of consultants supports this proposition.  Organisations employ consultants for many reasons, some obvious, others less so.  These reasons can include one or more of the following:

• The organisation lacks the resources or capacity to carry out the assignment, although they may have the skills internally.  They need someone to do the work.
• The organisation does not have the capability to do the work themselves. They need somone who knows how.
• The organisation wishes to skill its own staff by learning from consultants.  They need knowledge transfer.
• The organisation wants a new World View or a change in paradigm within the organisation which they believe only consultants can provide. They want a new way.
• The organisation wishes to justify its decision or course of action: “We asked Bloggs and Doe, the finest in the land, to coach our staff an approach that has been highly successful despite initial scepticism”.  They want legitimacy.
• The organisation wants a scapegoat if something goes wrong: “We asked Bloggs and Doe to coach our staff but unfortunately they weren’t up to it”.  They want absolution.

Thus external consultants are often seen as expensive, more highly qualified, more globally aware, politically advantageous and expendable whereas  organisational staff are seen as cheap, available, commandable, house trained and culturally aware.  External coaches need to understand this and deduce which of the reasons cited above is likely to be driving the assignment and hence what the implications for them might be.