A lot has been said and even more written about how organisations can measure the effectiveness of coaching.
It has become even more important for the organisations spending money on coaching interventions to know and assess where their investment goes. What this implies is that coaches are now expected to balance the person-centred approaches to measuring coaching effectiveness with those that can also show value of these coaching interventions to individuals in organisations. It is generally accepted that at the primary level, the effectiveness of coaching is in the “lived experience” of the client. It however still does not satisfy the need of the Head of HR in business, who needs to justify the investment in coaching to the CEO. The Kirkpatrick Model While there has been a debate in general about what coaching is and is not, there seems to be a growing consensus that it is a process that enables learning & development to occur (Parsloe, 1999; Downey, 1999; Grant, 2000; Clutterbuck 2003) and not some mystical process with immeasurable outcomes. I would like to propose perhaps another way of looking at measurement of coaching outcomes for business. This paradigm will perhaps provide a different approach to the concept of the “value” of coaching. If it is accepted that coaching is a learning intervention, it perhaps makes sense that coaching effectiveness can then also be subjected to the same methodologies used for assessing the effectiveness of learning/training. In 1975 Donald Kirkpatrick introduced a four-level model of evaluation that has become a classic in the industry. Figure 1: The Kirkpatrick ModelThe model itself is self-explanatory, which is probably the reason for the wide acceptance thereof. Level 1 measures the learner’s reactions to the training (sometimes referred to as happiness ratings). Level 2 measures whether or not there was a change in the skills, knowledge or attributes of the learners. Level 3 measures the change in behaviour that may have occurred as a result of the training while Level 4 measures the value that the training programme have resulted in for the organisation. If one replaces a training programme with a coaching intervention, the model would look something like figure 2. Figure 2 The Kirkpatrick Model adapted to coachingLooking at the model now, one gets the sense of the kinds of questions that need to be answered by the organisation at every level in order to assess the success of coaching outcomes. The ROI and the Kirkpatrick Model The Manchester study (Manchester Review, 2001) already started suggesting the use of this model in measuring the effectiveness of executive coaching. In the study they also give good examples of the kind of questions that were asked at each level (even though this was more at research level). In 1997, Jack Phillips, the chairperson of the ROI Institute, introduced the ROI (Return on Investment) measure as a fifth level to the Kirkpatrick model. Unfortunately, it is this level which received more attention in the Manchester study. While the exposition and treatment of this measure was useful, it served to divert the attention from what I believe are other important elements in the measurement schema. I suggest that the original four levels in the schema are sufficient for the purposes of providing comfort to modern businesses regarding the impact of coaching. Since business results are measured in more than just the ROI, I posit that the ROI measurement, if required, can be measured as part of level 4 measurements (and not as a separate level of measurement in the Kirkpatrick Model as suggested by Phillips). I argue that a ROI measurement is a global business performance measurement, which cannot be attributed to just one element in the organisation - the improvement in business performance may not necessarily be as a result of a single intervention, but may be coming from perhaps improved sales processes or better products. To therefore focus on ROI may not necessarily be the answer after all. In fact, if the organisation finds it really important to measure the global effect of coaching specifically, it makes a lot more sense to measure it within the “learning & growth” perspective of the organisation’s balanced scorecard. In this way the organisation is able to attribute the investment in learning closer to “source”. The calculation can look like the following simplified equationCost of coaching initiatives + cost of other learning interventions = Total cost of learning.This cost can then be included as part of the Net profit margin, which is the one part of the ROI calculation. The value of dialogue over numbers I prefer to open a dialogue with my clients that lead them to an understanding that’s deeper than the numerical dynamics of the intervention. I find that a lot of leaders in organisations have accepted that the impact of learning cannot be assessed only in a reductionist manner. They are generally aware of the systemic interactions of elements in their businesses.
Figure 3 Level four evaluation attempts to assess training in terms of business results. In this case, sales transactions improved steadily after training for sales staff occurred in April 1997. From Winfrey, E.C. (1999). Kirkpatrick's Four Levels of Evaluation In B. Hoffman (Ed.), Encyclopaedia of Educational Technology. http://coe.sdsu.edu/eet/Articles/k4levels/start.htm I have found that taking a balanced scorecard approach; especially what Kaplan & Norton (1996, 255) refer to as Echo-Engineering (figure 4), helps to highlight the systemic interactions of different perspectives. Once shared understanding has been achieved, the scene has been set to open dialogue on more detailed measures using the Kirkpatrick model.
Figure 4 Echo Engineering - Linking measures from the four perspectives To paraphrase Flaherty (1999), Coaching is not a matter of accounting - adding up positives and negatives about the programme… applying a formula that extends the calculation into the future, and then allowing the dynamics of the formula to determine whether coaching was successful or not. I believe that the success of any coaching intervention starts with clarifying the purpose for the coaching intervention, followed by an honest dialogue around expectations and what they would look like to both the organisation and the coachee. While facilitating this conversation, it is important to keep in mind that coaching is not about balancing the numbers; it is about developing individual capabilities for exponential collective impact. Like trust, respect and loyalty, it has no formula. Acknowledgements