| Kineo Interview with Charles Jennings |
|
Kineo Interview with Charles Jennings Jan 20th 2006 Introduction Performance analysis at Reuters Richness and reach Trainers, managers and the ‘conspiracy of convenience’ Informal learning ROI: Books 24/7 at Reuters Don’t fear informal learning Measurement: ROI isn’t everything Experimentation case study: online communities at Reuters Googleisation of learning Charles ’E-learning Bugbears How to get your learning function to the boardroom table E-learning trends for 2006 Introduction Stephen Walsh: Hi this is Stephen Walsh from Kineo, and welcome to our February podcast. This month I’m in conversation with Charles Jennings from Reuters. Charles is the head of global learning at Reuters and Reuters, he told me, is the largest publisher of content to the internet. So, as you can imagine, Charles has got a thing or two to say about how you design effective content to enable learning and performance. I spoke to him just before he addressed the Learning Technologies conference in London, and the topic of his talk was ‘From training for skills to learning for performance’. So I asked him to start by just unpacking that phrase for us and explaining what that means. Charles Jennings: There’s really two elements in that phrase, first of all the ‘from training to learning’ and secondly ‘from skills to performance’ and if we take the ‘training to learning’ first - I think that implies a change of view from something that gets done to you by a trainer to something that you, as a learner, do. So there’s a key change there. The second element is the change from skills to performance, and I think that implies a change to focus on outcomes rather than input, to a certain extent. And those outcomes vary individually everyone has their individual performance outcomes and skills are of absolutely no use if they’re not expressed in improved performance. And one of the other points I’ll be making during that presentation is that a high level of skills doesn’t automatically imply a high performance and we used the analogy with musicians, that you can have the most gifted musician in the world, but unless you can stick them on the stage and turn in a great performance they just don‘t have an impact. On the other hand some musicians can be the greatest performers - take Elton John for example - but they don’t necessarily have the greatest skill. I’m sure no-one would argue that Elton John is one of the top pianists in the world, but he has the ability to have real impact. So I think that the implications for learning professionals and learners is that it’s bound up in those two binaries really. Which is around, first of all, don’t think about training as something that gets done to you. Think about creating an environment where people can learn. Secondly, don’t think just about skills as being what we’re all about, think about the final outcome which is individual and business performance. Stephen: I guess that means a fundamental shift in the way that training and learning objects are couched. A skill is something that you may accumulate over a lifetime, but achieving performance to get a specific output or achieve a certain tasks seems to be much more immediate and much more point of need and, potentially, much more short term in its focus, would you agree with that? Charles: Well, I think that’s an interesting point. I think it is that performance is around being able to do what you need to do now, but actually I think that to achieve high performance, actually often needs - you need experience. So one of the other analogies I’ll be using about learning technology is what is the difference…it’s a conundrum that was posed by Jerome Bruner - an education psychologist, a constructivist, who posed the issue, the question, ‘what’s the difference between learning physics and being a physicist?’. And in fact there is a significant difference; we can train people, people can learn physics, but when they graduate from university they’re not necessarily physicists because wrapped around being a physicist is an understanding of the domain is, it’s about being able to make value judgements and decisions, it’s a lot about experience. So it’s an interesting conundrum in terms of performance can both be immediate but it can also be embedding in experience. And in fact I was at the chief learning officers’ symposium in Barcelona, just before Christmas, and a theme that came up there was the importance and the significance of experience. And I think that that’s one of the things that we’ll start to look more at and get a better understanding of the whole sort of learning domain. Performance analysis at Reuters Stephen: I guess the challenge for people who are responsible for designing …let’s call them learning interventions, for want of a better universal term…to enable that performance, you need to be able to identify what performance is required and that’s different from skills analysis. So, my next question, I guess, is how do you get to conduct a good performance analysis that make sure your training outputs are aligned to the desired performance outputs? Charles: Well, I think on research we’re fairly good. At Reuters we’ve recently being using an approach that’s called performance consulting. It’s a very simple, seven step process, devised by Nigel Harrison, who’s a British business psychologist and author of books on performance improvement and instructional design, and Nigel’s been working with us on this sort of model. We’ve been solving some pretty complex business problems that relate to learning and I think the key to the process is not to look at any problem that a manager brings to you as a training problem, first of all, you avoid ‘solutioneering’. I think one of the problems we fall into as learning development professionals is that we tend to look at problems as being training problems - it’s like, you know, if you’ve got a hammer in your hand everything looks like a nail - and the performance consulting model takes you right back to make sure that you define the problem very carefully and you understand first of all define who’s involved and affected by this particular business performance problem. Secondly, what’s happening now? Thirdly, what does the manager want to happen; what would success look like? And then I think the silver bullet of this pretty simple process is working with whoever’s involved - usually the manager of the business unit, to actually calculate the cost of that gap. And actually that’s usually a very interesting exercise, that very often managers will push back and say they can’t do it, but in fact I’ve never been in a situation where we haven’t been able to identify what the cost of poor performance is. And the outcome is usually a figure which is far in excess of what people think initially. So for example if you’re working with a sales manager and he or she has a concern that their sales force aren’t performing, you go through this process as to what’s the gap. They usually say, we don’t know. You can look simply and say what are your targets next year, and their targets might be £80 million revenue, and so what are they achieving this year - £50 million revenue and you say, OK well the cost of this gap is £30 million. That usually focuses the mind pretty well…when a manager comes to you and says they’ve got a training problem then realise they’ve got a £30 million business problem. And then from there you can start to identify the causes and possible solutions. And only then do you start to build some sort of action planning to put some solutions in place. And quite often you find the solutions don’t actually involve training at all, but actually stepping through that process, and it’s not a complex process, it doesn’t take long and in fact we do it usually by workshop approach, where you put the managers together and you can workshop it out quite quickly, but it will bring to the surface those problems that are really, can really be addressed by some sort of learning intervention, and those problems that may be addressed simply by the fact that they’re not paying people enough and they’re not getting quality people and therefore they’re not achieving their objectives, or the processes are wrong or there’s various other things, blockages in the way. So performance consulting is a process which can be used in any business problems and Nigel Harrison‘s written books about solving complex business problems that are nothing necessarily to do with training at all. Richness and reach Stephen: I asked Charles what role he thought technology had to play in helping making learning more performance focused. Charles: I always go back to Evans and Wurster’s book call Blown to Bits which was produced in the 90s, and it was a management book about the impact of technology on society - particularly on communication technologies. And Evans and Wurster talked about the richness and reach conundrum: in the past you‘ve only been able to create either a very rich learning environment, or one which reaches a large number of people. And I typify it, and you can in terms of the Oxbridge model versus the Open University model - I’m sure the Open University would argue that their learning programmes are very rich as well as having reach, but I think the reason that they’re rich is actually that they employ technology. And the Oxbridge model is around providing a very rich learning environment. In Oxford and Cambridge you have a tutor per two students and so you can provide that rich model but it doesn’t scale. You couldn‘t scale it across all the universities in the UK, for example, you would end up having a huge population of academics, that just wouldn‘t work. Whereas the Open University has been very, very good at providing learning which has great reach, so you can bundle up the learning and put it in a box and tie a bow on it and send it to the Antarctica and someone sitting in Antarctica can open it up and learn. Now, I think that technology actually allows us to bridge those two and therefore provide learning which has richness, but also has reach. So I think in terms of the back end of this performance consulting process, in terms of when we look at what solutions we can put in place, technology has really a key place to play in that. Trainers, managers and the ‘conspiracy of convenience’ Stephen: Coming back to the performance consulting piece, because I think that’s going to be very interesting for our listeners, because it strikes me that it’s almost learning a new language on both sides of the table, a new way to interact; it’s people within the learning development community not feeling like they are fulfilling requests for ‘where‘s this course, I want a two-day workshop on this, or I want a piece of e-learning on that‘, but having a much broader, more consultative view of how to solve performance problems and that the same time people like the sales manager that you describe not coming to the table expecting a piece of training to be delivered to them to address a specific need, but being willing to engage in a much wider dialogue about what the solution might look like and recognise that part of it might have nothing to do with training at all. How have you found that people have developed to have those different types of conversations within Reuters? Was it a struggle to change the level at which the dialogue was happening at first? Or did it take a couple of successes for the new model to take hold, or what was the history there? Charles: Yes, I wouldn’t suggest that we’ve cracked that particular difficult nut. I think that’s a very difficult issue that many of us have that takes quite a while in terms of…. there’s two elements to it. There’s the issue in terms of managers first of all understanding that a key part of their role as a manager is to develop their people and that any problem around the developing of their people is their problem and it’s not the training manager’s problem …so there’s that element of it. The second element is for the learning and development or training manager to understand that they’re not simply running a fulfilling service, as you said. And I typify it in what I call the conspiracy of convenience. And I think traditional training department were reactive, did run fulfilment services and they engaged in this conspiracy of convenience with managers. And if I can just explain that very briefly - the conspiracy is really that the manager comes to the training manager and says, ‘I’ve got a problem, my people need training’. The training manager says, ‘Fine, we’ll develop a training programme’. So, the training manager develops a training programme, delivers that training programme, no-one measures it. The business manager is happy because he or she feels that they’ve fulfilled the requirement that they had - which was the training problem. The training manager feels happy because he or she has done what their job is all about - i.e. have delivered training. And because no-one measures it, nothing necessarily happens, but everyone’s happy. So it really is a conspiracy and I think that what we need to do, looking forward, is to break that conspiracy, which is no mean task, but does require changes on both sides Stephen: And of course measurement, as you pointed out, shines a light on both sides of that conspiracy. Charles: Absolutely, as soon as you carry out effective measurement - and that’s another issue to discuss - but once you’ve carried out your measurement it does make the process transparent and really gets to the bottom of what value has been created. Informal learning ROI: Books 24/7 at Reuters Stephen: I asked Charles how Reuters had moved along the informal learning path and what kind of examples of performance support objects he could provide from Reuters’ experience. Charles: For a start I think that the generally accepted figure now is that 80% of organisation learning is informal and we need to find ways of supporting informal learning as well. And in Reuters one of the thing’s we’ve done in this move away from big e-learning programmes many of which were simply replicating classroom events on a screen and quite often not doing the same things that can be done in a workshop or a classroom as well. We’ve used some technologies particularly around performance support where we’re providing that performance support at the desktop. And we’ve rolled out - we’ve had great success in fact - with a tool called Books 24/7. We’ve rolled out the IT Pro library of Books 24/7 to around about 4000 of our technical staff and that’s one programme where we have been able to do some very, very detailed ROI calculations in terms of productivity savings and that performance support tool has shown huge productivity savings, some were, in the first year, some were over 2000% return on investment. And that’s simply productivity around asking the people who were using it whether it’s saving them time. And when we looked at the small increments of time they were saving which was somewhere around slightly under 5 hours per month in time saving, when you scale that up across the whole user base it comes to a huge amount of time and money saved. We also looked at how much it was replacing a need for people to go on formal training programmes and courses. It was doing so, but to a lesser extent, still that turned in another 1400% saving per annum. We didn’t even look at the savings that we we’re making in terms of the fact we’re no longer buying technical books because they have a library of four or five thousand technical books at the touch of a button. We didn’t even go there in terms of looking at the savings there because simply the productivity savings were enough to justify and in fact that’s been ongoing and has worked very well. Don’t fear informal learning Stephen: I’ve spoken to some L & D professionals who would admit they were quite scared about this 80% because they’ve lived their lives fulfilling on the 20% and to suddenly think that four fifths of the learning within the organisation is outside their control and not something that they can actively procure, monitor and measure can cause a couple of people to shudder, but I’m guessing you would see that as an opportunity as much as it is for a challenge. Charles: Oh, absolutely, I think it’s an opportunity to align with real organisational needs. Informal learning has always been there and it’s always been dominant, but many people are just seeing it for the first time with eyes wide open and it’s quite frightening to them. And we just need to look at ways in which we can support it really. It’s not something we can measure so easily. In fact I was involved in discussions internally in Reuters yesterday about whether it’s worthwhile, and if it was worthwhile, how do go about measuring informal learning? And the answer is, you can take some very base measures in terms of what people are looking at and so on, but it doesn’t give you a real measure. In fact I was arguing that with something like Books 24/7 we can how many hits which are the books that are used most frequently etcetera - It probably gives us the same value as a Kirkpatrick Level 1; it doesn’t tell us anything about learning, it simply tells us a little bit about activity and Kirkpatrick 1 tells us that someone was in a room at a time and gives us a bit more about their own view. If there‘s a hit it tells us someone looked at this or at least opened it, we can‘t tell even whether they read it, so measuring it is extremely difficult, I think it’s probably fruitless to spend a lot of time thinking about it. What we should be doing is making the tools available and then measuring the performance at the other end, and determining whether the availability of the information and tools and the particular format actually helps improve their performance. So if performance is going down, we need to go round the loop again and look at what we’re doing. Measurement: ROI isn’t everything Stephen: Learning for performance means a different type of measurement. It means you’re measuring business outputs you’re measuring individual performance, your abrogating that to business performance. What are your experiences in measuring learning for performance? Do the models of Kirkpatrick still bear relevance to this, or do we need new models when it comes to measuring performance relevant skills. I guess Kirkpatrick’s levels 4 is really about performance to some degree. Do you employ that method, do you have your own ways of tracking it? Charles: Well interestingly enough, I think that Donald Kirkpatrick’s model is still applicable, largely. I have some issues with the bottom level and the top level of the Kirkpatrick model. First of all I think that Kirkpatrick Level 1, which is almost universally used, the measure of reaction which is really the so-called happy sheets or the reaction of people who have been through some sort of formal programme. I think that actually has a very limited use. At the bottom end it really tells you whether the person who attended some sort of formal programme had a good lunch. I sort of say that in an offhand way, but it doesn’t tell us a lot more. But then again, when done well it can give you an idea about how things went, but I’d be so bold as to say that often the people going through learning programmes are not the best people to evaluate it. I think that we’ve spent the last few years focusing on how we can measure return on investment from learning and people like Dean Spitzer who’s developed his model of what was called training results measurement, I think it’s called something else now, which is actually some very, very good work and Dean has put a huge amount of thought into it. But it’s very, very complex and actually to carry out those ROI calculations to actually define the causal chains and to identify exactly where the problem is and how it’s been solved is very complex and in just normal day to day work we couldn’t possibly do it on all or many of our programmes. I also think that ROI is not necessarily the whole endgame. ASTD and IBM carried out a study and published it last October and they interviewed a large number of CXOs, so Chief Executive Officers, Chief Operating Officers, Chief Technology Officers, Chief Financial Officers and so on and so forth to understand exactly how they determined the value of learning in their organisation - the value of the learning function within their organisation. And they came up with some rather surprising conclusions and one is that, these CXOs are interested to see the ROI data, but actually they used other matrix as well - and in fact in some cases predominantly. So they have to feel that it’s the right thing. They have to have - it’s almost by perception - they have to perceive that the learning function is delivering value for the business to be happy, whereas I think Chief Learning Officers or Heads of Learning or whatever they’re called, spend a lot of their time trying to justify their existence by carrying out ROI calculations. And I think that’s very interesting and I think going forward into 2006, 2007 I think there’ll be a lot of focus on that in terms of how we get that more balanced and can perceive a value not simply through ROI. Experimentation case study: online communities at Reuters Stephen: If someone wants to face up to the fact that there is an informal learning culture in their organisation and they want to help facilitate and encourage it, there’s a certain period of experimentation required. It’s very hard to look at an organisation and say I think a Wiki on customer service would work here or maybe we should encourage people to set up blogs or we should use RSS or we should encourage podcasting or whatever it is, but because the culture will find what works for it. Charles: Yes I would agree with that, absolutely, I think we need to look at what’s going on now and look at what works for different teams, different groups in the organisation, provide the facilities for them to experiment, to pilot different approaches and get some feedback to see what works. And inevitably some will take off and be extremely useful and become embedded in the fabric of what people do and others just won’t work. And I think a lot of work has been done around, for example, building communities of practice. Again, you can’t simply go in a create a community of practice. What you have to do is go in and help people understand the basic tenants of communities of practice and what the difference between a community of practice is and a project group and so on. And then, you have to identify first of all champions in these areas but then provide them with the tools and support for them to build their communities of practice. Again, it‘s one of those things I don‘t think you can just impose externally. And we’re doing that in Reuters at the moment, we’ve got a workstream around collaboration, we’ve got a big technology transfer programme. Reuters produces several thousand software products every year and maintains them and we have a large number of teams of technologists spread around the world in various centres, and we’re looking at ways in which we can encourage them to work in a much more collaborative way and share their knowledge and build their knowledge and professional communities is one tool that we’re looking at. But there’s no way that we can simply go in and impose those, we have to nurture them and again I think that the term learning culture is a good one. You actually have to culture them; it‘s like growing a plant - you can sow the seeds and you can water them, but in fact the growing of the culture is down to the people who are part of those groups and they need to be actively engaged in it and they need to own it and grow it. Googleisation of learning Stephen: Charles has used the term the Googlisation of everything and that obviously has implications for leaning. I asked him what he meant by it. Charles: The Googlisation of everything is important for two reasons. One is that Google has as its mission statement to be the prime source of knowledge globally, which is a pretty big aspiration. And it’s moving into lots of other areas. People know Google as a search engine, but in fact it’s a collaboration tool, it’s in fact they’ve got Goggle transit which will allow you, in some states, some cities in the States now to be able to find out when a bus is passing your front door and so on and so forth. So that’s moving on a lot. But I think tied to that from a learning point of view is that the rate that knowledge is increasing, information is increasing - it’s about 30% per year and the idea of being able to hold in your head the knowledge that you need to do your job is getting eroded. In fact, Robert Keely at Carnegie Melon University has been doing a lot of study since the 80s asking exactly that question: how much knowledge you need to hold in your head to do your job. And I can’t remember the figures exactly, but in ‘87 it was around about 75% - people felt that they needed to hold about 75% of their knowledge, to do their job, in their head. By the mid 90s that had come down to about 20%. And I haven’t seen the recent figures but I’d estimate at around 8 to 10%. So it’s not about storing knowledge in your head anymore, it about finding it quickly when you need it. And coupled with this explosion of information, it‘s having quick tools that can sift and enable you to identify exactly what you need when you need it. Stephen: Right, so it seems like a key skill for knowledge workers in this generation and the next is going to be an ability to search, an ability to focus on data as relevant more than it is ability to accrue an innate skill. Or is that going too far, do you think? Charles: No, I think that’s right and I think when we look back into learning and development and training functions within organisations it shifts the balance. It means that we no longer should be focusing on transferring knowledge, we should be looking at helping people to develop the skills to be able to find the information and process that into knowledge when and where they need it. That’s always been, to a certain extent, in academic institutions, been a driver, in terms of you develop the skills for people to develop their own skills of finding and developing knowledge. And I think that within commercial organisations it’s just starting to it. Stephen: And is Books 24/7 an example of that? And are there others on the horizon that would fit into that model for you? Charles: Yes, Books 24/7 is. It’s a resource the same way as I think most of us use Google on a daily basis. And in fact I had a conversation with somebody recently who asked me and said does the Encyclopedia Britannica still exist, which I thought was an interesting question. The point is because, you know, why would you use Encyclopedia Britannica when you can hit Google and find whatever you want? Charles’ E-learning Bugbears Stephen: I asked Charles what his bugbears were on a daily basis in learning and development. He didn’t have to think very hard to answer. Charles: Managers demanding training when they haven’t analysed the problem - would be probably the main one. Managers seeing poor business performance as training’s problem - not their own. And lastly, trainers reverting to the training fulfilment model rather than looking at the performance consulting model and going down that road. I think the problems lie on both sides and although there I’ve identified two manager issues and one trainer learning issue I think that actually, there’s a whole issue which we need to address in that area. How to get your learning function to the boardroom table Stephen: I think a lot of people would look at what you‘ve done at Reuters and the strategic view that is taken of learning within Reuters right now - seen as a business tool, and seen as an enabler and away from that fulfilment and would probably look at that with some degree of envy. And would say I would like to do that in my organisation, how do I get there? For people who are ambitious to go down that path is there a road map, any specific actions or advice which you could recommend? Charles: Well I think, I don’t think there’s any one particular road or road map, I think that are some basic principles and we’ve talked about them really. One is to ensure that your learning and development or training department moves away from this fulfilment model and develops the consultative skills to be able to be a business partner. I think that when you look at some Human Resources departments, that have gone down this road, some have been very successful and others haven’t. Where they’ve moved from a transactional model, of providing a transactional service, to a business partnership or consultative model. That’s the first thing and that requires both the people in the training or learning and development department to understand the skills that they require and understand the models that would work for them but equally, and more importantly, it means that the managers within the organisation need to understand that change as well, and to do that one needs to get very senior level management support. Now, in Reuters I found that relatively easy. Senior managers have embraced the idea of learning and development acting as a strategic tool. In other organisations it may not be so easy. I think some of the main challenges that any organisation or any learning professional will face is to first of all overcome the whole issue around managers understanding that the performance of their team is actually their problem, and that they themselves need the skills and also the support from the learning and training professionals to overcome those problems. So that would be the first thing that I think would be extremely important to overcome that particular issue. The second thing is to accept that analysis of performance problems is necessary, but it’s also necessary to be able to do that very efficiently and effectively and quickly. Again I’ve seen many cases where training needs analysis - and I can’t stand the term training needs analysis because it assumes the problem is a training problem from the outset - where training needs analysis has taken sixth months. Clearly performance analysis needs to be done well, but it’s something you should be able to do within a week and identify exactly what the causes of the problem are and then start to look at the solutions to deal with it. And thirdly, and I think probably most importantly, is the issue of professionalisation of the entire learning function. That’s an issue around requiring people to do things that in the past they may not have done and have a set of skills themselves that they haven’t had in the part, or haven’t had to use. And in Reuters we have some very professional learning people and we’re also supporting a number of our people to build their professional skills so we put them through, or support them to go through, CIPD programmes in the UK and ASTD programmes in the US and so on, so really that’s around the whole professionalisation of the learning function. E-learning trends for 2006 Stephen: As you look out beyond 2006, you’ve probably alluded to them, but maybe just to summarise, what trends do you see taking hold? What’s on the horizon for learning? Charles: Predicting trends is always a dangerous area, but I think I can think of three or four trends. One is I think that there will be fewer big e-learning programmes. Many of those which are still simply replacements for classroom workshops - were just classrooms on the screen, I think that we’ll move away from that and I’m already seeing that trend occur, so that we’re looking at rather than having hours and hours and hours of e-learning offered up, e-learning is being blended in … that terrible word blended learning…but it is actually being blended in and used where it’s appropriate as simply a performance support tool or as a bit of pre-learning to take the knowledge transfer out of the workshop and use the workshop for the areas of learning and development where it’s really best suited. I think secondly the trend is more focused on outputs from learning, as I discussed earlier. And those outputs are not simply ROI in terms of hard numbers, but outputs in terms of business improvement and all the business that goes around how we look and measure and can evaluate business improvement. Thirdly, I think certainly there’ll be a closer examination of informal learning and how we can use technology to support informal learning and certainly I can see a groundswell coming through around looking at informal learning and trying to get our heads around exactly what we can do to facilitate support and nurture that. Fourthly, mobile learning -- no doubt which to a certain extent has been technology-driven but also I think workforces are becoming more mobile and people working globally, wherever they happen to be - providing learning to support that. And lastly, I think the use of gaming, which is an area of interest, but is still in the early stages -- the uses of gaming scenarios and what’s called serious gaming. And again, coming back to what I was saying earlier on, around the digital natives. I was reading a US-based article just last week around digital natives and it appears that children - not just boys, but girls - are spending somewhere upwards of around 5 hours a week playing games on digital devices. And as they come though into the workforce as learning professionals we’re going to need to look at ways in which we can basically help them learn for their organisational needs, using the tools and the approaches which they’ve grown up with. We’re at the very early stages there, but serious gaming scenarios are going to become more important, certainly something we want to focus on in 2006/2007. Stephen: It would be interesting to see how games and the whole notion of informal learning complement or collide with each other. Charles: Absolutely, and mobile as well - I think that those three elements: informal learning, mobile learning and serious games have a lot of power. And in fact again at Reuters, we‘re looking at, Reuters being an information company, in fact we’re the largest publisher of content to the internet, but we publish news feeds, we publish short snappy video feeds continually. And we’re looking at how we might use some of the approaches and technologies that we have in our own company to deliver mobile learning. Interesting times… |
Back to top