Wednesday, 13 November 2013

Why picking the right KPI is so important

In a recent post on SSIR Christina Triantaphyllis and Matthew Forti talk about the problem with focussing on the overhead vs. impact ratio when assessing charitable work. For me, this is just another step along the chain (donor -> charity -> local NGO) where the use of incorrect KPIs leads to exactly the opposite of the desired behaviour.

During my time working in rural India I was struck by the extent to which NGOs tailored their activities not to those actions that had the highest impact, but that would tick the most funding boxes. During the course of the year I came to understand the problem - funding is so difficult to come by that NGOs have to fight tooth and nail in order to secure finances. They cannot be picky about the sources of that income, nor can they afford to let an opportunity pass, therefore if a donor organisation says a particular report / action is required, that is what the NGO provides.

In the extreme, this results in very peculiar behaviour. NGOs recording the number of attendees of workshops, but not the results of any learning. Photographic evidence being required of meetings, rather than documented minutes / actions.

The source of this insanity? The laziness of the funding organisations. Measuring true impact takes effort. It costs money. It may show that your initial approach was wrong. That money has been badly spent. That the hypothesis was wrong.

If an analogy is needed, here is my best attempt: You start in London and need to travel to Manchester. You research before hand and learn that it is 200 miles. You can travel at 50mph, so you know you need to travel for 4 hrs. You set off in what you think is the right direction and, without checking a map, GPS or roadsigns, travel for 4 hrs, with the only monitoring being a regular check that you are, in fact, travelling at 50 mph. You assume that as long as you continue taking this action you will arrive at your destination.

The flaw in this argument is obvious. You need continuous assessment (measurement) that you are travelling in the right direction. Taking action does not automatically result in the right result. While there is some truth in the maxim "some action is better than none", it is not true that any action is always the best one.

Measuring charities on their overhead vs programmes spend is the wrong thing to measure. In the same way that measuring the number of workshops run is not the right KPI for field work. However, in a competitive market, the only action open to organisations looking for funding is to fall in line and do the best they can.

The only way this changes is with the first step in the chain...

Monday, 11 November 2013

International Development - a change of direction

It's been a while since I've been on here. It happens a lot. I write for a bit and then stop. Write a bit then stop. This time though, I've got a massive incentive to keep it up....I've started a Masters degree in International Development!

I'm realising very quickly that I need to keep writing my thoughts down so that I articulate myself properly and ensure I properly understand a topic. While I've coped with my first assignment, I recognised too late that I didn't know my subject area in enough depth to be able to argue it reasonably. That's not good enough, so from now on I'm going to try and post once or twice a week, whether it's on topics to do with my latest reading for my masters, or a topic I've found on a blog. Hopefully this will really get me involved in my subject, draw me into the Blogosphere again and (ultimately) get me the grades I need!

So without further ado, let's get started :-)

Tuesday, 9 April 2013

Why SIB's won't work

I was reading Kyle McKay's article on Goldman Sachs new SIB programme and while I agree with the majority of his points, I think that the fundamental point is missed here...

Myth 3 for McKay is that "The government pays only for success" - McKay is concerned that it will be difficult to decide whether a programme is a success due to the contractual difficulties in defining success and that the government will therefore end up paying for projects that have not "succeeded".

This misses the point that the project developers will be incentivised to produce projects that are at least perceived to have "succeeded". This in itself would not appear to be a problem until you consider that a project that has succeeded in its fundamental aims, versus one that succeeds as per a written contract are two very different things.

As soon as a project is designed specifically with the aim of improving a measurable outcome, rather than with the fundamental outcome in mind, it runs the risk of becoming a self-fulfilling prophecy. Focus shifts away from the aims of the project and onto improving the KPIs. The key word there is "Indicator". A positive shift in a KPI does not reveal a project that is working, in the same way that a negative (or zero) shift would tarnish a project as failing.

SIBs could result in the government paying for programmes that are not effective. That's a true statement. It could also result in far more projects being run that are not effective at all. That is the issue here.

Taking a new direction

In what is becoming a fairly regular(?) occurrence for me, I'm going to make a bit of a change to the focus of this blog. It's been almost two years since I last posted on here, but a lot has changed. I'm back from India and planning the next phase in my life.

The work I did with VSO (see my other blog for more details) had a profound impact on my life and left me wanting to do more than just moon-light in the development sector. Unfortunately, on my return there was a mortgage to pay and it was far too easy to just go back to my old job...

Times have changed, however. I've been back for over 18 months now and while I don't quite have the itchy feet to grab the passport yet, I'm definitely looking at my options. So first step - I've applied to do a Masters in International Development. It's a part-time, distance learning course, which should provide me with a bit of flexibility at least.

Step two - starting this blog. I'm going to try and start commenting on the blogs I'm reading and just giving my general thoughts on what they're saying - join the conversation! We'll see how it goes - it might be that I can't do this as regularly as I'd like, but I'm going to make a conscious effort!

So there you go - new direction in life, new direction for the blog. It's a nice symmetry, right? :-)