Why evaluate?
When an organization reaches out to us to ask about evaluation, it’s typically for one of 3 reasons. Which of these applies to you?
- You want to find out how effective your initiative, project or service is, or how well it is being implemented, or how you can improve it. If so, you’re in the right place!
- You are thinking of setting up a new service or project, and you have a pretty good idea there’s a need for it – but you need evidence of how much need there is. A needs assessment will help with this.
- You have been asked to get an evaluation done by a funder. It is a requirement. You have to do it. Yikes!
If it’s number 3, you’re not alone. It’s a common one. But although you have to do it, you can make the evaluation helpful to you. It’s an opportunity to learn, celebrate your successes, and improve your organization’s impact on the people you work with. And that’s what we all want, right?
What we'll cover
- Myth busting what evaluation isn’t
- What is evaluation?
- What does evaluation assess?
- What’s the difference between evaluation and research?
- What questions does evaluation ask?
- How can evaluation help your organization?
Before we get on to thinking about how evaluation will help your organization – and ultimately, the people who use your services – let’s take a look at what evaluation is – and what it isn’t!
Myth buster
Working as an evaluator, people can (initially, at least!) be a bit wary of our role. They see us as an auditor, or an inspector. But let’s put our minds at rest on that.
- Evaluation isn’t an audit.
- Evaluation isn’t an inspection.
So, what’s the difference between audit, inspection and evaluation? Very briefly, we can say that:
- An audit assesses compliance with rules, regulations and policies.
- An inspection investigates non-compliance with standards and good practices.
- An evaluation assesses the value of an initiative. It’s about learning, not compliance. Phew! Believe me, it’s learning that’s our game.
What is evaluation?
Now we’ve clarified what evaluation isn’t, let’s turn to what it is. Bear with me, it’s not as clear cut as we might like! There are many definitions of evaluation, and evaluators don’t agree on which is best. In fact, as evaluators, we often struggle to explain to others what we do. So, let’s try and unravel it a bit. First of all, here is a short definition (shorter than most!). I’ve drawn upon the Canadian Evaluation Society’s definition, but shortened it, and adapted the wording:
Evaluation is a planned assessment of the design, implementation or effects of an initiative.
Now let’s take a quick look at each part of this definition, to get a better understanding of what it’s all about.
Initiative
Evaluation assesses an initiative. The “initiative” being evaluated can be a program, project, sub-project, program activity, service, intervention, campaign, or in fact, just about any other initiative you want to assess.
Planned
Evaluation is carried out according to a plan (it is systematic). This makes sure it provides information that is trustworthy (credible), and reliable. If we don’t follow a plan, the findings won’t be as helpful.
Assessment
Evaluation assesses something. Typically, it won’t assess everything about an initiative, but will focus on what’s important in a specific context. Let’s now consider what evaluation assesses.
What does evaluation assess?
In my experience of evaluating all kinds of initiatives, this depends on the client’s requirements, and the stage the initiative is at.
Think about an initiative that has just started, compared to one that has been running a year or more. The focus of the evaluation will be different.
An evaluation can assess:
- The need for something – how much need there is, and who needs it.
- How something is designed, implemented and delivered.
- How appropriate, relevant or accessible something is.
- Progress made towards achieving goals and objectives.
- The effectiveness, outcomes, and impact of something.
- How efficient and cost effective something is.
- Something’s value, quality or worth.
What’s the difference between evaluation and research?
Clients sometimes ask me about the differences between evaluation and research, and which of these they need. People working in evaluation have different ideas on this. Evaluation is sometimes described as a form of applied research, or a subset of research. There are overlaps with research, and they use similar methods and designs. Evaluation applies the methods of social sciences research (interviews, surveys, focus groups, and so on) to a specific context.
The main characteristic that distinguishes evaluation from research is making value judgments. Evaluation makes judgments about an initiative’s value, quality or worth. Research doesn’t. So, evaluation differs from research in its purpose.
Evaluation and social research have a different purpose
- The purpose of evaluation is to learn or make decisions – the ultimate goal is to improve the design, implementation or effects of a specific initiative.
- The purpose of research is to test theory, and to produce findings that can be applied to a wider population (be generalizable) – the goal is to contribute to knowledge.
What questions does evaluation ask?
When writing evaluation questions, it is helpful to group questions under topics. I have grouped some example evaluation questions into 5 general program issues that evaluation questions typically cove, but these will vary between different evaluations.
How can evaluation help your organization?
How evaluation will help your organization will depend on the type of evaluation, and the stage of your initiative. An evaluation will provide you with useful information to demonstrate the need for your initiative, you will learn how to improve its design and implementation, and you will gain useful evidence of how it benefits people.
Summary: How evaluation can help your organization
In this article we covered:
- Why you might want to do an evaluation
- What evaluation is, and what it isn’t
- The differences between evaluation and research
- The ways evaluation can help your organization
Want to read more?
We hope you found this article helpful. For more information, check out our other articles about evaluation – there’s one about types of evaluation, and another about how to increase the use of evaluations. You can also follow us on Facebook to hear about our latest posts. If you need an evaluator, please get in touch. You can also read about our evaluation services on our website.