There are lots of questions about the relative value of one-off short courses as opposed to courses that are built into larger efforts to strengthen individual or organisational capacity. I'm trying to get to grips with how to assess effectiveness of such courses and here are some initial thoughts, also on EvaluatingTraining.
EVALUATING ONE-OFF TRAINING
INITIATIVES
TO PROMOTE SOCIAL JUSTICE
Barbara Klugman
Strategy and Evaluation Practitioner
February 2013
This is an area
I’m hoping to study further over time, but here are some initial thoughts…
One-off
training unlikely to result in participants making many changes, depending of
course on who they are and what the context is. But as a general strategic
approach, training does better when it’s part of an ongoing process of support,
networking, advocacy etc. Why? Because it allows the training to be located in
relation to the context and actual work (whether voluntary or paid) of
participants. This is particularly important because individuals often leave
the training to go back into organisational contexts that are not open to the
new learning the individual brings in. Hence training as part of organisational
change, or development and implementation of organisational strategies is
likely to be more effective.
What makes training work?
If you are offering a one-off training process, whether of
a few days, or weeks, it’s worth locating it within a theory of change. What do
we know about how to shift individual perspectives? Research evidence as to
what factors influence individuals to change their practice[1][2]
suggest the following theory:
Figure 1: Factors that
influence behaviour change
Context / Environment
Knowledge
+ Motivation + Skills = Behaviour Change
The Barefoot
Collective refer to this as: head, heart and hands/feet. [3]
Box 1: Training aims to contribute
towards enabling individual action through increased knowledge, motivation and
skills
KNOWLEDGE
in relation to
·
an
issue
·
an
approach (for example an approach to strategic planning, and organising, or
an approach to social justice)
·
how
the issue plays out in diverse contexts
·
what
other players globally are doing and how to link with them
|
MOTIVATION
through
·
initiating
or consolidating relationships of trust that can provide solidarity and / or
links to resources
·
greater
confidence in oneself and / or one’s organizational strategies as a result of
positive reinforcement in the process
·
“widened
consciousness” [4],
experiencing oneself and one’s organization as part of a larger movement
·
positive
emotional energy generated in the process
|
SKILLS
in relation to any for example
·
how
to do a situation analysis, plan or implement an initiative
·
personal
capacity: how to do a public speech, how to supervise staff
·
orgnaisationsl
stystems: how to set financial, information or other systems
·
movement-building:
how to mobilise individuals, manage alliances, engage the media…
|
Knowledge
The type of
pedagogy needs to enable people to participate in taking in and engaging new
information to make it theirs – receiving inputs or doing reading is seldom
enough on its own, especially for a non-academic audience. Moreoever, if the information provided does
not take account of its pertinence to diverse contexts (and your participants
are from diverse contexts) then the programme needs to include time and methods
for reflecting on relevance, adaptability to diverse socio-cultural, economic
or political contexts. You may want to create opportunities to talk about an
input or reading; opportunities for participants to share their own experience,
or even write-up stories or case studies and then allow the group to reflect on
these in relation to new theories provided in the training, or experiences
elsewhere in the world. People should not be bombarded with information,
irrespective of how creatively this is done e.g. with movies and stories. They
need time to digest new ideas and information, including unplanned time, time
to chat informally with others and so on.
They also need to have time to move about and exercise so that their
bodies remain relaxed and minds remain alert.
Motivation
If the purpose of the training is to support social
change, then participants leaving with new knowledge is not enough. The way the training is run needs to build
their motivation. This can be done in many ways, including through the
excitement of meeting others doing similar work or concerned with similar
issues and having time to learn about their experiences and build bonds with
them; through opportunities to visit groups doing inspiring work and get
concrete ideas about how to make change happen; through providing follow-up
opportunities that will further build the participants’ capacities, links to
others and so on. Pragmatically, motivation can be enhanced if the training
results in accreditation, certification or some kind of formal recognition.
Skills
People are less likely to use lessons learnt, if they are not
motivated and do not know what to do. Hence the more that training can help
people work out how to do something differently, the more likely that it’ll be
useful. This means creating opportunities in the training for trying out new
ways of doing things and for planning how to do them in the home environment.
Who comes may influence the
effectiveness of the training
We know that
when an individual goes back to their home environment, unless it is a highly
conducive one, the person may find it difficult to introduce ideas learnt
through training. Some issues to consider when choosing participants include
that the more people who attend from one place, the more critical mass they can
bring both to shaping up ideas during the training, and to influencing others
when they take these ideas to their home context; and those who are relatively
junior in an organisation usually have less power to influence decision-making
so that training people who have lesser organisational status means helping to
identify realistic entry-points, or focusing on building their personal
capacity rather than assuming they should make organisational changes. Also,
bringing in a group with very mixed capacities and different styles of learning
poses major pedagogical challenges.
Clarifying Your Theory of Change
and How to Assess Effectiveness of Training
The ‘Theory of
Change’ for one-off training suggests the need for careful planning of the
process, firstly by establishing your own theory of change. Is change in awareness enough? Is motivation
enough?
What is it that
you intend the training to achieve in relation to information – simply
receiving information, or participants internalising the information? Are you
hoping participants will shift their discourse, using new concepts and
language? Are you hoping that they will analyse their own situations
differently? You need to have clear objectives for each session of the training
and for the course overall which you make explicit. You can then work out how you will identify
whether each objective has been realised cumulatively over the course. You
could assess the degree of internalisatoin by the extent to which information
shared is reflected in the contents of a project plan or some other product the
participants are required to do during the training.
How will you
build motivation? Once you’ve decided on what aspects of content, methodology,
and the overall shape and style of the training will help to motivate
participants, you need to keep an eye on whether this seems to be working
during the course. You could assess motivation through an end-of-course
questionnaire, but that will only tell you about the degree of motivation at
the time of leaving the course. For this reason, it is essential to make an
assessment some time after the course, to see whether the actions the person
felt motivated to take at the end of the course have actually been taken.
Are you aiming
to build skills? If the training is not focused on skills-building, then you
need to consider what other opportunities participants will have whether within
or outside of your overall programme to actually apply their knowledge.
If you are
trying to deepen participant capacities to do things differently, you need to
be quite clear as to what these skills are, to ensure that the training content
actually provides the necessary inputs and opportunities for building and
testing skills – and by monitoring participants in this process you will see to
what extent they are harnessing the skills at the time.
While you can
and should build in assessments throughout a training programme to see whether
the shifts your theory of change intends you to achieve are actually taking
place during the training, it is essential to see whether these remain over
time, and whether participants are able to apply whatever has been gained by
the training. You may be able to use a survey to gather very specific
information, and supplement this by asking participants to share stories of
changes they have made at some realistic period – perhaps three or six months
after the training.
By way of illustration:
·
You could
ask: ‘Have you made any changes that drew on ideas you got during the course?’
yes no. If 'yes' please describe what you changed, when, where and with whom?
…..
·
If you
want to be more specific, which can help participants to reflect on their
experience, you could do a few questions like this in a survey, in each case
asking about a different kind of change, depending on what you think you can
reasonably expect, for example one question could focus on 'changes in your
organisation's or campaign's strategies'; another on 'in the way you talk with
colleagues about (whatever the topic of
the training was)'
·
If
participants did any work on framing an initiative or project during the course
then you can ask them to describe what they've done since then.
·
You could
ask if they have had any contact with anyone they met during the course and if
they say yes, ask whether this involved, and give them options like: sharing
ideas; working together, and so on, and then ask them to give more
details
In conclusion,
it’s worth thinking about the limitations of one-off training. Would it be
better to bring the same group together repeatedly to build a really strong
cohort with high level capacities, as opposed to giving basic skills
repeatedly? Presumably the answer depends on what you’re trying to achieve.
One of the most
profound impacts of training can come through the relationships built among
participants which may allow them to provide each other with ongoing support,
to strategise together, and thereby strengthen the overall efforts towards
social change. Establishing or
reinforcing mechanisms for ongoing linkages between participants can be a key
factor in enabling the training to make a difference once participants leave.
Monitoring the extent to which such mechanisms are used and in what ways, can
be another helpful way of seeing whether training is having its intended
outcomes.
More thoughts
to come as I work further with more groups on this issue…
[1] Fisher, W. A., & Fisher, J. D. , A general social
psychological model for changing AIDS risk behavior. In J. Pryor & G.
Reeder (Eds.), “The social psychology of HIV infection,” Hillsdale, NJ:Erlbaum,
2003:pp. 127–153.
[2] I have used the term ‘individual
action’ where theorists usually refer to ‘behavior change’ since the latter
term can be heard as problematizing participants and their current behaviour,
whereas the intention here is to deepen people’s knowledge, motivation and
skills in ways that strengthen their existing work. Thanks to Kate Hoshour,
International Accountability Project for this insight.
[3] Barefoot Collective, Barefoot
Guide to Working with Organizations and Social Change, The Barefoot
Collective, (published online), 2009:140.
[4] Lindell, I., ‘’Glocal’ movements: place
struggles and transnational organizing by informal workers’, Geografiska Annaler:Series B, Human
Geography, 2009: 91(2):123-136.
