Creative evaluation in online outreach
Creative methods for research and evaluation have been gaining increased focus in recent years. They are something that myself and Dr. Samatha Child have had the opportunity to present on several times this year. Often there is a keen desire to be more creative in approaches to evaluation but this can be limited by capacity, knowledge of tools and methodologies or simply not knowing where to start. Furthermore, the current pandemic has meant that many of these methods need to be adapted for online audiences.
Over the past two weeks I have been running a two-part workshop with a number of evaluators working within Northern Uni connect partnerships. The goal for these workshops was to support evaluators and practitioners in thinking about ways to embed more creative forms of evaluation within online or blended delivery projects. Through working through a structured process, the workshop helped evaluators to identify where these approaches can still be used and, in some ways, in a more in-depth way than may be possible in the classroom.
Creative methods are engaging, and by using engaging methods, we can improve the focus of participants on the programmes being delivered. It is hard to be passive when you are being asked to create, so these methods lend themselves well to supporting participant engagement. Some examples of creative methods can include: graffiti walls, photo elicitation, collage, vox pops, drawing, lego, plasticine modelling and poetry. There are also excellent volumes out there detailing the breadth of these methods such as Helen Kara’s Creative Research Methods or Helen Kara, Narelle Lemon, Dawn Mannay and Megan McPherson forthcoming book Creative Research Methods in Education
Within traditional face to face interventions, the time and space for evaluation can be limited. This has often led to evaluation being sidelined to the end of an event or supplementary activity. Yet evaluation does not need to be, nor should it be an ‘add on’. In embedding evaluative activities within the pedagogy of programme design, we can create ways of meaningfully understanding the impact outreach work has on participants. I would argue that in having to rethink programmes for a new mode of delivery, we have the opportunity to build this evaluation within programme design not only to improve the data gleaned from evaluation but also to create new, more engage pedagogical activities that can be part of programmes in their own right.
Despite these benefits, what can be a challenge is how to facilitate these methods online. This requires us to consider how to either re-imagine these methods in a digital form or use digital tools to capture the data. Both of these approaches have benefits and limitations and what is right for each project will vary. The approach I propose is to start from a focus on what data is needed to measure the outcomes you are evaluating. Is it:
The participants narrative about their artefacts- Are you interested in what they have made or the stories within that creation
The artefact itself - If so, what form does it take. Is it 2D? 3D? Performed?
The process of creation - Are you measuring changes over time and are these what need to be captured?
Once you have answered these questions, then finding a digital tool to support the data collection becomes a less challenging proposition. Some of the tools I recommend exploring include flipgrid (ideal for video), padlet (ideal for photos and getting participants to comment on images or each others work), Blogging tools to act like a digital diary (ideal for tracking change over time) or simpler approaches such as a email as a place to submit documents to.
As you can see, there are always ways and means of moving these methods online. However, this process can take some time to work though. As with using any new tools or methods, practice makes perfect. Try them with your teams, or run some test activities using these platforms with target groups to see how they react. There may need to be time and space build in to build their creative confidence for example.
What I would urge you to do is to try and incorporate these methods where they are appropriate to the objectives you are evaluating. They have the potential to generate rich data and to engage participants in a way more traditional methods may not be able to. If you or your team need some additional support in embedding these methods, please do get in touch.