Amy Finch, Head of Learning and Impact, Spirit of 2012
At Spirit, we ask all of our projects to ensure that they work to challenge and change perceptions of disability during the course of their projects. It’s one of the three most important priorities of our funding, as outlined in our impact statement. But how can projects understand whether they’re making a difference in this area?
In the past, we have asked projects to answer two specific questions:
Q: Thinking about disabled people in general, how much of the time, if at all, do you think they can lead as full a life as non-disabled people? (All of the time, Most of the time, Some of the time, Rarely, Never)
Q: Generally speaking, do you think there is a lot of prejudice in Britain against disabled people in general? (A lot, A little, Hardly any, None, Don’t know)
This approach has several advantages:
- The questions are externally validated tools that appear in major surveys, so they are viewed as trustworthy questions, and we can compare the data to national averages.
- Having just two questions allows us to add all the responses up and so we can show the collective impact Spirit projects are having in this area.
- We can compare projects to start to ask questions about why different interventions might lead to different results.
However, over time we have come to realise the disadvantages outweigh these advantages.
- Separation of “non-disabled” and “disabled” people into discrete groups, with the implication that one does and one is done to. This could be construed as “othering” disabled people and potentially undermine the intention of the project.
- It is unclear what “the right” answer would be in some cases, and thus what difference a project should be aiming towards between the baseline and end-line
- Some have questioned whether treating “disabled people” as a group makes sense, when there is such a variety of experience. There is a rich literature about what defines a common disabled experience, but such nuance is outside a simple survey question.
- Making some of these questions mandatory can mean that they appear to be random to survey respondents.
We also know that our projects are exploring this topic from many different angles.
Some of our projects are leading organisations in the field of disability arts or sports, who are looking to attract more non-disabled people as audiences or participants in inclusive activities.
Others are organisations are looking at reaching out to disabled audiences, removing barriers to their participation and ensuring that their activities are accessible, welcoming and rewarding for all.
Still others are directly confronting stereotypes or limiting perceptions – including self-perceptions – head on, providing opportunities for disabled people to develop their skills or have their voices heard.
So, how should you capture information?
1) Firstly, articulate clearly what making improvements in this area means for your project. Our Theory of Change identifies three main areas:
- (a) Disabled people are better able to participate.
- (b) Disabled people are more integrated in communities
- (c) There is an increased awareness of issues facing disabled people.
Your project may be focusing on just one of these areas, or on more than one but there should be a clear line of sight between project activity and how it contributes to those outcomes.
2) Look at some of the other outcome areas you are reporting on and how they can be used to demonstrate progress in this area.
If you are asking all participants if their confidence improved, can you look specifically at how this factor is changing for the disabled people you work with? Is it keeping pace with the rest of your improvements? If not, why not? Are there output-based measures that can provide useful data, such as increases in the number of disabled people accessing your service?
3) Pay attention to qualitative data. Can you hold a focus group to explore what your project beneficiaries think? This group shouldn’t be made up of the people who you think will tell you the most positive stories about your impact, but an opportunity to learn and test ideas.
4) Foreground the experiences and opinions of disabled people wherever possible. Project staff may feel that a venue is welcoming and accessible, but it’s the opinion of project beneficiaries that matters most.
5) Trial the questions you will be asking, and how you will ask them as early as you can so that you can make adjustments.
Making Routes used some of their grant allocation to pay for the surveys to be designed so that they were visually appealing, whilst Disability Wales had a focus group of young people co-create the surveys to ensure buy-in.
We are still looking for the “perfect questions” that simply and effectively support our projects to understand their impact in this area, and we recognise that there’s a lot more we can learn.
If you want to discuss how we at Spirit should be measuring perceptions of disability, please contact me on firstname.lastname@example.org