Before we head into a deep-dive on open-ended questions, don’t forget that there’s just over a week to go until our final Golden Ticket prize draw, happening on 14th December! There are four opportunities to win, with a star prize of £1,000 up for grabs, or three Fortnum and Mason hampers!
For every 30 questions you answer, you get a ticket to the draw, so even if you’ve only answered a couple since the giveaway started, there might still be time to get a ticket! The winners will be announced on the app and via social media, so keep your eyes peeled 👀
Right, onto today’s topic…
Open-Ended Questions
About 18 months ago, we asked our first open-ended question on Teacher Tapp. We were very excited about this, although we had hesitated about introducing them for some time.
Firstly, typing out lengthy responses is more demanding and time-consuming than selecting options from multiple choice. After all, Teacher Tapp was built to be quick and easy to answer, so to not introduce further challenges to your work life! Secondly, these responses are somewhat complex for us to analyse, a topic we’ll explore a bit later in this post.
So, why did we finally decide that we must ask occasional open-ended questions? Well, there were three main reasons:
- Unanticipated Responses: To construct effective multiple-choice questions, we need to predict potential responses accurately. This is typically feasible when schools operate normally, aided by the insights of our wonderful network of teachers on social media, but there are sometimes situations where this is impossible (the pandemic was one).
- Avoiding Leading Responses: For certain questions, it’s crucial that we capture your immediate, unguided thoughts, uninfluenced by the options presented.
- Richer, More Detailed Data: The open-ended format often yields fascinating insights into complex topics, as teachers express their thoughts, experiences, and opinions, which in turn inform our subsequent questions.
Who’s answering open-ended questions?
In these first 18 months, we have asked 108 open-ended questions, but you won’t have seen them all. Only 62 of these have been presented to every teacher on the app. Often, these questions are targeted at specific groups, such as primary teachers, science teachers, or headteachers. Additionally, some open-ended questions are presented to those who gave a specific response to an earlier question to delve deeper into their reasoning.
It might seem that open-response questions would inherently provide a more comprehensive and accurate picture of the teaching profession, but this isn’t entirely the case. One reason is that the response rate to open-ended questions is generally lower.
Currently, about 42% of those answering the multiple-choice questions opt to respond to the open-ended ones. However, this average conceals significant variation, with the highest response to an open-ended question at over 70% and the lowest below 10%. We’ve observed that primary teachers are often the most engaged in responding to our open-ended questions, while younger and less experienced teachers are more inclined to bypass them.
Your most and least favourite open-ended questions
The Top 5 Open-Ended Questions You’ve Engaged With:
- Reflecting on the Return: “What word or phrase would you use to describe your first two weeks back?”
- Considerations of Leaving the Profession: “In those moments where you feel like leaving the teaching profession, what is the main reason?”
- Behavioural Challenges: “Which pupil behaviour makes you most angry?”
- Magical Transformations: “If I could wave a magic wand over my school, I would…”
- Building Improvements: “If you could improve one aspect of your school’s building, what would it be?”
The Questions That Didn’t Resonate As Much: Despite the insightful responses we’ve received, some open-ended questions have seen less engagement. Here are the factors contributing to their unpopularity:
- Niche Topics: Questions about specific policies or curriculum areas unfamiliar to many, such as restorative justice and digital citizenship, tend to be skipped.
- Complex Queries: Questions requiring more in-depth thought or elaboration, such as offering advice to another teacher, are often overlooked.
- Response Fatigue: When open-ended questions appear at the end of a longer set of extra questions, response fatigue can lead to lesser engagement.
- App-Related Feedback: Surprisingly, questions seeking suggestions for changes to the app itself have generally been met with indifference!
What are typical responses like?
When presented with the freedom to express in detail through open-ended questions, we’ve observed a tendency towards brevity among many respondents! On average, approximately 10% of you offer a one-word answer, though this rate fluctuates depending on the question posed. Similarly, about 10% write responses exceeding 140 characters (the length of an old-style tweet).
Intriguingly, we often garner more insights from multiple-choice questions than from open-ended ones. For instance, we recently asked you to identify your strengths as a teacher using both formats. When posed as an open-ended question, the majority mentioned only a single strength. The challenge, it seems, lies in spontaneously recalling multiple strengths. And yet, when provided with a list of skills in a multiple-response format, two-thirds of you were able to recognise more than one area of strength. Remarkably, a quarter of the respondents selected four or more strengths!
What do we do to analyse your responses?
Analysing thousands of open-ended responses is definitely harder than analysing data from a multiple-choice question. We often start by looking at what sorts of words you are using in your responses. For example, the word cloud below shows how you responded to the question about what pupil behaviour makes you most angry.
If we ask a question that invites a scale of responses from positive to negative, we can conduct a sentiment analysis. For example, when we asked you to describe your first two weeks back in school last September, we observed 39% of you writing positive statements, 42% of you writing neutral statements and 19% of you saying something negative about the return to school. We can then look more closely at the groups of responses. For example, the word cloud below shows the words used by those of you who wrote something positive about the return to school.
We often categorise the text so that we can summarise your responses by demographic group. For example, when we asked you what the main topic of a recent INSET day was, you gave us 2867 unique responses describing what happened in your school. We then assigned them to one of 10 categories, from safeguarding training through to teacher wellbeing sessions. Sometimes this is easy – it’s fairly obvious that a session on ‘new rewards and consequences’ is about behaviour. However, it is often very hard!
One of our favourite things to do with open-ended responses is to link your free text words to other multiple-response questions you answered. For example, we recently showed how the words you used to describe your school were linked to the level of morale you reported.
Open-text responses can also help us understand why teachers are inconsistent in how they respond to other questions. For example, last year we asked science teachers about their departmental working environment and we wanted to explore the curious responses of those science teachers who described quite a difficult working environment (e.g. staff shortages, having to teach out of specialism, few opportunities for professional development) and yet said they enjoyed working there. When we asked them to explain why, it revealed how important friendship, support, cohesion and even banter is amongst teachers! It is possible to happily work in quite challenging situations if you have the right people around you!
Thank you to everyone who chooses to answer our open-ended questions. Your responses, whether brief or detailed, are invaluable in painting a vivid picture of the teaching landscape. These responses not only enrich our understanding but also play a crucial role in shaping the questions we ask in the future.
Is there an open-ended question you would like us to ask? Let us know via social media, email or the app.
Ups and Downs
On the rise
📈 Half of teachers would like additional safeguarding training on managing challenging family situations, up from 40% in October 2022
📈 22% say they have an NPQ – up from 16% in June 2022
Heading down
📉 Just 15% of primary teachers are not a subject lead for any subject in their school – compared to 19% in May 2021
Daily Reads
The most read article from the last week has been: 10 things to avoid doing or saying!
And here are the rest for your reference:
