Engagement with Questions
Redesigning the quiz editor to encourage teachers to create more engaging, media-rich questions.
Role
Research, Product Design
Timeline
June 2021 - July 2021
Cross-functional team
3 Engineers | 2 Designers | 1 Product manager
Want to skim through this case study? I got you covered.
Here's a 1 min TL;DR version.
Quick overview
This project was about create a unified question editor interface that would encourage users to create more engaging content, for both quizzes and slides.
What did I do?
I worked in a cross-functional team to create the final high-fidelity responsive mock-ups and prototypes for the new question editor interface.
The impact I made
With the new editor, we met all our KPIs like increased media usage (up 54%) and higher question type variety (up 15%).
What did I learn from this?
To not be afraid to suggest big changes on core projects. Be cautious, and conduct thorough testing, but if something shows promise, try it out.
1. Context and Problem
Product overview:
Quizizz has two primary products for in-person experiences, Quiz for interactive assessments and Lessons for interactive instruction.
Why we were working on this:
Student engagement is key to product success
Our interviews with students and teachers (qualitative data) as well as our collected data (quantitative data) clearly showed that questions with pictures, audio or videos got higher engagement levels in students as well as high rates of completion. Because of this we wanted the new editor to make it easier for teachers to add media to their questions.
Consistent experience for teachers and students
The existing interfaces for questions for the Quiz editor, Lessons editor and student side were all very different from one another. Having different interfaces for the same experience meant users had to learn more things. We wanted to create a question editor on the teacher side that is more more in-line with the student side experience. The question editor on Quiz was geared to offer a mobile preview, being a web-first product, a desktop preview would be useful for more of our users.
Save on development and maintenance cost.
As a small team of less than a 100 people, Quizizz wanted to reduce the development cost of maintaining three interfaces which all effectively served the same purpose. With a single unified question editor, adding features would be simpler, faster and less prone to breaking. The fact that is was something that would benefit the users as well made it a no-brainer as a project.
2. Usage goals
Encourage more media in questions
Our interviews with students and teachers as well as our collected data clearly showed that questions with pictures, audio or videos got higher engagement levels in students as well as high rates of completion. We wanted to encourage teachers to add media to their questions.
More variety in question types.
Not all questions can be MCQs, sometimes teachers need subjective assessments via open-ended answers or maybe fill-in-the-blanks.
From the students' side, this would also prevent a quiz from feeling repetitive.
From the students' side, this would also prevent a quiz from feeling repetitive.
Higher adoption of answer explanations feature.
Allow teachers to break down complex questions, concepts, and their answers for their students.
Help students go from “??!” to “Oh! That makes sense!”
Help students go from “??!” to “Oh! That makes sense!”
3. Existing Questions Interface
Question editor on Quiz
The editor gives you a preview of the question on a tablet screen and the interface makes the user focus on the text aspect of the question.
Question editor on Lessons
Primary focus is again kept on the text content and selecting the correct answer. The media options are not positioned to feel like an important part of the interface.
Student side question interface
The student-side interface was visually very different from the teacher-side.
4. ✨ The new question editor ✨
Identify gaps -> brainstorm -> design & prototype -> test -> re-iterate -> repeat till needed -> succeed
The new Question Editor
What we did and how we did them:
Encourage more media use in questions
- Add images directly through google image search.
- Search and embed youtube videos into your questions without leaving the tab.
- Significantly increased visibility for media options.
More variety in question types.
- Option to switch between question types while editing a question.
- Host a poll, fill in the blanks, ask for long answers or even draw your answers. The new editor made switching between super easy
Higher adoption of answer explanations.
- Increased visibility for the feature.
- Made to feel more like an essential part of the experience rather than a good-to-have option.
- Use both text and media in answer explanations.
5. Adoption
Unanimously preferred
When the new editor was launched we gave our users the option to revert back to the old editor if they wanted to. 30 days into the launch the number of users reverting was only 5% and that came down even further to 3% by 60 days after launch.
6. Impact
Our key metrics were all successful
Media used per 100 questions
Old editor:
13.56
New editor:
20.94
Impact
+54%
Quizzes with more than one question type per 100 quizzes
Old editor:
34.88
New editor:
39.96
Impact
+15%
Answer explanations used per 100 questions
Old editor:
0.41
New editor:
4.32
Impact
+10.3x
However, not everything went according to plan
Math equations used per 100 questions
Old editor:
2.52
New editor:
1.97
Impact
-22%
7. Social media response
We got unanimous applause from our loyal community of teachers
8. Learnings and Reflections
To not be afraid to suggest big changes on core projects
While working on this, I was fully aware that the new design might be met with criticism simply owing to the tendency of people resisting change. This is exactly why we decided to test the feature in real life with real teachers and see the reaction before launching to our users.
What I learned from all that? When working on core products, be cautious, and conduct thorough testing, but if something shows promise, try it out.
What I learned from all that? When working on core products, be cautious, and conduct thorough testing, but if something shows promise, try it out.
Could things have been even better?
For a major update to a core product, this was done with a very tight deadline. To save time on design and development, we decided to remodel the editor to be similar to the existing student-side interface. With more room to experiment we could've tried other more radical ideas.
Though we achieved all our targets, I was left wondering if things could've been better.
Though we achieved all our targets, I was left wondering if things could've been better.