User test for Edtech Learn Collaborate
Outcome:
The user test report and insight contributed further potential funding - including $1M grant
At the end of MVP1, we were be able to make changes on 6 out of 12 insights from the user test.
Feature improvements and functions for future phases
The Problem:
Teamwork is one of the most important skills for the real job world. However, college students often lack sufficient opportunities to develop this skill during their college years, a situation exacerbated by the pandemic. According to Debra Mashek's 2021 article, there is a significant gap between the skills employers need and the reality of students' experiences.
Employers:
rated “Ability to work in teams” as
“somewhat important” + “very important”
94%
Background:
To solve this problem, the founder had a vision to build a product, where
Students can create real life scenario projects, invite other people and participate class assignments for their future portfolio.
Instructors can assign, overview, and grade class project efficiently
Although the agency had already built high-fidelity mockups and was ready to move on to development, the founder requested us to conduct a user test. They wanted additional insights into the overall user experience and any necessary features from our UX designers and researchers. Given my background in higher education and mentoring college students, I felt this project perfectly aligned with my experience and passion.
Tools used in this project: Figma(Fig jam), Airtable, Maze, Zoom, Loom, Otter, Click up
Responsibility:
The wireframe has been already developed by the agency, the design/development agency from the discovery user research. My colleague and I were responsible in this project include:
Prioritize the usability problems. Prioritize the problems that are most likely to prevent users from achieving their goals on the platform.
Build User testing and recruit potential user testers.
Conduct user testing.
Analyze the data to identify usability problems & ideate design suggestions.
Track design changes & deliver final report including charts and visuals to the stakeholders
Provide user walkthrough video
My colleague had been working in a prior project within the company, therefore they took a charge on outlining the sprint plan and tracking them. Most of main tasks were equally divided among 2 of us, as we would take initiative on any other tasks that require attentions voluntarily.
Project Overview
Onboarding. Review wireframes, prioritize the usability problems and work with the agency
Prep, plan & recruit for the student user testing
Conduct remote, modified user testing - Student
Conducted user test sessions with 4 out of 9 college students. The focus was on evaluating the ease of completing the student flow tasks.
Synthesize data - Affinity map & Maze quant data
Discovered very clear 3 P0 issues, which interfered users completing the tasks. Majority of users failed/struggled on 1) ‘Mark completed task’ function, 2) Instruction note for ‘Milestone Summary’ and 3) completing ‘Milestone Summary’ function. In addition, most of the users expressed on the platform’s complex hirarchy.
Provide a short summary for the design decision
Phase 2: Instructor User Testing - In addition to evaluating the ease of flow completion, we placed a greater emphasis on user interviews to gain further insights. This approach was crucial as instructors are likely to be the decision-makers for adopting the platform at their institutions.
Collaborating data synthesis
Created a collaborative Figjam Board for different synthesis options
Organized all insights in relation to the user flow to visualize largest friction points.
Organized data points into an affinity map to find repeating insights & emerging patterns
Discussed and organized themes and groupings based on priority, pain points, celebrations, future features
Cross-checked and paired with the quantitative data from Maze report - Usability test score, Misclick rates, click path, heat maps
“Provided for ways to make team-based class projects more effective, enjoyable, or productive,”
62% answered “None”
College Students:
What’s the biggest challenge: “coordinating group projects and keeping group members accountable”
40%
Phase 1: Student User Testing - Users worked on 5 tasks using maze, sharing their screen and camera. Then the moderators asked post-test questions to get more insights tasks.
Data Synthesis - data extraction
Gathered in-session notes based on immediate observations and feedback
Transcribed audio and video of the zoom sessions
Reviewed session recordings & transcripts to ensure all insights were captured
Extracted and highlighted significant data including quotes, behaviors, click paths, emotions, and direct feedback
**Due to the time constraint, Instructor flow started simultaneously while finishing student flow’s data analysis & summary
Prep, plan & recruit for the instructor user testing
Conduct remote, modified user testing - instructor
Conducted moderated, remote user test sessions (Maze & zoom) with 5 out of 10 college and high school instructors.
Synthesize data - Affinity map & Maze quant data
Instructor data was more complexed to bring up to the common themes as there was more interview pieces. To tackle this, my colleague and I set up multiple collaborative analysis sessions, moving sticky notes and validating thoughts during the discussions. After a few rounds of collaborating affinity map, 4 major themes were 1) Grading related issues, 2) Classroom overview functions, 3) platform hierarchy and 4) communication & feedback.
Provide a short summary for the design decision
Track design changes & deliver final report (58p) including charts and visuals to the stakeholders
P0 (level 1) issues: With the insights, my colleague and I suggested possible immediate solutions. For example, to tackle the complex hierarchy on the platform I suggested vertical arrangement inspired by one of my student participant. My colleague continued and developed on rough wireframe. The design agency adopted the suggestion to the prototype, resulting much simpler and intuitive structure.
——->
Before: 2 column structure confused the users about the hierarchy structure —> After: Vertical arrangement gives clear hierarchy
Collaborative problem solving: Due to the limited amount of change order to accommodate test insights, the steak holders, PMs, designers, and developer collaborated on ideation, utilizing current functionality within the phase as much as possible. For example, messaging and giving feedback was a big part or the result from both of student & instructor user test. Instead of adding new function, my colleague and I suggested utilizing email trigger, which was already included within the phase. The insights that could not be accommodated within this phase were prioritized and suggested on the next phases.
Sticky Deliverables: These insights and suggestions were recorded on the report with visuals, including heat maps, click maps, tables, task difficulty charts and before & after images. To add more storytelling, my colleague and I created 2 user personas: Abby the student user & Brian the instructor. These persona voices were used throughout the insight result, combined with the real participants’ quotes.
Provide User walkthrough Video & Hand over
Constraints
Time crunch for the research: planing, building, recruiting, conducting, synthesizing and providing summary
Limited amount of change order to accommodate test insights -> Collaborate and participate ideation, utilizing current functionality as much as possible
Overall design support, other than user test - design change tracking, drop down lists, video walkthrough
Project outcome
The report contributed to the positive outcomes for future $1M grants & fundings for further development
Contributed to meaningful design changes - At the end of MVP1, we were be able to make changes on 6 out of 12 items from the user test.
Design improvements and functions for future phases
Please contact for the detailed outcomes, I am happy to talk about our final report including the visuals of before & after.
Earned to navigate constraints and define design priorities effectively within the current project phase.
Acquired skills in collaborating with stakeholders, product managers, developers, and designers to ensure alignment and mutual understanding of product decisions.
Developed the ability to make collective product decisions as a team, leveraging diverse perspectives and expertise.
Gained proficiency in using new tools such as Maze, Otter, and Loom for enhancing the research and analysis processes.
Bridged expertise in education from my educational background, contributing insights and understanding to the project's design decisions
