Analyzing 30,000 Student Course Evaluations to Inform Curricular Redesign
Thompson Rivers University (TRU) redesigned their course evaluation survey and wished to identify key takeaways from the original surveys as part of the sun-setting process. Kai Analytics was tasked with analyzing 33,722 raw-comments from stakeholders.
As a part of the wrap-up of TRU’s old course evaluation process, Kai Analytics was asked to analyze the 33,722 raw text comments of previous student surveys and find themes that could be used to help design the new course evaluation program. We got there by finding key takeaways in student responses, breaking down the differences between sub-groups, and understands the general feelings about the course.
Identified 3 major takeaways from previous responses to use in future course design.
Identified the different needs of sub-groups within the student population.
What we did
Advanced text analysis effectively cleaned, parsed, and summarized the comments into over 200 major themes across sixteen questions. Amidst the thousands of positive comments, text analysis uncovered a wide-range of areas for improvement, as well as hidden themes that allowed the institution to engage more deeply with curricular alignment.
Drafted recommendations to guide the creation of an updated student feedback system.
3 Big Highlights
Who Said What?
We use charts like the one below for Sub-Group analysis. Dots in the top right corner were mentioned more often than dots in the lower left corner. The farther a dot is from the 45-degree line, the stronger the correlation to a group. The chart below shows us that when asked for ways that their program could be improved, students who were more unsuccessful were more likely to suggest math and science courses.
Work Smarter Not Harder
Qualitative analysis doesn't have to feel daunting. Using our in-house text analysis methods, we were able to synthesise nearly 34,000 individual comments like these in a short period of time, minimizing bias without sacrificing accuracy. And we wrapped it all up in a readable report. Gone are excel files like this one, instead we use intuitive graphics to tell the story within the data.
So What Happened?
Informed by the sub-group analysis, Kai Analytics was able to make two key recommendations for future program development.
Develop an outreach program for unsuccessful students using the themes identified in the report as a way to address their ongoing concerns
Using follow up interviews combined with attrition data, it is possible to create a student attrition model