Analyzing 30,000 Student Course Evaluations to Inform Curricular Redesign
Thompson Rivers University (TRU) decided to redesign their course evaluation survey and wished to identify key takeaways from the original surveys as part of the sun-setting process. Kai Analytics was tasked with analyzing 33,722 raw-comments from stakeholders so TRU could use their data to inform the redesign.
What We Did
Kai Analytics used text analysis to effectively clean, parse, and summarize the comments into over 200 major themes across sixteen questions. Amidst the thousands of positive comments, text analysis uncovered a wide-range of areas for improvement, as well as hidden themes that allowed the institution to engage more deeply with curricular alignment.
Identified 3 major takeaways from previous responses to use in future course design.
Identified the different needs of sub-groups within the student population.
Drafted recommendations to guide the creation of an updated student feedback system.
What would have taken us weeks or months took them a matter of days, and we were able to use the resulting summary data and reports to inform our colleagues.
— KRISTEN E.H., MANAGER OF INTEGRATED PLANNING &
EFFECTIVENESS, THOMPSON RIVERS UNIVERSITY
3 Key Insights
Who Said What?
We use charts like the one below for Sub-Group analysis. Dots in the top right corner were mentioned more often than dots in the lower left corner. The farther a dot is from the 45-degree line, the stronger the correlation to a group. The chart below shows us that when asked for ways that their program could be improved, students who were more unsuccessful were more likely to suggest math and science courses.
Work Smarter, Not Harder
Qualitative analysis doesn't have to feel daunting. Using our in-house text analysis methods, we were able to synthesize nearly 34,000 individual comments like these in a short period of time, minimizing bias without sacrificing accuracy. And we wrapped it all up in a readable report. Gone are excel files like this one, instead we use intuitive graphics to tell the story within the data.
Who Said What?
Informed by the sub-group analysis, Kai Analytics was able to make two key recommendations for future program development.
-
Develop an outreach program for unsuccessful students using the themes identified in the report as a way to address their ongoing concerns
-
Using follow up interviews combined with attrition data, it is possible to create a student attrition model