Well-designed surveys coupled with natural language processing algorithms are the best way for universities to understand the sentiments and satisfaction of faculty and staff. Having a common framework for conversation can help you better understand your employees’ needs, prevent university disputes from escalating, and if university strikes do end up occurring, the results of the surveys can help resolve them. Here is how we have helped other universities resolve strikes and deescalate work disputes.
1. Design the Survey
Good survey design practices include a mix of multiple-choice questions and succinct open-ended questions. To maximize the value of multiple-choice, Likert scale, or Net Promoter Score (NPS) questions, pair them with a follow-up open-ended question. For example, you could ask,
“On a scale of 1 to 10, how likely are you to recommend this company?”
Then, follow this up with an open-ended question,
“Please tell us the reason for giving this score?”
By setting up a question this way, you accomplish two things. First, you get an overall sentiment of employees’ thoughts around the issue. Second, you also get to know employees’ recommendations from their responses. If you decide to use Net Promoter Score, your baseline is: less than 6 is negative, 7-8 is neutral, and 9-10 is positive.
If you are wondering what kind of questions you should be asking to get the most insightful responses, here are some standardized questions that focus on these core areas:
Communication – How well does the organization inform its staff on matters that affect them?
Diversity and Inclusion – How accepted does staff feel and how safe do they feel in their work environment?
Employee voice – How safe does staff feel about the ability to voice their opinions?
Brand image – How proud does staff feel about working at your company?
Leadership – How clear are the long-term goals and strategic direction to staff?
Organizational change (if applicable) – How well have recent changes or restructuring been explained?
Employee review – How fair is the employee review process?
Supervisor – How well are staff treated by their immediate superiors? (*this is one of the top reasons why people leave their jobs)
Personal well-being – What is the work-life balance? Are staff under mental stress?
Collaboration – Is there synergy across departments?
Overall satisfaction – Would you recommend your company as a place to work?
It’s important to remain consistent with how you phrase your sentences and have consensus on the tools that you will use (e.g., everyone will input the data in excel), especially if you have multiple sub-sections in your survey. Typically, consensus on these matters can be achieved by simply having a review committee represented by a senior person in each department of your company or university come together and discuss what tools and methods would work best to design and analyze a particular survey. Once the survey is put together, it should be reviewed, edited, and approved by stakeholders to create buy-in and engagement. Lastly, you want to test the survey on a select few employees to measure the clarity of your questions.
2. Administer the Survey with Privacy and Confidentiality
Administering the survey requires privacy and confidentiality and will help achieve a good response rate to reinforce the survey’s validity among stakeholders. While universities hope for more positive comments than complaints, employee engagement surveys often illicit strong emotions or personal experiences from staff and faculty, so it is important that your employees know that their answers are respected and kept confidential.
There are a couple of things to note when addressing privacy and confidentiality. First, you should use anonymous links. With anonymous survey links, there is no way to trace the responses back to the person (unless they self-identify themselves in their open-ended comments, but Kai Analytics can automatically mask that for you with Named Entity Recognition). Depending on the survey tool you decide to use, you want to make sure you disable the collection of meta data information as well as IP information. A potential downfall of doing this, is that you won’t be able to control how many times an employee takes the survey. Hypothetically, a disgruntled staff could submit multiple responses. This is something to keep in mind when you’re looking over your data.
However, this is rare, and Kai Analytics can detect similarities in responses through our text analysis methods.
Another common method of administering the survey, is fully delegating this process to a third party, like Kai Analytics. If you choose to work with us, we will help you develop a secure email list and ensure a confidential process where each employee can fill out the survey only once. We will also make it clear to your employees that their email will only be used for the purposes of this survey and that their confidentiality will be ensured.
Another benefit of working with Kai Analytics is that we can tie department and socio-demographic information directly to your employee’s response, thus, shortening the survey burden. This means that we do not need to ask your employees for their age, gender, department or tenure. Also, we do not report any social-demographic categories that have 10 or less responses to protect confidentiality and to minimize the risk of identifying specific people. In such a case, we would work with you to find solutions for aggregating the data.
Finally, before launching an employee engagement survey, it is good practice to release announcements addressing common questions and concerns ahead of time through a trusted draft . If you work with Kai Analytics, we can create an announcement for you explaining to your staff how their information will be collected and used.
Overall, administering the survey and announcing it to your employees is a good opportunity to speak from your heart. By being honest, empathetic, and sensitive to your employees when providing an explanation as to why you are conducting this employee survey you instill trust in your employees. In turn, they will be more likely to return the favor and respond to your survey.
3. Analyze Results
Ideally, you want to collect a lot of quality responses. Quantitative responses, such as Likert scales, should be relatively easy to collect and straightforward to analyze and visualize. Most modern survey tools on the market have built-in reports and dashboards to help you report on the data, but do check whether your survey tool is compliant with updates in data privacy and storage laws.
The challenge that many organizations face is working with open-ended responses. If you are a small organization and you expect to collect about 100 or 200 responses, then you might get away with reading and analyzing your responses manually. The most common way of doing this is thematic analysis. This includes reading and rereading your responses, and developing themes you think these responses fall into. First, you should read your survey responses and jot down ideas that come up in the content of your data. This stage is of the process is called "coding" as you are creating "codes", or shorthands, for the content of your data. Once you have your codes, you should arrange them into broader themes.
Thematic analysis is a great way of analyzing qualitative data. However, once you reach 250+ responses, you should begin looking into other analysis options. While you can continue reading every individual comment, deciphering meaning between typos, incomplete thoughts, emojis, etc., it could become a daunting task. This task is made even more challenging when for every single comment you must group or tag it by theme, e.g., “unhappy with compensation,” “satisfied with career development prospects.” Even though this method is slow, it is consistent, as you have human beings reading and sorting every single comment.
Alternatively, you can collaborate with Kai Analytics. Our proven text analysis approach minimizes bias that could otherwise occur when analyzing qualitative data manually. One challenge with in-house analysis is that your analyst is closer to the people they work with and can therefore subconsciously be influenced by the comments that they read. Kai Analytics uses natural language processing technologies (statistics, not A.I.) to minimize bias as statistical algorithms score and sort through the open-ended responses. This process is significantly faster than manual reading and sorting of responses, making it the best solution to analyze open-ended responses to situations that require immediate action and objectivity.
4. Reach Conclusions
Once you have categorized your employees’ responses into themes, dive deeper into the data and segment it further to reach meaningful conclusions. For example, a number of your employees are saying that they are unhappy at work. Consider these questions:
Which types of employees are saying that (e.g. adjunct vs. tenured)?
Are there differences by gender, age, or years of experience?
What department are these employees from?
In one of our open-ended response analysis for a liberal arts college, we found that employees had significant concerns around cross college relations. More significantly, we showed that the concerns were more prominent for among administrators than instructors. For example, a much greater percentage of administrators (25%) than instructors (5%) felt that cross college relations was affected by low compensation. This indicated to the college they may need to review the pay equity scale between administrators vs. instructors. Similarly, we found significant difference across segments such as the length of tenure, gender, and more. By staying mindful of unique populations on campus, their needs and their pain points, you may uncover unexpected results when you analyze your data. This will help you reach insightful conclusions and targeted solutions to your problems.
If you choose to work with Kai Analytics, we will develop a final report for you. We will provide you with commendations and recommendations that we reached, the rationale, supporting graphs and visuals. Our clients especially find the commendations useful for building morale and highlighting their successes on their website and social media platforms.
5. Implement Changes and Improve your Employee Morale
Once you have your key conclusions, communicate them to your employees and commit to acting on them. Develop and implement an action plan based on the results of your employee survey. Our previous clients’ organized task forces dedicated to overseeing these changes in the university and maintaining weekly communication, ensuring feedback and engagement. At this point, the more actionable changes you are able and willing to commit to your employees, the higher the buy-in will be. Making promises informed by what your employees want and sticking to them, will once again instill trust in your employees and will help you avoid and resolve disputes.
We wish you luck with your employee surveys and are looking forward to providing consulting services to those who need it. If you are interested in our services, please contact us by filling out the form below.