Anonymous surveys offer a multitude of benefits: Rewarding data, privacy for participants, and overall security. Participants can be fully honest and open, bringing their concerns to the attention of the organization’s leadership without fear of repercussion. But what happens when participants don’t believe their responses are fully anonymous? In a work environment where distrust and suspicion flourish, participants will doubt the good faith of the research. If there is no communication on how a survey is anonymous, many participants will be wary of whether their responses could be traced to them in any way, and they will almost certainly hold back negative critiques. This is especially common in scenarios such as employee surveys, where employees could perceive honest answers as endangering their livelihood and working relationships. Recently, we saw these concerns about survey anonymity in a YouTube video by Loe Whaley, and wanted to address them so that you can know what to look for in an anonymous survey.
#1: Segmentation and Masking for Department, Age, and Gender
It’s natural to ask questions when a survey collects specific information like age, department, and gender. Couldn’t that information betray the source of the response? Despite how alarming it might seem, if an organization is carrying out the survey correctly according to industry standards, the answer should be no.
Why would an employer need to know the department of a respondent? One department may have a particular problem that another department does not—information which could be completely lost in the aggregated responses. If many employees of the same gender and age are the only ones experiencing a problem, discrimination may be at work. Analyzing responses like this, called segmentation, could change lives.
This doesn’t mean fears around segmentation aren’t valid. If only a handful of individuals respond to a survey, anyone who knows them within that department could easily identify their response if it is segmented. Thankfully, there are industry standards to mitigate this risk: Any segment with less than a certain number of responses (10 at a minimum, though for some institutions the number can range to 15 or 20) will be masked in the segmented results. This means any results will only be included in the full aggregate results for the entire institution. While some nuance for that segment may be lost, it is worth it to protect the privacy of respondents.
#2: Asking for your name (Or ID # or email)
Asking for your name (or an ID number or email specific to you) in an anonymous survey is probably the area of biggest concern for participants, however even in this case it is possible to preserve anonymity:
1. If the form asking you for your name/ID is separate from the survey itself, the information may be collected to confirm that you have responded to the survey, or, if there is a survey incentive, to enter you into a draw. In such cases, the form where you enter your identifiable information should be completely separate from the survey with accompanying language explaining this. When you are redirected to the survey, the server does not store any information that would trace your response to the separate form that collected your information.
2. Sometimes, an anonymous survey has the option of not being anonymous should you choose to identify yourself. This is usually for recruiting volunteers for follow-up research or pulse surveys. Pulse surveys are brief questionnaires that check in with previous survey participants on how they feel about changes made in response to the main survey. This option should involve clear, explicit, and full consent with accompanying language explaining that by providing your information, you are waiving your anonymity.
If there is no language explaining that your personally identifiable information is not being collected in connection with your response or identifying an optional response where you waive your own anonymity, that is a cause for concern. Unless the survey is clearly labeled as neither anonymous nor confidential (a slightly different approach where only a select few can access identifiable survey data), participants should absolutely be asking for more transparency. The following examples show good communication in this kind of situation:
“If you wish to be provided with the study results please contact our researcher via the information at the end of the survey. This will not be linked to the questionnaire or data files in any way.”
“This questionnaire has been approved by [name of human research ethics committee]. For the detailed research purpose, description, potential risks, benefits, and confidentiality please click here.”
“Are you interested in participating in online research? These sessions vary from 45-60 minutes in length and can be accomplished remotely. Participants will receive a $25 gift card for their participation. By selecting “yes," your responses will be associated with your contact information so we can follow up with research questions.”
Concern #3: Individualized Links
If you’ve ever taken an employee survey, you’re probably familiar with accessing it through an email, along with a notice asking you not to forward it to anyone. Does this mean your individualized link is connected to your survey response?
If the survey is following industry standards, the answer should be no. Individualized links can show the online survey tool whether a participant has clicked the link but can’t determine what the participant did afterwards. Individualized links help with follow-up reminders, so participants who have already completed the survey aren’t overrun by unnecessary reminder emails. Of course, this method can't determine whether you clicked the link and immediately closed the window, but it provides at least some idea as to which participants have most likely not completed or even looked at the survey.
Participants aren’t supposed to forward the link to anyone else because if a different participant uses someone else’s link to complete the survey, they will continue to get follow-up reminders for the rest of the survey period. As some participants don’t enjoy taking surveys and may consider them a nuisance, receiving unnecessary emails does not encourage future participation. On a more obvious note, you shouldn’t forward survey links to anyone outside of your organization at the unlikely but existent risk that an unrelated person will participate, adding an erroneous response to the results.
Concern #4: IP Tracking
This concern may be slightly more niche as many individuals don’t think of IP tracking on a regular basis, but IP tracking can affect survey anonymity. Some online survey tools can identify the approximate geographic location of a respondent based on where they are accessing the website from. If everyone on your team works in Canada, but you’re the only person working in a resort in Bali, the survey tool may collect that data, identifying you. Make sure that the survey instrument is not tracking IP, GeoData or other web analytics such as Google Analytics.
There are legitimate research requirements for IP tracking such as preventing duplicate responses to a survey (e.g., in the case of raffles) or needing to verify the location of the responses for geo-sensitive research. But these should be made clear at the beginning of the survey for consent and/or accompanied by a letter from the human ethics or institutional review board.
All online survey tools have information about how IP addresses are tracked, and if you have a reason to be worried your location will give away your identity, we recommend reading up on the survey tool’s policies and asking questions accordingly.
Concern #5: Indigenous Sovereignty
We mentioned issues surrounding segmentation in our very first point, however there are broader concerns related to segmentation when it comes to race and ethnicity, concerns that reach into open-ended questions and every part of the survey. While each minority group has unique concerns, as a Canadian company we are closely acquainted with the concerns of Indigenous groups. In certain regions, segmenting responses by race/ethnicity could unintentionally lead to identifying which specific Indigenous group, band, or nation a response came from, not only risking the privacy of individuals but the privacy of their community.
In both the past and present, researchers have harmed Indigenous communities by withholding healthcare, disrespecting traditions, perpetuating harmful stereotypes, and other grievous injustices. It is the responsibility of both organizations and survey vendors to acquaint themselves with these concerns. Our senior analyst, Kevin Chang, took the First Nations Information Governance Centre’s Ownership, Control, Access, and Possession (OCAP) course to better understand these risks and how to address them, and we highly recommend it. There is no one-size-fits-all solution for Indigenous sovereignty, and it’s important for those involved in survey implementation to continuously educate themselves and listen to Indigenous peoples to protect the sovereignty and privacy of both Indigenous individuals and their communities.
Our Final Thoughts on Survey Anonymity
For anonymous surveys to be successful, it is crucial that organizations communicate to participants how and why it is anonymous. The participant’s job is simple—deliver honest responses knowing that the survey is following important industry standards such as masking groups with less than 15 responses, aggregating data, and scrubbing personably identifiable information (PII) mentioned in open-ended comments. It is never wrong to ask questions and explore how anonymous surveys work. For every survey Kai Analytics has been involved with, we have remained available to answer these questions and more throughout the process, helping put participants minds at ease.
If you’re interested in carrying out an anonymous survey, Kai Analytics can handle every stage of the process while paying attention to important details that preserve anonymity, as well as answer tough participant questions promptly and effectively. Book a 30-minute consultation with us today to find out how we can help you make your anonymous surveys more effective.