Survey Demographic FAQs
When designing survey tools with our clients, we often receive many questions surrounding the demographic portion of the questionnaire. Demographic questions can be tricky for a number of reasons, and there is no real consensus around best practices. In this post, we will share Elevate’s answers to some demographic FAQs that are backed by literature and our own experience in consulting.
General Advice:
When should we ask demographic questions?
Identities can be sensitive and personal! For many, being asked to disclose identifying information can bring up questions of intention and safety (which I will address next). While this information can be used in ways that are harmful, we also know that it can be essential in identifying and addressing disparities. For instance, we suggest asking certain demographic questions to see whether programs are reaching individuals in a representative way and if there are trends in differences in impact along demographic categories. This data can be used to directly inform action, whether that be expanding a program or service’s reach, or modifying content to better serve particular groups. If you plan to use the information to monitor disparities and directly inform action, we recommend including demographic questions. In some cases, funders also require demographic data collection for reporting. If you are collecting demographics for this reason only, then we recommend limiting the questions to only those required by funders.
Another consideration is when in your survey to ask these questions. It is common to see demographic questions clumped together in one survey section, but this is not the only way to do it. For instance, if you are curious about someone’s housing status and you have a section of your survey about housing services or preferences, you may include it there. Use your best judgment in terms of what will flow logically for participants. If you do choose to dedicate a section of your survey to demographics, it is common to include this section at the beginning or end of the survey, and there are pros and cons to both placements. First, consider importance based on the actions you will take with demographic data. How essential is it that you collect this information from all participants? If your answer is “very!” you should include this section at the beginning of the survey before the rest of your content. However, if the body of the survey is your priority, you may be presenting participants with barriers up front by including identity questions at the start. In order to not deter participants, you can include demographics at the end of the survey. This way, you will still have survey data for individuals who choose not to answer these questions but have an increased likelihood that participants will skip the demographics
What about informed consent and safety?
Collecting consent from participants and creating a safe survey atmosphere are important ethical considerations in survey design. In order to address them, we recommend including what we call “leader language.” This is a brief section of text before the demographic questions that include the following:
Tell participants why you are asking for their demographic information, including how it will be used. Describe any decisions or actions that will be taken based on the data.
Tell participants how you will protect individual data. This includes communicating about anonymity or confidentiality, who will have access to the data, how it will be stored, and to who it will ultimately be reported.
Finally, we recommend you make the questions optional and remind participants that the survey is voluntary, meaning they are not required to participate and they may exit the survey at any time. When applicable, specify that whether folks complete the survey or decline will not impact their services.
In the end, you will have something like this:
Elevate recognizes that disparities exist in access to our services. In order to better understand and address these disparities, we would like to collect some information about your identity. Your information will be kept confidential and reported in the aggregate. Your participation in this portion of the survey is voluntary and you may skip any questions you do not wish to answer. Access and quality of services you receive from Elevate will not be impacted by how you choose to respond.
What demographics should we include?
The rule of thumb we share with clients and follow ourselves is to only include what you plan to use. While it may seem like the most thorough and exhaustive list of demographics is the best choice, this is typically not the case. For reasons we mentioned above, demographic questions can create barriers to participants so it is best to keep them short. We encourage you to spend time on the front end considering how you plan to use this information, if at all.
For example, some commonly collected demographics include language spoken at home, ability-related questions, and sexuality. If you plan to expand your service offerings into multiple languages or implement changes to accommodate differing abilities, then it is appropriate to ask about these things. If you have no intention or capacity to take action based on your findings, then these questions probably do not need to be included. Sexuality is a question that we do not often include because it is not often relevant or actionable and it can be particularly sensitive. It is important to think critically about which demographic questions will lead to action.
When should we follow the census standards?
The census survey standards are not our favorite, and in fact, there are many in the fields of research and evaluation who disagree with the ways these questions are asked. However, the census is a huge database, it is free to access, and it is the most complete and accurate way to paint a demographic picture of a given area. For those reasons, we do find ourselves using them when:
trying to make a direct comparison to a general population.
trying to set or fill quotas based on a general population.
there is a requirement from a funder to have comparable data.
Question-wording:
Race and Ethnicity
It is important to recognize that race and ethnicity are actually different categories though they are often lumped together (this can be confusing). Race is a social construct that categorizes individuals by their physical attributes. In the US, these categories are mainly tied to skin color: Black, white, brown, etc. Ethnicity has to do with an individual's affiliation with a larger social group of origin such as religious, cultural, tribal, or national groups.
Oftentimes, these two categories are lumped together into one question rather than asked separately. The way we prefer to do so is shown below. We always recommend making these questions optional, check all that apply (to include people who are of mixed race or who identify with multiple ethnicities), and including the option to write in and self-describe. Including “mixed-race” as an answer choice does not pick up on important levels of detail and can perpetuate gaps in data that do not help in addressing existing disparities.
Which of the following best describes your race or ethnicity? Check all that apply.
American Indian or Alaska Native
Asian
Black or African American
Hispanic or Latino/a/e
Native Hawaiian or Other Pacific Islander
white
A race, ethnicity, or origin that is not listed: [text box]
Prefer Not to Answer
It is important to note that these questions can and should be altered based on your targeted population. For example, if you were reaching out to refugee families in Nashville the above categories would not be appropriate. You might instead ask for the country of origin or country the individual moved from. In other cases, you may want more granular information about a certain group. Hispanic or Latino (here we included o/a/e to be gender inclusive) are catch-all terms for groups of people from many countries with many different racial identities. For instance, an individual can be Mexican and identify as Hispanic and white, or black, or indigenous or mixed-race because Mexico is a country of origin with a diverse racial makeup.
Because race and ethnicity are so nuanced, there is no perfect way to capture them. This is also why we use the language “which best describes”. However, letting people share with us how they self-identify is our current best practice.
Gender
Similar to race and ethnicity, gender is also socially constructed and nuanced. Here we also use an incomplete list but encourage participants to self-identify if their preferred gender is not listed. It is also important to allow individuals to check all that apply. If you are interested in capturing the identity of transgender participants, we suggest asking this in two parts as shown below.
Which of the following best describes your gender? Check all that apply.
Female
Male
Non-binary
Prefer to self-describe (please specify): [text box]
Prefer Not to Answer
IF you want to capture trans people specifically, you can add a follow-up:
Do you identify as transgender?
Yes
No
Prefer Not to Answer
Should we add an “Other” option?
We strongly discourage clients from using “other” as an option when capturing demographic information. This is because we know that language matters. The word “other” can make participants feel othered or as if their identity exists outside of what is considered the norm. Using the language “prefer to self describe” or “is not listed” along with an open-ended text box invites participants to be specific about their identity rather than lumping them into a single label that exists outside of those recognized.
Another important related note is to include a prefer not to answer option. This is mostly necessary in cases where you are requiring an answer to the question in the survey programming. This essentially gives participants the option to skip the question. However, it doesn’t hurt to include it generally to make participants feel like they do have the option to keep their identity private.
Additionally, we recommend listing the answer choices in alphabetical order. It is common to see Male listed before females and white listed before the rest of the answer choices. This ordering implies a hierarchy while an alphabetical ordering is neutral.
We hope that this blog post and the resources included below give you some things to think critically about as your design demographic questionnaires. We encourage you to keep up with new standards and to stay current with best practices. Please feel free to reach out if you have other survey design or demographics questions.
Thanks for reading!
Sources/Resources:
https://www.ygender.org.au/article/how-to-ask-about-gender-on-surveys-and-forms
https://www.vanderbilt.edu/lgbtqi/resources/how-to-ask-about-sexuality-gender
https://www.ohsu.edu/sites/default/files/2021-03/OHSU%20Inclusive%20Language%20Guide_031521.pdf
Designing Quality Survey Questions by Robinson, Sheila B. & Leonard, Kimberly Firth: https://www.abebooks.com/Designing-Quality-Survey-Questions-Robinson-Sheila/22902834220/bd?cm_mmc=ggl-_-US_Shopp_Textbook-_-product_id=COM9781506330549NEW-_-keyword=&gclid=CjwKCAjwzeqVBhAoEiwAOrEmzSZsM5eXlwSb-oLVZO9HC7G8iqmhSGMfkxf1hlYAHtuSqynK92iFhBoCDE0QAvD_BwE