Data collection audit: What are we already collecting?
Written by Jessica Gibbons-Benton
A huge part of what we do at Elevate is working with data. We help folks think about why they want to collect data, what questions they hope that data will answer, design methods to capture those answers and implement a plan for how to use the results. However, step number one is always to ask, “What data are you already collecting?”
You might be surprised at what great sources of data you already have! Whether data collection is being implemented formally or informally, odds are you are paying attention to the impact of your work as well as trends in the work more broadly.
Let’s briefly go over some different approaches to collecting data:
Two Approaches to Collecting Data
Primary Data:
Data collected directly from individuals for your specific purpose
Captures knowledge of stakeholders within the system or program
Secondary Data:
Collected by a different source for a different purpose
Often for understanding system-level trends and patterns
Two distinct approaches to collecting data are using either primary or secondary data sources. Primary data collection is data your organization is collecting firsthand. This data comes directly from your stakeholders and is related to your program, intervention, or work specifically. Secondary data comes from sources that are often much broader in reach, like national or statewide data sources. Secondary data can also be smaller in scale but related to your work topically or theoretically, like a case study from another organization.
Examples of Data Collection Methods or Sources
Qualitative
Primary Data
Interviews
Focus Groups
Listening Sessions
Secondary Data
Policy Analysis
Reports by third parties
Journalistic pieces/news articles
Quantitative
Primary Data
Surveys
Intake forms
Other program documents
Secondary Data
Census data
Administrative data (e.g., MNPS data)
Both primary and secondary data can be qualitative or quantitative. Qualitative data is non-numeric and is designed to capture the experiences of individuals or groups. These methods are generally inductive, meaning we develop the theory/hypothesis from the information we gather rather than trying to test a known theory/hypothesis. Quantitative data refers to data that is numeric. Quantitative methods are usually deductive, meaning you start with a theory and then test it. Some examples of data collection that you may be already doing include collecting information about participants from registration, intake, or other program forms. Or, you may collect quotes or anecdotes during events, in meeting notes, or debrief sessions.
Sometimes data collection happens in silos within an organization, where each department or program is doing great work to collect the data they need. However, this can mean that there might be data collection going on that other departments, programs, or leadership are unaware of. By creating a more complete picture of the data being collected organization-wide, you can have a better idea of what questions you are currently able to answer versus where gaps in information may exist.
To conduct a data audit, we suggest talking to staff about the different approaches and methods of data collection mentioned above. Then, work as a team to fill in a data grid like the example provided below. Once you have completed the data collection grid or a data map of your choosing, you'll be ready to start building out a more intentional evaluation or data collection plan–and we would love to support you in that!
Data Collection Grid Worksheet
Tool/Method - How is the information collected?
Outcomes Measured - What information is captured?
Participants - Who is the information about?
Frequency - How often is the information collected?
Required (Y/N) - Are we required to collect this information?
Data Collector - Who administers the data collection?
