Data Collection Audit: What are we already collecting?

A huge part of what we do at Elevate is working with data. We help folks think about why they want to collect data, what questions they hope that data will answer, design methods to capture those answers and implement a plan for how to use the results. However, step number one is always to ask, “What data are you already collecting?”.

You might be surprised at what great sources of data you already have! Whether data collection is being implemented formally or informally, odds are you are paying attention to the impact of your work as well as trends in the work more broadly.

Let’s briefly go over some different approaches to collecting data:

Two Approaches to Collecting Data

Primary Data:

• Data collected directly from individuals for your specific purpose

• Captures knowledge of stakeholders within the system or program

Secondary Data:

• Collected by a different source for a different purpose

• Often for understanding system-level trends and patterns

Two distinct approaches to collecting data are using either primary or secondary data sources. Primary data collection is data your organization is collecting firsthand. This data comes directly from your stakeholders and is related to your program, intervention, or work specifically. Secondary data comes from sources that are often much broader in reach, like national or statewide data sources. Secondary data can also be smaller in scale but related to your work topically or theoretically, like a case study from another organization. 

Examples of Data Collection Methods or Sources 

Qualitative

Primary Data

• Interviews

• Focus Groups

• Listening Sessions

Secondary Data

• Policy Analysis

• Reports by third parties

• Journalistic pieces/news articles

Quantitative

Primary Data

• Surveys

• Intake forms

• Other program documents

Secondary Data

• Census data

• Administrative data (e.g., MNPS data)

Both primary and secondary data can be qualitative or quantitative. Qualitative data is non-numeric and is designed to capture the experiences of individuals or groups. These methods are generally inductive, meaning we develop the theory/hypothesis from the information we gather rather than trying to test a known theory/hypothesis. Quantitative data refers to data that is numeric. Quantitative methods are usually deductive, meaning you start with a theory and then test it. Some examples of data collection that you may be already doing include collecting information about participants from registration, intake, or other program forms. Or, you may collect quotes or anecdotes during events, in meeting notes, or debrief sessions. 

Sometimes data collection happens in silos within an organization, where each department or program is doing great work to collect the data they need. However, this can mean that there might be data collection going on that other departments, programs, or leadership are unaware of. By creating a more complete picture of the data being collected organization-wide, you can have a better idea of what questions you are currently able to answer versus where gaps in information may exist. 

To conduct a data audit, we suggest talking to staff about the different approaches and methods of data collection mentioned above. Then, work as a team to fill in a data grid like the example provided below. Once you have completed the data collection grid or a data map of your choosing, you'll be ready to start building out a more intentional evaluation or data collection plan–and we would love to support you in that!

Data Collection Grid Worksheet

Tool/Method - How is the information collected?

Outcomes Measured - What information is captured?

Participants - Who is the information about?

Frequency - How often is the information collected?

Required (Y/N) - Are we required to collect this information?

Data Collector - Who administers the data collection?