We’ve been using a data analysis process for quite some time here at PortMA, and I think it’s worth sharing how we do it and why we think it’s valuable. This is a way in which we assure quality because here’s the secret, people make mistakes.
Estimated reading time: 4 minutes
People make mistakes when it comes to numbers, 17’s become 71’s when they get transcribed from a spreadsheet into a report. While that is always going to happen, it is absolutely unacceptable for that to find its way into the final version.
We have to have tools and methods to prevent mistakes, and it all boils down to replication – doing the same thing twice and comparing it. Let me share a couple of examples of how that’s done and is a part of every process that we have here on this side.
Interpret the Data
First of all, when you’re coding something (when entering data), you have to apply an ID to the source material and you’ve got to double code it. What that means is that you have one person who’s entering information and attaching what they’ve entered to an ID. Then you have someone completely different entering the same information in the same ID.
This may seem like it’s unnecessary replication, but it’s the only way to assure quality because if there’s anything about those individual coders (the individual process) that is subjective, then they’re going to have a different interpretation.
You’re never going to get a person who has one perspective – no matter how good that analyst is – to be right 100% of the time.
When two people are interpreting things differently in the world of data, you’re going to have no value or variation in your resulting analysis. It’s not going to be because there isn’t something happening out there in the universe. It’s going to be because you have a data anomaly; you have a problem with the way the data was collected.
So you have two people do the same thing and then you use that common ID to merge what they’ve done. You compare the results, and when the results are not the same, then you investigate it and you figure out. “Okay, why is that different and how do we correct it?” We do that in everything we do, anytime there’s any sort of manual coding process.
(You can listen to the full podcast episode below.)
Replicate the Analysis
The other piece has to do with the production of information based on data. There’s actually a tradeoff that has to happen between data analysts in order for you to assure quality.
Two Rules to Keep in Mind for Analytics
- The person who’s doing analysis and populating a report can’t be the same person who’s checking the quality of that report. If somebody is completing an analysis and writing a report based on that data, then the quality assurance step has to be another analyst of equal skill who can go to the raw source data, repeat the analysis, and then check the numbers off of that.
- That’s doing two things, it’s catching the typos, the 17’s that became 71’s, but it’s also checking the logic, checking the thinking. It’s making sure why something was done, and checking that the way it was done makes sense.
Deciding to Go Left or Right in Data Analysis
In data science, there are often decisions that have to be made where you’re choosing between two perfectly reasonable directions. In a broader context, one can be much better and much more informative than the other. You’re never going to get a person who has one perspective and one ID – no matter how good that analyst is – to be right 100% of the time. They’ve always got to replicate it, which is true of science in general.
Replication is everything in science. Having your methods be clear and laid out in such a way that someone else who understands the academics of it can do the replication is what the world of science is based on. As marketers and as data scientists working in marketing, there really is no difference.
In the next post, I really want to dive into the idea of the Message-to-Market Match – which is such a fundamental theory in marketing – and talk about how that applies to the way we design experiential marketing campaigns.
Further Reading
Additional Resources
FOR EXPERIENTIAL MARKETERS
- Experiential Measurement Blueprint
- Event Impression Calculator
- Experiential ROI Benchmarking Reports
- Event Measurement Video Tutorials