Recently, we wrote a recap report at the mid-point of a program. One of the things we noticed was that the survey responses were significantly disproportionate to the engagements by event type. To ensure the results were representative (i.e., not over- or under-representing respondents from one type of event), we applied a weighting scheme.
Introduction to Weighting Data
Weighting data is a common practice in survey research. It allows more accurate generalization of your findings compared to the greater population.
Survey research draws information from a sample, or subset, selected from a larger group (e.g., the entire population). It’s important that your sample reflects the general population so you can draw accurate conclusions that apply to that population.
Weighting Data Case Study
Using the program mentioned above as an example, we conducted surveys at four types of events:
- Large, planned events
- Medium-to-small, planned events
- Unplanned, or spontaneous, events
- Community-based events
Our preliminary analysis showed that there were statistically significant differences in consumer demographics and attitudes at the various events.
However, the impact of these differences was not clear, because the majority of our responses came from events with the lowest engagement rates.
In essence, our data over-represented these consumers.
To weight data, calculate and apply a multiplier to your data. The multiplier assigns a higher or lower value to responses, correcting the disproportionate effects.
Weighting data is a powerful technique, when used properly.
Best practices require knowing the exact parameters of your population and being able to establish a modest correlation between the weighting variable and other key variables. (A weak or non-existent correlation increases sampling error.)
Weighting data is not always required, especially in the work we do, since we almost always survey consumers who have attended the events we are measuring (typically referred to as a convenience sample).
However, weighting data is a method that we use to help us ensure the quality and integrity of our analysis.
Photo Source: https://www.flickr.com/photos/61056899@N06/