 |
| This is the first page of the article. We have withheld the author and the publication name, but you get the picture! |
Typically, this magazine provides well-written articles by industry experts, a healthy mix of executives and suppliers telling their stories, reporting on their experience and sharing their opinions. But one article in this issue raised serious questions for me.
The five-page piece summarized research that was positioned as a "benchmarking study". It was easy to read, broken out in relevant sections, providing readers with simple bullet points and solid advice.
Sound good? Yep. Then what am I complaining about? Buried in the story was a brief statement that the research was based on eight respondents. Yes, only eight associations participated in this study.
So when the article concludes that “half of participants stated that…”, they are saying that the answer is shared by just FOUR association respondents. So what? Does this really constitute a "benchmarking study"?
One might have thought this would be prefaced by an acknowledgement that only a few people participated, and that the conclusions may not be applicable to the rest of the field. There was no such disclaimer and the article outlines advice that would lead the reader to believe this applies to a much larger population. Whether the advice was sound or not is irrelevant. My question is should an industry association publish this as if it is meaningful research, rather than a very small snapshot of a diverse field?This is at a time when associations need to find and share research that has relevance -- both in the conclusions drawn,and in the number of participants on which those conclusions are based.
I don't fault the author or her company for getting this published. It's exposure; good for them. But it makes me question whether the publisher was just trying to fill space? With content marketing and the proliferation of "thought leadership" magazine articles, blogs and other online sources, I encourage all readers to take a step back and not take everything at face value. Question what you are reading (yes, including this blog and any other Greenfield publication).
If you don’t think what you’re reading is relevant, or you have concerns about the methodology, raise your hand and say something. Your organization depends on your critical analysis, and your opinion matters!
Do you have any thoughts on the research being published in our industry? Share them with your peers and colleagues by submitting your feedback below.
