About 36 years ago, after putting it off as long as possible, I entered Professor Keston’s classroom to begin a year of required statistics. A year! Tried to sit in back but those seats were already taken by other psych-major-math-phobes. With dread in my heart, I took a seat near the front. By the time I successfully passed the final I was forever changed.
Dr. Morton J. Keston taught many lessons that come in handy in my business and personal lives. The greatest is you don’t have to be good with math to be very, very good with statistics. As personally valuable as that lesson has been, it’s the others that are applicable here. Those lessons are: Be curious. Consider the source, possible spin, and our own bias.
Consider the Source
Considering the source includes whether the pool of data is large enough. For example, we shouldn’t let results of five or ten or even fifty calls or emails sway us one way or the other—we need more to have a valid data pool.
We should also consider the make-up of the pool. For example, (since we’re about to be bombarded with political polls) we should wonder how many probable voters weighed in and whether party affiliation was appropriately mixed. Alas, this information is hardly ever included with poll results.
Double-alas…it’s common that two thousand or fewer were polled. Is that valid considering there are around 146 million registered voters in the U.S.? The answer can depend on your personal viewpoint, which brings us to another of Keston’s lessons: There’s more than one way to position “results.”
Consider the Spin
All those years ago, I and my classmates learned to critically analyze how studies, experiments and tests were shaped and their results presented. I send a silent thanks to Professor Keston every time I apply those skills in training assessment or in analyzing sales performance. However, in everyday life, it’s more about how results are presented.
For example, an author spun the results of a 1967 experiment having little to do with bodies into a book on “body language.” In another example (back to politics), news programs often put the preferred statistic where it’s more likely to stand out. You can spot this by watching competing news channels like Fox and MSNBC as they cover the same poll.
Not that looking at competing news coverage is likely to make a personal difference. It’s quite common to believe polls that appear to agree with our viewpoint and disbelieve others, which brings us to bias.
Dealing With Our Own Bias
It’s human to see what we’d like or expect to see. The best way to deal with this bias is to apply curiosity. Did the form of measurement fit what it was supposedly measuring? Was the data pool large enough, and broad or narrow enough? Does the interpretation of results make sense? By applying curiosity we tend to be more aware of our perspective–our own spin–and take that into account.
Now to Busting the Myth
The statistic mentioned at the beginning of this article is often presented like this: Buyers have completed 67% of the buying process (or “journey” for those who’ve been watching too much of The Bachelor) before a sales rep is involved. Many people find that to be an impressive statistic. We, however, are curious about where it came from.
You have to do a lot of digging to find the original source. Once you do, here are the points Professor Keston would hope you’d spot:
- There was just one study. (For extra credit, this means we don’t know if the study or results are statistically reliable.) Fewer than 100 “executives” were questioned. Is that a valid pool? Nope. Do these subjects match your target market? Not necessarily.
- The focus of the study was on how they conduct research as part of their buying decision. Does this have solid correlation to how much or when a salesperson is or is not involved? Nope.
- Astoundingly, 67% of the people in the study did most of their research via the internet.
What’s that? You don’t find that last point astounding? Well, most people wouldn’t and so the study’s sponsors—a company with a vested interest in web-based and social media marketing—positioned its results to sound more impressive. So there you go.