Life is full of natural phenomena: height, shoe size, and people. And some that are but one step removed: SAT test scores, bond investment return, employee performance and rum reviews. What's common to all these is how the results are distributed...
You need to know two things: first, that the larger the sample ("n") the smoother the "bell" becomes, and second, that the "bell curve" represents what is called a normal distribution. This distribution is both common and expected. Deviations from it are suspect and represent abnormal influences (like rum score bias, for example. Now let's look at a typical bell curve:
You'll note some interesting things. In a typical normal distribution you'll find what is called the 68-95-99 Rule. Simple. This means that the normal distribution will have 68% of the results within one measure called the standard deviation. And 95% withing two measures, and 99% within three measures.
Let's consider a Five Star rum rating system. A fair and normal distribution of reviews would be something like:
1 star: 2% of rums
2 star: 14%
3 star: 68%
4 star: 14%
5 star: 2%
Mind you, things are not so perfect in real life, especially with smaller samples, but even so the distribution will follow this pattern.
And yes, yes I can hear it now, especially from the reviewers whose distributions were abnormal and biased (funny, but all of them were biased to to higher scores, hmm). "But I chose better rums in the first place!". This, of course, is horseshit. The facts:
It doesn't matter.
Whether you picked what you considered just great rums, or just poor rums, the distribution of scores made by the same reviewer will still be normal. It's like playing darts: it doesn't matter whether you aim at the center or at the perimeter - all your shots will exhibit a normal distribution around your aiming point. Of the rums reviewed, most should get your median score, with fewer and fewer getting higher or lower scores.
It's the way it is, live with it. All the Reviewer's Reviews were carefully analyzed and honestly reported with an eye toward revealing possible bias. To get our high score a reviewer had to show a few things...
First, a normal distribution of scores. Those whose scores were biased to the high end lost points. Second, we have a short list of world class rums of the five basic styles that have garnered widespread acclaim over many years, and via many venues. Though their scores may differ slightly, we expected to find some recognition of the harmony, complexity, balance and presentation of these great rums.
Last we took particular note of which rums scored at the reviewer's extremes, as these were particulary revealing of both reviewing skill and/or bias. Nonetheless, a normal distribution was basic and primary to our scoring. All factors were reported and our own scores were applied to the reviewer. Not surprisingly...
A normal distribution, lol...
