A quickie! And don't you love quickies? Sure ya do. Normally I wouldn't waste my time on reviewing a whisky site (hint: this is a rum site), but one of our valued members posted a mild defense of Canadian Whiskey on a thread names "Canadian Whiskey: The Spam of Spirits".
Now he didn't counter the premise: that Canadian Rye Whiskey is generally a blend based on thin grain spirits, often doesn't contain rye, is bulked up with a bit of pot stilled or other whiskies, and up to 10% of damn near any additives you can think of. However tasty, it's a mixed drink in a bottle, lol.
OTOH he did encourage me to give Canadian Whiskey yet another try, even suggested two based on a couple of brief but very reviews from a website called the "Whiskey Advocate". And blog. Upon reading these little tidbits, the hairs on my neck immediately arose.
Why?
Simple. As a former copy writer/marketing manager I tend to sense shilling within a descriptor or three. And if it reviews like a duck... but to be fair, let's do an analysis and see! First let's look at the Advocates' rating scheme:
And the envelope please...95-100
A classic! All components are balanced appropriately, with the complexity and character expected in a classic.
90-94
Outstanding! One of the best for its style. Distinctive.
80-89
Good to very good. Plenty of character and no identifiable flaws. Worth seeking out.
70-79
Average. No unique qualities. Flaws possible.
60-69
Below average. Major flaws. Avoid.
. . . . .
Wow! Talk about a high score bias! Almost all the scores are well above average, with way, way too many spirits earning scores in the coveted 90-100 range.
Let's go on...
This is so far from a normal distribution's bell curve it isn't remotely funny. In a normal distribution the lowest and highest ranges should be roughly equal. Here they are 2 vs 401 whiskies. Now that's distributive bias. And the middle two ranges should be likewise, here 97 vs 758 whiskies. Actually, the largest range here should have been 70-79 ("average" whiskies), not 80-89.
The imbalance should be clear.
You'll also note these:
1. The "reviewers" (read copywriters) have scored 1258 whiskies. That's a ton. It is claimed that many other whiskies were sampled but not reported.
2. Each "review" conveniently publishes a link to obtain - are you sitting down? - to print out a "shelf talker". I'm serious. Can it get any clearer? No way!
3. The website states "Every attempt will be made to procure products that are in good condition and unadulterated. To insure this, distillers, importers, and wholesalers are encouraged to submit samples for review". In other words, freebies?
4. The site states that products are rated in comparison of whiskies of the same style. Thus a blended Canadian whiskey can obtain the same 95 as a 40 year old Islay. To me, this seems yet another sop to the trade.
So far, not so good.
But is there a counter to all this?
Maybe. First of all, the site claims some really impressive reviewers:
(Emphasis added)Due to the increasing amount of new whisky releases worldwide, Whisky Advocate has expanded its number of Buying Guide reviewers to five people: Dave Broom, Dominic Roskrow, Lew Bryson, Gavin Smith, and John Hansell. Everyone on this list is an authority in the whisky world, an experienced whisky reviewer, published author, and veteran writer for Whisky Advocate, formerly Malt Advocate.
The mere fact that Dave Broom appears on this list gave me pause. I've followed his reviews for years - like Michael Jackson, he's calls 'em as he tastes 'em. His distributions are absolutely normal (see Reviewers' Reviews in the Scuttlebutt section). So why aren't the Advocate's scores more balanced?
Can it be because the site also claims "The tasting notes and ratings published are those of John Hansell, publisher and editor of Whisky Advocate magazine". Now we're gettin somewhere, read on...
The Advocate also makes a strange claim:
Aha! Let's deconstruct this. The Advocate, no doubt keenly aware of what appears to be a high scoring bias, may have anticipated posts like this one, and wants us to believe that there are so many, many, many, in fact "...too many products sampled to publish reviews on all of them" - so - they just publish the positive experiences.Please note that we sample far too many products to publish reviews on all of them. The products listed in our Buying Guide usually reflect our most positive experiences. However, we will also review products we don't particularly like if there is a point we are trying to make.
Are you buying that? Why not publish all of them, or at least the most available ones, and take up another few more measly megs on your provider's hard drive? Doesn't cost another provider's pretty penny, fairly easy to do, and - uh- just might be valuable to whisky drinkers. Really now. Hell, they did the sampling, that's the hard part! Sorry.
And perhaps realizing that someone like me also just might make that point, have they then reversed course to assure us that "...we will also review products we don't particularly like if there is a point to make"?! So out of 1258 reviews (and apparently many more simply not published), just how many crappy whiskies did they select and report "to make a point"??
Two. Yup, just fackin two! Is this reasonable? Or does this amount to just shilling style bullshit? You've been given both sides, you decide...
*******
Special Note: The Advocate was contacted by a member, and responded in defense. Page down about 4 replies to "Late Breaking News! The Advocate's Reply"! You won't believe it!