The response that nobody asked for.
Having recently read 2 blogposts on the topic, this qualifies the topic of “Grading Whiskies Out Of 100” to now be styled as “a debate”* – so I thought I’d add my own slant on what I’ve read so far… irrelevant as it may be, because personally I don’t tend to review whiskies or give them scores. As a CONSUMER of reviews though I do have some thoughts.
Felipe’s hypotheses are that as whisky tasting is a transient and varied experience, and therefore no person could reliably, repeatedly assess a whisky to the fine level of detail that a 100-point scale demands. And also that tasting something in a non-blind-tasting scenario is likely going to influence your score based on what you generally think about that distillery to start with. He therefore advocated a 20-point scale with half-marks as sufficiently granular (therefore, a 40 point scale).
Mr MacRaild responded firstly by making the very important point that whisky scoring is primarily about communication, and that a number in isolation is more or less meaningless/worthless** without an accompanying tasting note – but that the 100-point scale allows for more expressivity and nuance than coarser gradients. Through using a “benchmark” you know well you should be able to provide some consistency in your scores, and the real key is providing a decent tasting note alongside the number.
Angus also refers to Serge Valentin’s excellent Whiskyfun site & reviews, and in considering this topic one of my immediate points was that Whiskyfun publishes not only a 100 point score, but also an SGP (Sweet/Grainy/Phenolic) number, which is an attempt at codifying what a whisky will taste like. There’s also the star-rating, which I’m totally buggered if I can make sense of, but anyway…
Similarly he refers to the work of one James Murray*** – a chap known widely for publishing his own bible, the only copy of which in my house is from so long ago it was before he started putting pictures of himself looking like a Dragonlance Wizard on the front – and how his reviews are arbitrary and without context†. I’d also like to add that I find his scoring system problematic, which I’ll come on to later.
So, fundamentally in my mind the question of “the best scoring system for whisky” has 2 prongs to it:
1 – Standardisation
2 – Top weighting
By “standardisation” I suppose I’m talking about what you think the numbers mean. An easy first example of this is to look at the very famous Robert Parker “Parker Points ®” wine scoring scale.
So, if we take that as the baseline for a 100 point scoring system then there’s one immediately obvious problem, in that you’re only using half of the numbers. Parker says that anything less than 60 is basically undrinkable, and therefore the range of 10 marks in the 50-section are just to indicate whether it’s merely shit, or absolute-mega-fully-accessorised-copper-bottomed-ocean-going-uber-shite.
Pioneering whisky author Michael Jackson operated on a very similar 50-point 100-point scale.
A look at Serge’s score histogram from his FAQ demonstrates that indeed the numbers below 50 in this scoring scale are more or less worthless.
There is the occasional amusing moment to be had below 50, such as Johannes van der Heuvel’s (in)famous MaltMadness reviews of Loch Dhu where he gave the 70cl bottle 11/100 and the 20cl bottle 14, because at least with the smaller bottle there was “less of it to hate”.
As a qualitative measure though, below 50 is worthless – and that’s what bothers me about Angus’ assertion that the 100 point scale is more expressive. But then, I’m also bothered by Whisky Magazine’s whisky reviews (for example, issue 153 on the snazzy new site) – where they grade on a 10 point scale, but give point scores so that it’s a 10-point scale with 100 graduations.
But the essence of Parker’s scale is that for anything you’d consider buying you’ve basically got 30 points of graduation, or if you’ve been gifted something there’s a further 10 points below that to consider whether you can get away with regifting the bottle or leaving it at a party, or whether simply pouring it down the sink is the kindest thing to do.
Mr Murray offers his guidance, thus:
So straight away I’ve got questions…††
– this system incorporates half-marks, so it’s a 200-point scale. Given Felipe’s assertion that 100 points is too fine for the human palate to cope with, how can we expect to believe someone can graudate on a 200 point system?
– the assertion here is that nothing will ever score 98 or above, so it’s actually a 195 point scale.
– like Parker (although different enough to avoid plagiarising) it looks like in reality anything below 65 won’t likely crop up, and represents a hard pass anyway. So that leaves an available useable scale (65-97.5) of 66 points. Makes sense, I guess?
Murray breaks the score up into 25% for the nose, 25% for the palate, 25% for the finish and the remaining 25% for the “overall balance”, meaning that 75% of the score comes from the taste.
Serge doesn’t apportion marks and, as far as I’m aware, has only scored one whisky with 99 points. I guess you’ve always got to leave yourself *somewhere* to go.
In writing this I’ve stumbled on some interesting stuff about statistical analysis of scoring, and subsequent discussion of how to aggregate different scorers’ figures to arrive at consensus – although I’m not sure that’s necessary either.
It probably comes back to what your reason for scoring whiskies is in the first place. I feel like if I was going to score whiskies it’d be for some sort of “how much did I like this?” personal record, which I think Phil Storry quite well-covers/uses on his blog, having a 1-5 scale with half-marks, giving him a 9-point range.
The Old Man of Huy pondered this issue back in 2014, and after evaluating several systems, proffered this useful scale:
One of my favourite scoring systems is from thoroughly nice chap Mr Ben Cops – who makes no apologies for doing it for his own record/benefit, and admitting that it’s evolved as time’s gone on, and is currently scoring between D grade and A⊕⊕’.
I *think* this puts him at 15 graduations, plus the ‘ modifier which he can use to distinguish between 2 identically scored whiskies that turned up in the same session… I absolutely love it. Plus, as far as I can tell due to the search tools on the blogging platform there’s no way to retrieve reviews having a certain score. Keeping it real.
So, moving onto the the almost-forgotten “second prong” of the discussion (although we’ve surely touched on it by now) – that of “top weighting”.
I think the examples til now prove that people using a lower-graduated system don’t mind giving things 5/5 or whatever, because their system allows for a whole bunch of things to be equally good. The more detailed crew however seem to almost universally shy away from using the top marks – because to do so would represent “the finest whisky I will ever taste”, and that’s not information you have access to. The best you can hope to know about is “the best whisky I’ve ever tasted so far”, and by definition you need to leave somewhere to go. But then this means that those graduations between around 93 and 99 mean there’s so much more at stake. The difference between an 94 and a 95 can’t possibly be the same as the difference between a 77 and a 78.
The good news is that people have been arguing about this for years (such as this discussion on the Malt Maniacs site, which can’t be very recent) (and the people of Reddit inevitably had a go), and will likely continue to do so (here’s Dave Broom pondering on the same topic upon the launch of scotchwhisky.com).
Billy over on Spirited Matters doesn’t use ratings at all: he spends a good deal of time describing the experience he’s having and then reflects later on whether it was good or not. Sjoerd at Malt Fascination does score out of 100, although it’s frigging difficult to get a score over 91 out of him and as far as I can tell he’s only scored from 60 to 94 to date. But both of those blogs I read for the verbiage & enjoyment of the writing, not really for the opinions about whisky.
Initially I had aspirations of describing a far superior rating system – however in writing this the key point has become apparent in that really the value of a system kinda depends on what its purpose is. And the final summary really takes us back to Angus’ initial point (and which echoes Serge’s thoughts in that Malt Maniacs discussion), which is that a score in isolation is more or less meaningless. To be of value it should be the summing-up of a review.
Even if we were all hypothetically to agree to adopt the 100-point scale, as we can see from above, there’s then still the matter of all agreeing on how to best use it. And my feeling is that if you tried to arrange for all of the UK’s whisky bloggers to meet in a pub to discuss it you’d never get them to agree on what pub to go to in the first place, let alone get near a conclusion.
THE END (see new added Conclusion below)
* although traditionally a debate is a formal structure involving speakers for the affirmative and negative of a specific assertion, and merely having a load of people shouting what they think endlessly in each others’ vicinity doesn’t really follow that form, nonetheless I felt by wading in it would enable bloggers with nothing to really add to the discussion a chance to refer to the discussion as “a debate”.
** depending, of course, on what your motivation for making use of that information is – for instance, bottles that sell out immediately after being scored in the mid-90s on Serge’s blog
*** I’ve not met Mr Murray personally, but from what I’ve heard anecdotally he sounds a bit of a prick, and he once tried to get another whisky pal of mine fired for having the temerity to eat a sandwich within 50 feet of him.
† Without turning this into JM-bashing, I don’t think there’s any denying that publishing a tasting note like this is of more or less no value to anyone.
†† Also, why does he denote a revised tasting note by drawing boobs next to it?
Addendum/Conclusion (not often you see one of those round here…):
Thinking about this overnight, I couldn’t get past the silliness of the 100 point system as it is, because supposing you accepted 60 as the minimum drinkable score, that gives you 60 graduations of detailing just how awful a whisky is below that, but if it’s superlative then even using half-marks you’ve only really got 4-8 gradients.
Therefore, we can reasonably conclude that in practice a 100-point scale is in reality a 40 point scale.
Now, if Felipe could concede that his 20 point scale – assuming half marks – is actually a 40 point scale, would grade 0 as terrible, 10 as average, 20 as the upper-end of good, 30 as excellent and 40 as perfection, then we can fairly quickly see that the 100 point scale is in effect a 40 point scale that you start by adding 60 to to get started.
And therefore everybody’s talking the same language.
Except Ben. Don’t you go changin’ man.
