The Value Score: 2.0 / by Jason Hambrey

Of all the features of my website, the most feedback I get is about my value score - it's a mathematical formulation of value based on the average cost for a given rating and a deviation from it - i.e., if, based on standard "market" values, a whisky rated 85 is worth, on average, $53, then if the whisky is cheaper than $53 dollars, it is of high value, and if it is more expensive - it is of low value. The statistical formulation is shown in a previous blog post here, if you want the details. Because of the interest (and importance) of a value score, I have added a page to the website describing the best value whiskies. Check it out!

The story of how it came about is simple - I decided to graph all my whiskies which I had rated according to price and value. What I found, surprisingly, is that there is a rough trend - higher scores, on average, came from whiskies that cost more. After carefully selecting 300 standard whiskies which I had rated, I came up with an "average" line. You can see what I mean in the graph below:

The value score has served well, and I enjoy the result: I only rate the whisky, I input the cost, and mathematics does the rest. However, it relies heavily upon assumptions (of which there are many) - how the average line is defined, what whiskies I consider "standard" to set the market value, and what standard deviation to use (I am an aerospace engineer, so please forgive the jargon if you are lost). The implications of each assumption is actually staggering- so it has taken me some time to digest the score methodology itself. However, given my data of 500 or so whisky reviews, I don't have enough data to let stats do all the work - so these assumptions are necessary.

There has been one outstanding issue with the score as is - I have found that higher rated whiskies are not quite highly rated enough. For instance, a Longmorn 18 year old, at a price of $140,. which I rated a 92, was a bottle I bought two of even though it would have a value score of 64. That being said, it's not a bad value score and $140 is a decent amount to spend.

This lead me to look at options to tweak value scores at the higher end of the scale - by increasing the "average" line of what a whisky is worth for higher scores, or by changing the standard deviation for higher scores - meaning that a greater difference in price from the average cost of such a whisky matters less. But, as I said before - is this valid? Really, it implies that for higher rated whiskies value doesn't quite matter as much dollar for dollar. And then, you think, is it value? After tweaking around with the analysis, doing some more number crunching, I realized that, on a global scale, interesting tweaks help a little, but not a lot. So, the options for me: continue to refine and adjust my assumptions to try to come up with something "perfect" or just use the value score as a rough indicator, rather than the law. Coming up with more assumptions to adjust the score just means that it is more fine tuned to myself, specifically. Option 2 is way easier, and way more appropriate - it is a relatively simpler formulation of how I regard value which actually fits a broader population than just me. All this to say - despite its flaws, I have decided not to change the value score. A few considerations:

  • Beyond the assumptions, which I deem to be reasonable, the only subjective part of the value score is my taste rating of a whisky (which, indeed, is subjective - palates are incredibly diverse).
  • All prices are set to the Canadian Market. Thus, whiskies may be more valuable in different regions of the world as certain whiskies are cheaper than others depending on the market. I always say what the price is based on, but all scores are adjusted for inflation/increase in value so that the value score remains consistent with how the market value is increasing.
  • For different areas of the world - take a look at the average line in the graph above, representing the average cost for a given rating. This line corresponds to a value score of 72. If your whisky is the equivalent of $38 (the standard deviation) more expensive than this line, its value is 40/100. If it costs $38*2 = $76 less, its value is 15, etc. If it is $38 cheaper than the blue line for a given rating, its value is 91. If it is $76 dollars less, its value is 99. Etc...if you are in to this send me an e-mail.