Looking through results of 13 wine competitions, researchers found:
Of the 2,440 wines entered in more than three competitions, 47 percent received Gold medals, but 84 percent of these same wines also received no award in another competition. Thus, many wines that are viewed as extraordinarily good at some competitions are viewed as below average at others.It gets worse: the likelihood of winning a gold medal was no greater than chance alone.* To conclude, the report asserted:
- there is almost no consensus among the 13 wine competitions regarding wine quality,
- for wines receiving a Gold medal in one or more competitions, it is very likely that the same wine received no award at another,
- the likelihood of receiving a Gold medal can be statistically explained by chance alone.
Still, the study is quite useful at highlighting how subjective taste is. Experience is potently persuasive, and if we think X tastes better than Y, we probably believe in our experience quite strongly. In an existential sense, that's fine: we can't drink scholarly research papers, so we should drink whatever we happen to enjoy in the moment. On the other hand, we shouldn't be fooled into thinking that our strong experience of quality makes it so. We take in subtle information that we don't even appreciate (yes, including price), and our "experience" of the wine--or beer--is an admixture of observation, expectation, and perception. But it's not a fixed, accurate representation of some external thing.
The world, alas, is neither fixed nor entirely external. Our assessment of it is hopeless confused by myriad subtle interactions, many of which we aren't even aware of. Charlie Papazian, the sage, had it right the first time: "Relax, don't worry, have a homebrew."
*In the denser language of the report: "Thus, in binomial terminology, if the probability of receiving a Gold medal in a single competition is equal to 0.09, the expected number of Gold medals to be obtained in five competitions is almost identical to the observed frequency. For the 375 wines entered in five competitions, one would expect by chance alone (for p = 0.09), 234 wines receiving no Golds, 116 receiving a Gold in just one competition, 23 receiving Golds in two competitions, two receiving Golds in three competitions and no wine receiving Golds in more than three competitions. The observed frequencies closely mirror these numbers."