You love the blog, so subscribe to the Beervana Podcast on iTunes or Soundcloud today!

Friday, September 21, 2012

Further Lessons of Blind Tastings

This blog post may have the appearance of navel-gazing, but then again, I'm being redundant.  ("Your blog actually has some merit."  It's the "some" we emphasize around here.)  Although for the most part this little blog lives in a quite corner of the internet, untroubled by waves of attention or traffic, it is actually connected to the world wide web, and from time to time one of the posts go viral.  My Westvleteren versus the world post of Monday was an example.  Traffic rocketed to nearly 400% on Monday and doubled the next three days, buoyed by retweets and links from Reddit and Beerpulse

This was surprising to me, but perhaps it shouldn't have been.  One thing about the beer geek community is that we take ratings very seriously.  This is beautifully American and democratic.  We assign to the commons the task of assessing quality and ratings are prima facie evidence of a beer's merit.  Actually, I think everyone has an uneasy relationship to ratings--we know they have value somewhere in between the 0%-100% range, and fight about where to place the figure.  The debate is between those who highball it and those who lowball it.  So naturally, the comments (here, on Beerpulse and on Reddit) tended to follow that debate.  On the one hand:
You mean people's perceptions of beer often matters more than how much it actually tastes?  Entirely unsurprising. I think the essential thing to take away from that article is that there really isn't that much difference between great beers at a certain level - Westy 12 isn't honestly that much better than something you can get for less than fifteen bucks at any decent liquor store. But obtaining Westvleteren 12 is so much more of an experience for people that it colors their enjoyment.  (via Reddit)

And on the other:
Way too much conjecture in your post. So because you don't like the style and YOU think it's too heavy, etc. (which was stated matter-of-factly, rather than as an opinion) than it's overrated by all those other people? And your explanation is that they like these beers because they are enamored by the association with monks? Weak. Like the guy above says, why should anyone put any stock in the results of your "panel" or treat them as at all representative of other beer drinkers? 
While I obviously align myself with the first (insightful), the second one expresses the view of a majority or large minority of beer geeks.  S/he's arguing, in essence, that taste is objective.  An outlier like me can slag a beer, but who cares what outliers think? We know that the beer is good because thousands of people have agreed it is so.  To suggest the ratings are wrong is to raise the disconcerting specter of a world in which "best" isn't empirical.  Some people really don't want to live in that world.

I don't want to take a side in the debate.  If you take either argument too far, you end up in absurdville.  What I would instead like to say is that the way we think about the question differs profoundly when we're confronted with a half-dozen unmarked glasses of beer.  Every time I do it, I enjoy the destabilizing sensation of not trusting what I think I know, and then learning something different than I thought I knew.  It's a tonic to lassitude and cynicism that comes from being a beer geek, and in an hour's time, a reminder of why I became a beer geek in the first place.  But maybe those lessons aren't transferable. 


  1. The problem with the term "overrated" in my opinion is that there is an inherent sense of "I know better" in stating it. The first quote implies that because some people don't like something, it isn't as good as other people say it is.

    I agree that it's better to end up between both quotes rather than on one side or the other, but I think the first quote tends towards obnoxious elitism. Of course, the second tends towards believing the majority must be correct so that's not so good either. I guess my opinion is that you can think something isn't good, but crapping on everyone else's opinions while doing that is obnoxious.

  2. Apparently I'm late to the party. But....

    I would agree that the answer lies somewhere in the middle. But as someone who has taught styles classes for years, I would also argue that tasting is much more subjective than even the most experienced tasters would like to admit. And blind tastings, while incredibly enlightening and useful, aren't always as blind as people would like to think. Knowing certain things, like the list of beers that are included, or even the style, can influence perceptions. Moreover, if there is any pressure to be "right" about your choices (pick out the ringer, pick out the incredibly rare beer, are you really being served what the tasting leader says he's serving, etc) the results are likely to be skewed.

    Are blind tastings more useful or honest than evaluating something where you can see the label? Sometimes, but a whole lot depends on the circumstances of the tasters and the tasting itself. Does someone else's blind tasting panel trump my own tasting experience? Sometimes, but I'll still take any else's opinion with a grain of salt, because I know how subjective this is. I've done too many blind tastings and experiments to put 100% stock in anyone's ability to be objective (including my own tasting ability).

    Not so sure? Try the following experiments.

    1. Take three similar styles of beer (pils, IPA, whatever, the closer they are to each other the better). Have someone pour 4 oz of each identical glasses. Then have them pour another 4 oz of one beer into a fourth glass and give you all four blind. Then you decide which of the two beers are the same and which two are different. Then have them do it again and see if you can still tell. If you can do it perfectly every time, double the amount of beers and see if you can pick the identical beer out of a group 8 consistently.

    2. Take 4 beers of the same style and taste them blind. Write good notes with good descriptors and then rank the beers from your favorite to least favorite. If you really want to be hard on yourself, give them numerical scores. In three days, taste the same four beers the same way. Compare your descriptors, scores and ranks and see if they're the same. Again, if you can do it consistently, double the number of beers.

  3. Blind tastings are a very good exercise to compare beers with as little prejudice as possible, but whether they can define which beer is "better" it's quite relative. I've seen other similar blind tastings where W12 came slightly on top. I'm sure that if you did the same tastings, with the same beers, but with a slightly different group (or even the same), the results will likely be different. On top of it, there are, IMO, other factors that make one beer "better" than another, factors that are hardly ever taken into account in rating sites.

    That said, I don't believe in the idea of "best". There's no beer that is the absolute "best". There's no doubt in my mind that W12 is a great beer, but I would never, ever drink it in a hot afternoon after cutting the grass or having taken a long walk, I'd prefer a run of the mill pale lager instead.

  4. "One thing about the beer geek community is that we take ratings very seriously." Which ratings? BA and RB? Or are there other ratings?

    If it's the latter two, then beer geek=fool.

  5. I'm happy to endorse Bill's suggestions, too. But there's something I was trying to point at in this post: tastings are valuable when they allow us to experience our beer in new ways. It's the experience I would emphasize, not the beer (for many of the reasons Bill cites).

    Everything is mutable: tasters (both their hardware and software), circumstances, beers. In a kind of existential sense, beer doesn't exist innately. It exists provisionally. That's why the very idea of "best" is hard to endorse. There are beers that can't find circumstances favorable enough to be enjoyable, and others that survive even the worst circumstances--but mostly, as that guy on Reddit said, " there really isn't that much difference between great beers at a certain level."

    The cool thing about tasting beers in different ways is to both encounter our assumptions and thereby also encounter the beer itself.

  6. Here's an idea... Instead of arguing over the value of rating and evaluating beer, why don't you learn how to comprehensively achieve the goal of evaluating a beer, Jeff?

    I was going to say... "Or educate your readers," but that hasn't been in the cards for years.

    There is an evaluation process which any moron can follow, but identifying the flavor components of a beer takes experience beyond just drinking a lot of beer.

    If someone is giving there opinion about beer; i.e. I like or don't like, that's fine. But they better have some real seasoned beer evaluation skills if I'm going to value (or even argue) their dissected description.

    OTOH, if someone wants to just drink a beer, enjoy it and shut the F*CK up.... That works too. :-0