If I wanted water, I would have asked for water.


Wednesday, July 03, 2013

Consumer Reports on Craft Beer: #Fail

Consumer Reports, the independent magazine that tests and rates products, has for the first time turned its attention to craft beer.  I am a big fan of CR and have periodically subscribed in advance of car and appliance purchases.  When the magazine's testers have empirical metrics to assess--the actual miles per gallon of a car, the range of features of a refrigerator--they do an excellent job of cutting through the wall of PR you get from the companies themselves.
Source: Consumer Reports

When they turn their attention to subjective measures like the flavors of beer ... not so much.  The magazine took 23 beers, apparently at random, and put them through their rigorous--if wholly clueless--tasting regime.  What did the hardy band of untrained, eager tasters discover?  Oh, things like:
  • "The best ales have intense, complex, and balanced flavors" but
  • "The best lagers are very tasty but not quite complex or intense enough to be excellent"
  • The worst beers "are decent but not as balanced, complex, or intense as the others, and some have off-flavors—hinting of cheese, soda water, or even paint."  One of those worst beers?  Sierra Nevada Kellerweis.  One wonders which off-flavor in that beer caused it to be downgraded.
Americans have Protestant hearts.  Deep down we believe that anyone with a can-do attitude and a little elbow grease can accomplish or figure out anything.  We don't need no stinkin' education or priests to tell us what's what--we're fine on our own, thanks.  There are many ways in which that's a wonderful approach to the world, and it helped people like Henry Ford and Steve Jobs punch through boundaries.  But it comes with a massive blind spot, one exposed when untrained people trying to assess "best" among a widely differing group of beers with no regard to tradition or style. 

I imagine the scene of CR secretly assessing the art at MoMa.  "The best works are bright, detailed, and harmonious, but the worst are monochromatic, muddy, and lack intensity.  We found the Cezanne and Van Gogh paintings to be vivid and descriptive, but the Seurats duller, less lifelike.  One rater compared the lowest-rated painting, a Rothko, to a smudge on the side of his house."

Context matters.  If you don't understand why a beer tastes the way it does, you're not going to appreciate the flavors you apprehend.  If all you're looking for is complexity and intensity, you're only going to like a small percentage of the beers out there.  But that doesn't mean those are the only good ones.

10 comments:

Darlene said...

I think Consumer Reports should stick to appliances, cars, and the like and leave the beer tasting to the consumer since the taste of beer is so subjective. If you need to know about a beer, go to RateBeer or Beer Advocate.

Jeff Frane said...

My cousin worked for CR for years and was responsible for a number of beer ratings (NAIL, mostly) in the past. He had the most astonishing capacity for beer I've ever seen, and I've seen a lot. Those tastings were really just an excuse for him to get a lot of free beer and it appears that the tradition lives on.

Bartimaeus said...

Their assessment of homebrew was amusing as well. Give a cheap kit for light lager to several novices and surprise! The homebrew was not very good.

Jeff Alworth said...

Darlene, I am not always a fan of the ratings sites, but I agree: they will guide you far more ably than CR.

Jeff--that's fascinating. The hodge-podge they reviewed did suggest beers someone sent them, not an honest effort to get a representative sample of the country's most popular craft beers.

Bartimaeus--totally.

Samurai Artist said...

I dont know, sounds pretty much like how the average Beer Advocate reviewer determines ratings.

Perros De Maiz said...

There has to be a better metaphor for no training than "protestant heart".
Seems like a bit of a condescending generalization.
I guess I know what you mean, (not a great metaphor) though maybe you think they deserve it. You might be right.

Ed Chainey said...

Consumer reports would do far better if they choose a style of craft brew - Pale Ales, Brown Ales, Bock Beers and then did their comparisons solely amongst those within each class, rated by accuracy to the style, flavor, balance, price, etc. Rather than publishing two blanket and inept reviews in two years, they might have been able to publish 4, 6 or 8 good and meaningful reviews in the same amount of time. CR is ot ready for prime-time beer reviewing.

THOMAS CIZAUSKAS said...

Spot on! "Context matters. If you don't understand why a beer tastes the way it does, you're not going to appreciate the flavors you apprehend."

Eric Steen said...

Smudge on the side of the house or subconscious art of graffiti removal?

https://www.youtube.com/watch?v=pUa9x1-SMLo

Jeff Alworth said...

Eric--perfect.

Post a Comment

NOTE: Blogspot has been eating some comments, and there doesn't seem to be anything I can do about it. IF your comment doesn't appear, it's not you, it's not me, it's the genuiuses at Google. Sorry--