Some time back, the Oregonian's main movie reviewer was a guy named Ted Mahar. His reviews were generally positive--and this was a problem. It got to be that we nick-named him "three-star Mahar" because he never really panned or gushed about a movie. Everything was, in the manner of the children of Lake Wobegon, above average. And therefore his reviews were of little use.
Not too long ago, a friend of mine castigated me for turning into a three-star Mahar. It's true--most of my ratings fall in the range of A- to C+ (here's what they mean). I plead helplessness, though: most beers here fall in that range, too. Obviously, few beers qualify for the "world class" standard ("a superlative example of the style or an exceptionally original beer"). But few fall below the standard attained by a C+ beer, either ("a well-made beer that is a fairly common example of its style or a near miss on originality").
I am, obviously, a homer, and so you have to suspect me of putting my thumb on the scale. But scan through the aggregate scores on BeerAdvocate and RateBeer--they're no better. This may be due to different factors, but the upshot's the same. With only very rare exceptions do breweries put out either badly ill-conceived recipes or beer with off-flavors. We have a problem of compression.
I'm almost to the point of abandoning ratings altogether and just letting the descriptions stand on their own. (I've actually done that more and more with recent reviews.) It's easy enough to change the scale so there are more calibrations between "common" and "world class," but that doesn't exactly resolve the problem. Worse, it exaggerates the effect of personal preference. I'd be willing to entertain some wholly novel style of ratings, though, if only I could think of one. Any suggestions? How do you make sense of subtle differences between beers?
_____________
Share
Interesting subject Mahar. ;-}
ReplyDelete"Personal Preference" does come into play when rating a beer, but it seems to be getting harder to rate beers as the lines of style standardization gets fuzzier.
I'm all for beers that are different, that are out of style, but how do you rate a beer that has no industrial standard? Personal Preference is one thing, but rating one of these beers becomes impossible as there is no standard for comparison. Brewers have blurred the lines of normality of style.
At this point in the brewing world brewing a classic Pale Ale is the standard. Same with other basic beer styles. Anything less than near classic in style falls below the C rating. What used to be an "A" rated beer, now barely gets a "C." We've moved beyond rating a perfect IPA and giving it an "A." There are a multitude of great IPA's. The bar has been raised.
Any beer with even the minutest flaw can be dumped in a rating. We're guaranteed a 80% chance of getting a pristine to-style beer. Why waste your time with anything less. It's the other 80% that are all decent, that get harder to rate.
It may be easier to just pick apart a beers structure and evaluate it's components. Give people a good description of the beer and not rate the beer at all. Of course, that means some a little more in depth than "It's Good," or "It's brown, malty and rich." ;-}
I agree that most beers fall within a certain range. For me, it's about A- to C+. I don't think there's anything wrong with that. It reflects reality. I'd urge you continue giving letter grades. However imperfect, they do provide a useful gauge.
ReplyDelete@Soggy
ReplyDeleteI agree with letter grading beers for the general beer public. Number ratings are OK when judging, but not necessary for an informal rating.
In regard to beer ratings vs quality. The quality out there is high, but that should make for stricter ratings. Then again, rating non-stylistic beers comes into play again. ;-}
This comment has been removed by the author.
ReplyDeleteI say we institute more simplicity and start to enforce Reinheitsgebot! That'll teach those brewers!!
ReplyDelete:-P
In all seriousness, I think the current rating system, be it A-F or 10-1 is sufficient for most of us.
Those who are mostly uneducated about all the varieties have something to gage, and the rest of us only use it as a loose grade, including personal preference and taste to our own evaluation.
I'm a college professor, and this is a constant problem for us too, especially in a fuzzy discipline like English.
ReplyDeleteOne technique is simply pass / fail -- would you recommend this to us -- should we spend money on it? Is it good enough? Or is it not worth spending money on, especially in such a rich environment?
A variant is pass/fail/wow! -- or, you could think of it as three recommendations: don't drink this; by all means, have a go; and "you must drink this amazing beer before another moment passes."
Another technique is just going with narrative assessment.
These work well in tandem, I think. The default for a reviewed beer should be "this is good, have a drink." You might only mention a "rating" if you think it's something very special OR if it is not to be drunk.
My wife designs surveys, and she says you should never give an odd-numbered scale for ratings, because people tend to pick the middle. So a lot of people use a four-point scale: something like "high pass / pass / fail / low fail."
For beer or restaurant reviews, I find descriptions or ratings that take into account the cost very useful. Given how many good restaurants there are in Portland, and how many good beers, it's very useful to hear: "this is good, and a GREAT value," or whatever the entry on that matrix would be. Does that make sense?
Just some thoughts.
--JT
The way I do my ratings on Seattle Beer News is ranking a beer either Not Worthy, Worth Trying, Buy It, or Find at All Costs.
ReplyDeleteIt's simple and tells the reader what I want them to know: whether the beer is worth their money or not.
The majority of my reviews fall into the Buy It or Worth Trying categories, a decent amount in the Find at All Costs, and just a few in the Not Worthy (It's not like I go out of my way to review crappy beers).
Thanks for the input, folks. I see that it's actually worse than I thought. Ratings are useful to different people for different reasons. On the one hand, they offer a kind of technical point of precision for the beer geeks, but on the other, what people really want to know is: should I try to find this beer and drink it?
ReplyDeleteJeremy, I used to do research, too, and we tried to force opinion wherever possible, too (answer categories typically included things like: unsatisfied, somewhat unsatisfied, somewhat satisfied, and very satisfied).
Maybe I'll just try to incorporate more pointed opinion in my narrative account, ala Geoff's suggestion, and keep offering graded ratings, too.
I am unsatisfied!
i read a lot of video game magazines and one of my favorites got rid of their ratings scale a couple of years ago.
ReplyDeleteat first i was a little irked, but then i realized that an A ranking in a game doesn't necessarily mean i'll like the game - it just means the reviewer really liked it.
so, i read the reviews now and look to see if it is something i'd be interested in. if not i pass, if so, i pick it up.
i think the same could be said about beer. even an A rating from you in a beer isn't necessarily enough to get me to drink it, it's the flavor of the review that's makes it more or less appealing.