We Won’t Participate as Judges in Wine Competitions: Here’s Why
Just a few wines to be tasted at this years Indy International Wine Competition
Photo courtesy of Dave Falchek
It's official. We are done judging big, blind, medal-focused wine competitions.
We did not arrive at this position without much thought and
discussion. Ultimately, we believe that transparency and clarity are
core values that should permeate the wine world — from the creation of
wine, to the marketing of wine, to the writing about wine.
Everything
that happens in those areas should relate in some way to answering this
question: Is this providing more transparency and clarity to the
consumer, or less?
We have decided that medal-focused
competitions provide less clarity and transparency to the wine consumer.
We feel that medals only confuse consumers instead of educating them, and that they provide little real value.
Our
position going forward will be simple: The editors and writers at the
New York Cork Report will not accept invitations to judge wines at
large-scale, blind-tasting events with the goal to hand out "medals" to
"winning" wineries.
We want to explain, and -
this is vitally important — we mean no disrespect.
The vast, vast
majority of competition creators, organizers and judges perform their
roles with the best of intentions. Often, we find that the wines we
think are best are the ones that win top honors. Anthony Road
Wine Company's 2008 Semi-Dry Riesling winning the Governor's Cup is one
example).
But that cannot and does not change the reality: There are so many medal-awarding competitions
that the events have lost any sense of meaning to the average consumer,
and even wine-loving consumers can't possibly know the significance of a
single bronze or silver or gold medal awarded at the many, many events.
Furthermore, the very act of blind judging a wide range of wines should
be viewed as a parlor game and not some
official declaration of merit.
Good
intentions give way to nebulous marketing
We
can't stress this enough: The organizers of wine
competitions are people who constantly impress us with their enthusiasm
and event planning. Collectively, we have judged at many events and have been invited to judge at many more. We admire the goal of
wading through oceans of wine to sort out the very best for consumers.
The
problems with judging will be addressed below.
Even
the medal winners can't explain much about the meaning of such an
award. Evan recently stopped by a Finger Lakes tasting room that was
drowning in medals. He was told, "Our 2006 Merlot won Silver at the So-and-So Wine
Competition!" He asked the staff to explain what that meant. "Well, it
probably means that the judges liked our wine very much!" they replied. He asked who the judges were. They didn't know. He asked how many wines, by
percentage, got at
least a silver medal. "Oh, I don't think it's very many," came one
reply.
Sadly, that's wrong, by almost any measure.
On Long Island, Lenn has been similarly regaled by tasting room staffers with stories of medals awarded — often incorrectly. He's up on some of these things, so he often knows that they are wrong when they tell him that their riesling won gold but it actually won bronze. The average person off the street can't possibly know; there are too many medals from too many competitions. Ultimately these medals and discussions of them have become nothing more than white noise, like static on your television.
Medals have almost no
defined meaning that the wineries themselves can even explain, let alone
their consumers. Ask a consumer what a medal means — really, grab a
customer in a tasting room — and there's almost no chance they'll be
able to offer anything close to an answer describing where it comes from
and why the judges awarded it.
It seems that
wineries simply hope the use of medals will make their bottles more
attractive. We understand the impulse. The business of wine is a competitive one, and
discretionary dollars are being held tightly. But ultimately a state
that is attempting to attain world-class status does itself a disservice
with an over-reliance on meaningless handouts.
We
can promise that almost every
tasting room customer would be shocked to find out that often the
standard for getting at least a bronze medal is simply to create a wine
that is not mortally flawed. That's it. That's the baseline.
The
first problem with judging: Subjectivity
At
Evan's first wine judging competition, a huge annual event that we won't
name, he remembers a debate over a flight of pinot noir. One judge refused
to award a particular wine a gold medal because, in his words, "There is
plenty of fruit but not nearly enough supporting oak." Evan, understandably, was stunned. A
judge demanding more oak? What next?
He didn't have to wait long to find out. During the next flight, a judge detected a whiff of Brettanomyces in
one of the wines. She decided it was a nice addition to the wine,
adding character. The judge to Evan's right was offended to the point of
near-insanity. "Brett is a FLAW," the judge declared. "And a flawed wine
wins no medal." The other
judge persisted, arguing that it should be a gold medal wine. Evan thought he was about to witness a fistfight.
How can
you or anyone else tell a judge how to evaluate wines? The beauty of wines is that we
have the opportunity to decide for ourselves what makes a wine special.
Now, that's not to say that there aren't clearly discernible qualities
and flaws. But if I love oak and over-extraction in, say, cabernet sauvignon, and you appreciate a more restrained approach, which one of
us is right? If I think the best wines are indicative of where they were
made, whereas you believe the best wines are hedonistic missiles,
place-be-damned, who's correct?
If you sit in
on a judge's panel at just about any wine competition, get ready to hear
the same conversations. And then ask yourself how anyone can possibly
hand out medals when it's over, as if one wine correctly identified that
7×4=28.
The second problem with judging:
Blind tasting
Everyone on the NYCR team has come to love blind
tasting. It is great fun. It is also a bit like a sporting event or game, not befitting the
anointing of medals that ostensibly carry serious value.
There
is perhaps no wine more fitting to explain this problem than Finger
Lakes riesling. The best winemakers in the Finger Lakes often remind
their customers that riesling is a "food wine." It certainly is. It is
versatile, ranging from dry to sweet, and pairs harmoniously with a
range of dishes. Winemakers have such things in mind when crafting their
products. But they are not producing rieslings designed to impress judges in sterile, blind-tasting settings.
Now try to imagine tasting dozens
and dozens of these wines with hardly a bite to eat. The acids are
ripping at your mouth, and in the sweeter flights the sugars seem like a
welcome respite. In the cabernet flight, there is no juicy steak to
accompany a rich wine, and the judges
are left to consider them bereft of that partnering.
But
most importantly, blind judging robs the evaluators of the most
significant parts of the wine — its context.
Tell a judge he's drinking
cabernet, and he'll immediately try to lock in and ascertain the country
of origin, then the region and perhaps sub-appellation. But the mind is
a funny thing. Instead of simply enjoying (or not) the wine, and
thinking about it individually, the judge begins to add context where
there is none provided. How did the other wines in the flight taste in
comparison? What might that say about this wine? When was the last time I
tasted a wine like this one? Where was it from? Should I allow myself
to believe this is Bordeaux, when I'll feel awfully silly when I'm told
it's from somewhere else?
Delving deeper, we
find that judging a wine that is simply known as cabernet sauvignon is
extremely constricting. We don't want
a Napa cab to taste like a Bordeaux. We expect Chile to turn out
something else entirely. If we're tasting a Bordeaux cabernet that tastes like
Napa, we're bound to be disappointed. But tasting blind, we might convince
myself it's from somewhere else, mistaking place and winemaker
intention. Whoops.
We've had judges tell us that we should forget about figuring out where a wine is from and simply
taste it to see if we like it. Fair enough. But in that one statement, we
see exactly why wine has become so homogeneous, so dangerously banal.
Judges are not required to give a damn about a wine's sense of place.
We find it vital. With no standard, how can we expect judging to be
consistent?
Ah, but see: It's not consistent.
Not even a little.
There is ample evidence
that judging is like throwing darts
When
Robert T. Hodgson set out to research the reliability of
judging, many of us suspected he would find that judging is
inconsistent. Instead, he found that medals are awarded in a fashion
that almost appears to be random. Hodgson wrote, "It is reasonable to
predict that any wine earning any medal could in another
competition earn any other medal, or none at all." Indeed, he found
hundreds of examples of wines that earned gold medals in one competition
and no medal at all in another.
Put another
way: If you make a competent wine, you can enter enough competitions and
that wine will almost certainly win gold eventually.
No
study is perfect, but we suspected that after this study was released,
drastic changes would hit the wine judging circuit. We have yet to see
any. Hodgson stated that his goal was to provide some measure of judging
reliability to help these competitions improve. We see the result being
supportive of the idea that these competitions
ought not exist at all. After all, judging in mass competitions is
putting wine into just about the least most suitable place for good
evaluation and enjoyment.
And for wineries
that might protest, it should be said that the little study that has
been done only indicates that tasting room customers really don't care
much about medals. Why should they? As we've already explained, they
don't know what the medals mean.
Clarity?
Consumers don't know which wineries entered a particular competition and
which didn't, they don't know the judges and what the judges are
looking for, they don't know how many medals were awarded, and they
don't know what a medal is supposed to signify.
That
should say everything.
Our decision, and
our call for others to join us
In the
future we will politely decline invitations to judge at
these events. That does not mean we won't participate in wine seminars,
conferences, etc. This is simply about mass judging. The
wine competition circuit has become quite an industry itself, but there
has to be a good explanation for the purpose it serves.
We ask our
colleagues to do one of two things: Pledge to join us in this decision,
or provide a suitable answer for the problems we've outlined above.
We're more than willing to listen, and to change our minds if it can be
proven that these competitions help the consumer.
But
right now, we're sitting them out. - The NYCR Team
good post, but there’s a third way - just judge at the decent competitions. Like wines, there are good and bad competitions. It’s difficult to run a good competition, bit some good ones exist. The primary problem is that good judges in these sorts of settings are rare. Very rare.
I should also mention that we will be modifying our own NYCR Wines of the Year competition format to align with this new, public stance.
And even the good ones don’t escape the challenge of what medals really mean to consumers.
Jamie: That’s a fair point, but is there a list somewhere that will tell us which competitions to agree to and which not? I don’t think so
I’d also like to thank Evan for pulling this post together and doing the heavy lifting for it. Capturing dozens of back-and-forth emails into a coherent post isn’t easy. He did it with skill and aplomb.
Brilliant post, and one that I wish I had written. Your writing does much more for the industry than judging. I would wonder though how much of what you say, or suggest can also be applied to the points.
Good comments. I made a wine that won, Gold, two silvers and nothing just this year.
Bravo.
I noticed a while ago that many of the Finger Lakes wineries I consider superior are the ones that spend less time talking about their medal count. In my experience, the best wineries focus on their product instead of awards. When I’m in a tasting room, I’ll purchase a wine with interesting characteristics or a wine that will pair well with a food I like. Winery staff members who help me to understand and appreciate the wine will make a sale. A bottle displayed with bling will not.
As Wine Judge, I must agree with some of your points. Fatigue is a serious problem (one that many have criticized Parker for after he mentioned multiple wine marathons).
I would like to mention that it is through these competitions that I was able to bring Red Newt Riesling and Ravines to my customers, I’ve used 2 X G.C. winner Chateau Lafayette Reneau as a springboard to other Seneca Lake wines and got to really come to know more than a few wine makers. I recognise these are unique circumstances that only pertain to the handful of retailers who qualify to be in these events.
oops hit post too soon- but perhaps the NYCR writers would benefit from trying some wines blind, discovering a few that they felt were superior and doing further research and tasting on those standouts. The IEWC and G.C. are perfect venues for that.
Dear NYCR team,
While I completely understand your frustration with the many shortcomings of show judging as practiced here in the United States, I wonder if many of these same criticisms couldn’t be leveled at the practice of rating wines in general. For example:
“…there are too many medals from too many competitions.”
If this is really an issue, isn’t “there are too many ratings from too many reviewers” even more true? Just look at all the magazines, newsletters, blogs, bulletin boards and CellarTracker users-surely those numbers far exceed the number of wine competitions.
“Medals have almost no defined meaning…”
Hmmm…I’m pretty sure there is (or should be) a set of criteria provided to the judges of any show and available to the public setting forth what each level of accolade means. Just as there is (or should be) for any rating system. Unless you’d like to suggest that the 100-point scoring system is so transparent that it requires no elucidation…
“blind judging robs the evaluators of the most signficant parts of the wine-its context.”
How important context is to wine judging depends on what the criteria for judging are, regardless of whether we’re talking about medals or points. If the criteria include only sensory assessment of balance, length, intensity and complexity, the need for context is minimized. If the criteria include some notion of typicity (however defined), context is vitally important. And while context may be important, it is often more important to avoid any perception of impropriety or bias that can result when tasting open label. Some context may be gained while still tasting blind by appropriately narrowing the classes; e.g., “Aged New York Cabernets, four or more years post vintage.” or “Young New World Rieslings, no more than two years post vintage.”
“…judging is like throwing darts.”
Ummm, yeah, but… A wine faring differently in one competition to another is not particularly different from one reviewer liking a wine and another not-and I don’t think anyone would find that particularly unusual. So because wines can earn different ratings from different reviewers we should stop rating wines? I believe the same study also found that certain judges were more consistent than others, which should provide some basis for improvement.
I’ve only judged at this sort of wine show once, a long time ago, and I understand it can be a frustrating experience. But I think probably the only way to make the system better is to participate and work from within to improve it. Removing yourselves from the system may make you feel better, but having less qualified judges taking those seats and awarding those medals won’t make show medals more meaningful or less confusing for consumers.
Lenn,
Interesting.
From my perch, I think it’s a good idea for untrained people to stay away from judging wine, but that would eliminate a zillion wine critics, too, and those useless points that critics issue.
Your blanket statement against blind wine evaluation is completely one-sided and from the standpoint of an untrained evaluator. If you want to understand the value of blind wine evaluation, which is for quality control reasons, you should attend the workshops that Cornell U operates for winemakers at the agricultural station in Geneva, NY.
But I agree that the blind tasting used for many competitions is flawed.
Jamie - I’m an enormous fan of your work, and it’s great to get feedback from you. If your suggested third way could be easily achieved, it would be worth discussing. Ultimately I keep going back to Lenn’s point in the comments, which is that I’m still not sure it clearly serves consumers. But can I ask: Can you add an example or two of the best competitions? Again, thanks for stopping in.
Joe (and Ryan) - Regarding wine ratings, Lenn and I like to rib each other because I don’t favor point scores in any form. But I have much less issue with point scores than blind tastings. First of all, Lenn can taste one wine in a day for review, and he can taste 15, but it’s not going to be several hundred. Palate fatigue is not a potential problem as it is in judging. Also, Lenn tastes non-blind and cares deeply for the background, the approach, and the story. Wine writing is subjective, and we don’t apologize for that part of it. Readers can decide whether they like the approach or not, or even if they like the approach but don’t like a reviewer’s writing or evaluation priorities. But there is more context in a NYCR review. Finally, readers can decide if the reviewer is consistent. They don’t have that opportunity with judging; they almost never know who the judges were, and even with that information, they might not have any idea what those judges typically favor.
Ed - Speaking of fatigue, great point! Not sure why we didn’t stress it even more in the original post. This is a major issue. Now, you bring up another very important point in this debate: Often, judges will travel and have a chance to experience regions and wines they otherwise might not. For example, the Finger Lakes International Wine Competition, held in Rochester each year, offers judges a chance to travel to select Finger Lakes producers for tasting and dining the day after the competition. It is a wonderful idea, and very effective. I simply hope the industry would find a way to continue to bring in retailers, writers, and professionals in lieu of these judging events.
Ryan Love - Not a big surprise, is it, that your idea of the best producers are exactly not the ones bowling you over with medal results. Great observation.
We knew this would generate great discussion. Glad to see it happening.
First, I think it’s important — or would at least be great — if we could keep the discussion to the topic at hand. Adding scoring/rating/points into this discussion (especially in a NYCR context) gets away from the main argument.
Thomas: You said “completely one-sided and from the standpoint of an untrained evaluator” but didn’t expand on it. Could you? A winemaker blind tasting his or her own wines is a bit different than judges at a competition doing it.
I think it’s important to say that the NYCR crew blind tastes all the time. I actually taste blind regularly if I’m reviewing several of the same variety/blend — at least on day one. Then I un-bag the wines and obviously know what they are on days two and three.
To me, one of the most important things — and why I taste for 3 days with and without food — is that NO ONE is drinking wine the way that judges or most critics are tasting them. I won’t even score a wine based on an at-winery tasting because spending a minute or even five with a wine is an artificial construct.
We try to be very consumer- and reader-focused here on the NYCR. They are the only reason we use the 100-point scale — they asked for it.
Who is asking for medals? Obviously wineries love to use them as marketing tools, but I think that’s the easy way out and not all that effective. When EVERYONE has a gold medal, no one is differentiated.
Oh, and did I mention how much I have disliked the judging gigs I’ve done? It’s exhausting, mouth-raving work and no where near as educational as sitting down with a wine (or even 15 to Evan’s point) and spending several days with them. I’ve tasted wines on day 1 and thought “wow, that’s not very good.” But a day or two later the wine was transformed.
If I tasted that same wine at a competition, it wouldn’t fare very well, right?
I guess in some ways I’ve brought scoring into the discussion after saying not to. Sorry about that.
Lenn -
See, the scoring discussion is good to have right here, right now. Because your method does indeed allow for some evolution, some “opening up”, and the chance to see how the wine plays with and without food. Not everyone rates wine that way, and as much as you and I might disagree with scoring in general, I simply have to admire the way you do it. Your method is so different than wine judging, well, it’s like comparing apples and anvils.
You guys nailed it with this one.
The fact is that for most of us, wine is an intimidating subject.
For example: Brettanomyces? Heck if most of us can even pronounce/spell the word much less recognize it in a glass of wine.
Thank you for your passion and for consistently encouraging your readers to trust their own tastes.
To quote this post, “The beauty of wines is that we have the opportunity to decide for ourselves what makes a wine special.”
The oldest expression in the wine world is the one where “wine doesn’t travel” and “it tastes better at the winery than …”
CONTEXT is the key to an intellectual appreciation, and sometimes a deeper affection, and certainly a better understanding of a particular wine.
I’m not sure who has ever come away from a blind tasting with a deep appreciation of a wine - not in comparison to drinking a wine in proximity to the vineyard and/or with the winemaker present.
If that is LESS honest, than so be it. I’d much rather read that story - and I’d rather appreciate that wine vicariously through the writers on NYCR, than hearing about a silver medal produced by a clinical dissection of wine’s flaws or lack thereof.
How boring.
“Thomas: You said “completely one-sided and from the standpoint of an untrained evaluator” but didn’t expand on it. Could you? A winemaker blind tasting his or her own wines is a bit different than judges at a competition doing it.”
Lenn,
First, I said nothing about a winemaker tasting his or her wine blind. Where did you get that from? The training at Cornell is for those of us in the wine business to learn how to accurately identify the things we often run around telling other people we think we know about wine but often find that we know less than we think.
Evaluating wines for hedonistic purposes is never going to get anyone to a point of agreement, whether the evaluation is done blind or in context. Aesthetic evaluations are personal and subjective-period-they have little to do with the wine and more to do with the evaluators. I know that seems cynical, but no one has come up with any system of evaluation that has proved otherwise to me, except for those training programs designed to hone our sensory equipment.
To truly evaluate a product you must first agree to and establish standards (no standards exist in the critical wine evaluation world); then, evaluators must be trained to identify the things that have been set as standards and the things that fall outside those standards and are considered flaws; then, the evaluation process must include a test of the evaluators for their abilities and their accuracy (this is done by intentionally adding into an evaluation both flawed products and duplicates).
Brett is a perfect example of the fallacy behind hedonistic evaluation. Some people hate any noticeable Brett and some people accept a certain noticeable level. Plus, each person has a unique threshold for identifying Brett (and other things in wine).
More important, however, is the question of whether evaluators know the true problem connected with Brett; the one that has an effect on the course of a wine’s life span in the bottle. Only trained people can even come close to talking about that issue.
Brett is only one intricate component in wine that many critics are woefully ill equipped to evaluate.
Unfortunately, far too much of the discussion about wine is not about the wine but about ourselves and our perceptions. For that, all you need is an opinion. This situation is what renders most wine competitions and most wine criticism no more useful to me than handicapping a horse race.
Thomas: Where did I get that?
“…you should attend the workshops that Cornell U operates for winemakers”
“For that, all you need is an opinion. This situation is what renders most wine competitions and most wine criticism no more useful to me than handicapping a horse race.”
Thomas and Lenn seem to be saying the same thing from different angles. I will add to my comment insisting that Context is key and add Thomas’ valid points about Opinions.
The opinion of a group of “untrained” sommeliers, “untrained” journalists and bloggers, and “untrained” wine buyers, when gathered up in all of their blindness and distilled down to one “winner” is indeed flawed, I think we all agree on that.
I think a good point to make is that the Opinion of the staff of the NYCR, when derived from a contextural point of view, and presented transparently, is far more valid and valuable than the mathematical rendering of the contest.
I use quotes to highlight the “untrained” not to mock Thomas’ words at all, but to point out the clinical evaluation of wine for quality control has no bearing, nor relationship, to the processes that go towards wine appreciation - just technical evaluations.
Cornell’s program is no doubt rigorous, but it doesn’t promote the warm and fuzzy feeling we’re all searching for, does it?
I was intrigued by this interesting discussion today. After a lengthy discussion tonight on Twitter about the relevancy of wine competitions, I felt we needed to ask non-industry consumers “If a wine wins a medal at a wine competition, are you more likely to purchase it?” See the more than 60 responses posted at http://www.facebook.com/fingerlakeswinecountry.
I was also surprised that no one referenced Clark Smith’s recent article in Wines & Vines. http://www.winesandvines.com/template.cfm?section=columns_article&content=77574
Lenn,
“…you should attend the workshops that Cornell U operates for winemakers”
Geez, Len. Nowhere does that quote say that the winemakers are tasting their own wines. The Cornell programs are geared toward winemakers to hone their skills, but anyone in the business is welcome to attend, and I reccomend the programs to all who want to learn more than they think they already know.
Jim,
If wine evaluation is about warm and fuzzy, then why all the fuss about how it is done? Is it that those who evaluate would have consumers believe that they come at it with a level of knowledge or an insight that others do not possess?
I’m not so sure that disagreeing with someone’s opinion makes that person’s opinion any less valid, and I’m not so sure that agreeing with it makes it any more valuable. But an opinion that is based on knowledge and training, and can be demonstrably measured, well, now we have something on which to pin our hopes.
Why is it that every time someone calls for trained wine evaluators, someone else brings up the canard about “making it too technical?” When you ask a professional in any field for an opinion, wouldn’t you prefer that the professional know the subject in dept?
The reason this conversation concerning competitions and wine evaluations is an evergreen is precisely because the activity rests on sand.
Sorry Lenn, one n got lost…
..and it seems recommend became reccomend. It’s after my bedtime!
Guys,
If you don’t like the way wine competitions work, you can work from within to try to make them better or you can pick up your ball and go home (like you’ve just announced).
The game will still go on, you just won’t be a part of it. And the game won’t be better for your absence, or less confusing for the customer.
And once you’re out of the game, complaining about the rules just comes across as griping about a competitor. You rate wines-and what else is a wine competition but another way of rating wines?
Yes, wine competitions have their drawbacks. But so do individual reviewers and the various methodologies they employ. It seems a bit disingenuous to say on the one hand that the democratization of wine criticism in the form of CellarTracker reviews, bulletin boards, blogs is a good thing-that everyone’s impressions are equally valid-and yet slam competitions, because many of the same criticisms apply.
The dynamics between large wine competitions, wine critics ratings and consumer-driven reviews are so completely different that they defy comparison.
At the same time, if a wine can manage to rise above competition in all three of these arenas, then it is pretty certain to be not just good, but *really* good. Very few wines can achieve this, which puts us all back at square one, which is that people are potentially faced with SOOOOOO many wines and are still looking for guidance. (Personally, I think they are better off getting that guidance at the point of sale, but that’s another issue….)
My suggestion to you - Lenn, Evan and gang - is to continue to do what you do, but develop a system that is unique to you… NOT points and NOT medals. (The drawbacks in both of those systems are growing more glaring with each passing month in this Internet-fueled age.) Keep that up and ultimately people will simply know that the first resource for advice in NY wine will be what NYCR thinks.
Joe -
This isn’t about “taking our ball and going home.” We’ve obviously given this a lot of thought. In our view, the best way to make a change in wine judging is not to do it at all, because there seems to be no good way to do it in mass formats that would offer any real, discernible benefit to consumers. Staying involved in some small way would only uphold the notion that these competitions serve a public purpose. Unfortunately, the evidence suggests that they do not.
Again, look at the research that shows that just about any halfway-competent wine will win gold if entered enough times. How can we effectively work to change a format that leads to such results? We’re not blaming organizers or judges. They work hard and they’re talented folks.
Wine is simply one of the most complex things in the world when it comes to evaluation. Reducing it to sterile, palate-busting mass tastings is one of the least effective ways to find out if it’s any good.
We’ve already addressed the scoring issue, but you haven’t responded to our points. Take Lenn’s very important point that he tastes wine over multiple hours and even days. We all know that some wines are tight on opening and show better with more time or alongside a meal. This will never be possible in such settings. We also know that some wines open with a blast of something unpleasant upon opening, and that unpleasantness can blow off. Should those wines be penalized because there’s not enough time for them to show their full profile?
I recently saw a Finger Lakes winery boasting of a gold medal for a 2006 Merlot. I’ve had the wine. We all know that 2006 was a largely disastrous harvest for red varieties in the Finger Lakes, and the results show it. That same Merlot was entered in numerous other competitions and did not fare very well. But to consumers, they’re only told it’s a “gold medal wine.” They’re not told, “This wine won one gold medal, but struck out in a bunch of other competitions.”
It’s like a baseball player who strikes out in his first 20 major league at bats, then hits a home run. Then he strikes out 10 more times. Any team would send that player back to the minor leagues. But by wine competition standards, the fans would not even know about the strikeouts! They would only know he’s a home run hitter, based on one unlikely thunderbolt.
There’s just not a good, simple solution that we can see. Our hope is that if others make the same pledge, competitions will either have to change completely or go away. In the interests of consumers, that’s the best result we can foresee.
Morgen, Tish - Thanks for the comments and links. Tish, very interesting work on your medal story this month. Can you post a link here so others can get to it?
Thomas - It’s not an old canard to target the technical aspect of this endeavor. It’s a valid question. Yes, wine can be evaluated on technical merits, but these competitions create so many problems in truly finding the best wines that they simply become “everyone-gets-a-medal” events. It’s worth discussing whether they can arrive at any results at all that are truly beneficial to consumers.
Two people whom I respect deeply, and some of the best judges I’ve spent time with, are Chris O’Gorman (from Merryvale Vineyards in California) and Lorraine Hems (a wine educator at RIT and NYWCC). They plan to stop by and comment, and I look forward to that. They are both smarter than me - by far! - and I think they’ll add to the discussion.
We appreciate the depth of thought going on here. I think this is productive, even if some of us will have to agree to disagree. Cheers.
5 days ago, before this article was posted, in the discussion about the Governor’s Cup, I made a comment which is appropriate to reproduce here:
In Europe they do not fuss so much about wine as they grow up in a culture where wine is served at the table and they probably neighbor a grape growing region. Wine is seen as a beverage to enhance the meal.
Here we have somewhat of a different attitude reflecting our never ending quest for(unattainable)perfection. And when it comes to wine it is a relatively new phenomenon. Therefore we have developed a whole bunch of guidelines to help the uninitiated. We try to make a scientific endeavour out of wine tasting developing rating systems which are supposed to provide a numerical ( as if there was such precision) evaluation of a wine with all sorts of scales the most favored being the 100 points scale which start at 75!
When I have wine at dinner I do not use a rating system or a scale. I tend to reflect on how well the wine goes with the food. Is it enhancing the food or is it taking away from it. And if the wine is extraordinary how does it differ from my expectations.
The point of all this is that sometimes it is useful to pull back and reflect on our discourse. There is no such thing as the best of anything. Like there is no best car, or best tomato or best carrot or best computer without some context by which to have that evaluation. Similarly there are today an increasing number of very good wines and it is the consumer’s good fortune that there is so much to choose from. If I were to drink 10 different wines a day for the rest of my life I will have tasted but a fraction of all available wines. The same is true for self proclaimed or otherwise annointed wine judges. They could not possibly have tasted more than a fraction of the wines of the world. When they judge a wine, we have to ponder, are they telling us about the wine or about their own taste for wine. For those of us who have the self confidence to enjoy our various wines we are not necessarily impressed by wine judges or competitions. And those of us who are curious will attempt to taste more wines we do not know than keep going back to the same wine. And those who are more conservative will keep buying the same wine over and over again because they have found what satisfies them and do not need to further experiment.
Dan Kleck, a winemaker from the early North Fork generation, once told me that if you want to have a wine cellar on the cheap, start a wine competition. You get 2 or 3 of each wine taste 1 or 2 and save the 3rd one and get paid for the whole exercise. A bit cynical perhaps but packed with wisdom.
Joe - Let me add an analogy here that I think is rather apt.
As a political reporter in my day job, I often hear politicians bemoaning the federal earmarking process. But then I ask if they will stop bringing pork back to their home districts. They all say they will not. They say, “The process is terrible, and it must be changed. But I’ll work within the system to make that change.”
You don’t need me to tell you whether any real change ever happens, right?
Evan,
How is a wine entered into multiple competitions until it wins something different from one that is sent to multiple reviewers? In either event, the winery will promote the top result and consumers will be largely in the dark about other, less flattering, results.
That is about marketing and the use of ratings of any sort to promote wines-not limited to wine show medals.
Your political analogy made me chuckle, but folks leaving congress have yet to make the system better, just like voter apathy concentrates power in the hands of certain-generally more extreme-factions.
Lots of good points being made. Sorry I am so late to this party.
One of the obvious reasons many wineries enter wine comeptitions, especially small wineries, is that they cannot get reviews of their wines to begin with, so the wine competitions are the only place they can get what they can account for as a review. Small wineries make big deals out of their medals, because they can’t get a review from their local paper, let alone the NY Times, the WSJ, Wine Spectator, The Wine Enthusiast, Robert Parker, or Steve Tanzer, or even a wine blog as prestigous as NYCR.
They have nowhere else to turn. The competitions rate their wines - Gold, Silver, Bronze. And that’s something they can show their customers as a rating from the outside world about the quality of their wine.
However imperfect, the competitions have been performing this service for many years.
As for their quality, it varies, just like animal judgings. What one judge puts up for Best in Show, another wouldn’t recoomend for Best in Breed. No matter whether it’s the AKC or 4H. People generally have good intentions. It’s subjective and antique. So is wine reviewing. But it’s the best we can do.
I think transparancy is really the key.
Evan,
Just to be clear before I depart the discussion and await Lorraine Hems, whom I’ve known for a long time and am sure she will contribute something smart to the conversation:
I’m not saying that wines should be evaluated only for the technicals; I’m saying that those who evaluate wine ought to be formally trained to do so, and that a set of standards by which to gauge both the evaluation and the evaluator ought to be established.
The reason this issue keeps popping up (and you guys seem to be saying it, too) is that there are no consistent standards either from competition to competition or from reviewer to reviewer.
Maybe I’m a true cynic, but I don’t believe that people are magically endowed with the skills to evaluate wine just because they drink the stuff.
Carlo - You bring up a very important point, and you state it well. It’s too bad that smaller wineries can run into trouble getting reviewed. Obviously the NYCR seeks to put up no barriers to coverage, but for major publications, it is a challenge, certainly. Thanks for reminding us about this.
Thomas - Gotcha, and good points.
One theme that I’m noticing in the comments here is how useful the medals are/can be for the wineries.
That’s not really our point here though — we think they do consumers a disservice.
The other theme of rating wines being as problematic as judging them is iffy (at best) in my mind. As Evan said already, if people come to this site and see a score, they know that I gave it. They can very easily click through the site and the reviews posted here, likely find a NY wine that they are familiar with and learn over time if our palates are similar. I’ve done the same thing over the years with WA and WS and know what critics my palate is more similar to than not.
Even if someone who sees the medals awarded at a big competition knows who awarded them (rare) they likely won’t know who the person is. And because many judges are not writers, the consumer (even if he/she tried) would have a hard time gaining an understanding of the judges preferences and track record.
This isn’t about discounting the abilities of the judges themselves. I’ve worked with outstanding palates and minds at competitions. It’s the system that is at fault.
totally agree with Jamie on this. Rather than write-off the competitions, a better play might be to only agree to those competitions that have merit.
You will never be able to control the deceptive use of the results, but that doesn’t make the process / competition itself bad.
I myself am a big critic of wine medals because I think they offer little consumer value - but we can only change that by making the awards more legit and giving consumers a better frame of reference (i.e., helping to create the Superbowl of wine competitions).
PLease do not get me started on Hodgson’s study.
Hold on a second Thomas…do I need a course from Cornell to evaluate and/or judge wine - because I can’t afford Cornell University. But I do have 22 years of professional wine experience. And I judged the recent Florida International Wine Competition.
Am I magically endowed - or just you?
Are we evaluating the hedonistic/intellectual aspects of these wines or are we like diamond-cutters looking for flaws?
Dude - At the very least, in your scenario, we’d have to do two things:
1) Ascertain what makes for a legit or good competition. What sets them apart? I’d love to hear examples. And
2) Start educating customers aggressively about medals, and push wineries to have more info handy. For example, when I see some BS gold medal for a 2006 Finger Lakes Merlot, I’ll want to ask: How many competitions did you enter this wine in, and what were the other results? If they can’t answer, I won’t have any interest in the medal they’re plugging.
Of course, they could always lie…
Jim,
Never said anyone NEEDS a course from Cornell-just suggested that sitting in on the sensory courses offered to winemakers at the AGRICULTURAL STATION (not at the university) might help develop skills for those who would be wine evaluators. I am sorry if a little learning is scary, but, to me, it is necessary.
“Are we evaluating the hedonistic/intellectual aspects of these wines or are we like diamond-cutters looking for flaws?:
And here you go again with the canard.
No point in me responding again to what I am forced to view as willful misundertsanding of my point.
“Am I magically endowed - or just you?”
Based on that snide response, you took the comment personally. I can’t vouch for your insecurities, but I know that I am not magically endowed, and never claimed to be. I know perfectly well the purpose of the personal pronoun and didn’t use one.
Argue your points, tell me why you disagree with my position, but don’t become abrasive in the process just because you meet with someone who expressed direct opinions; it’s unbecoming of a 22-year professional.
Oh, and I top you in the business by four years-I have covered as many areas of the industry as your resume claims that you have, but with the addition of having been a grape grower and winemaker. You see, I can use the personal pronoun when it’s needed-I can also do investigative journalism so as to know with whom I am talking.
Hey Jim,
If you want a fine view of me, talk to Eric Fry at Lenz. If he tells you the truth, you should have fun listening.
One point that no one has seemed to mention is that wine competitions are expensive! Paying 50-100 dollars PER WINE to enter one of these things isn’t cheap when you start entering half a dozen or a dozen wines in a half a dozen of them. Why do WS, WE and W&S see so many more wines than even the largest competitions? Partly because they’re nationally relevant (way more people read the ratings section of those magazines than the results of the Indy Wine Competition) and partly because they’re FREE to enter.
I would love to hear opinions on the economics of paying for these medals and how much more wine it sells. The Facebook comments Morgan posted about don’t scream “increased sales”
It’s an interesting post and a principled stance. But does it matter? Are wine competitions bad for wine, wineries, the public, or anyone at all?
Marketers tell me that gold medals don’t help in retail stores, but help wineries sell wine in tasting rooms. Is it really worth taking a principled stand to prevent that?
Brad - Just did a midday count on Morgen’s snap poll. 40 people said the medals would not lead them to buy. 26 said that, while the medals wouldn’t lead them to buy, they might be more willing to taste a medal-winning wine while in a tasting room. 10 said the medals would lead them to buy. Pretty strong indictment, if you want to put value in something like that.
There are indeed barriers to entry, or at least barriers to entering a large number of competitions. And I think consumers truly don’t understand that these competitions are not heavily representative of regions, and certainly not representative of the small, artisan producers. If the Minnesota Twins win the AL Central, is that as impressive as winning the World Series? Certainly not, but by wine competition standards, the Twins would just call themselves “champions”. Who knows what the competition was?
Blake - Obviously we are not taking a stand simply to prevent wineries from selling a few more bottles in their tasting rooms. And really, if medals and wine competitions were to go away completely, do you think winery staffs would have no other opportunity to increase sales? Your premise seems to be that without these competitions, sales would go down. I’d like to think that without medals, wineries would have to be ever more creative. How is that a negative? There are so many ways for them to improve what they do to reach customers. I don’t think a little kick could hurt.
For us, it’s about doing the right thing to make sure our readers and wine consumers have the best information possible. If the end result is that wineries lose medals as a marketing tool, that would likely be a net positive, as it would force them to engage in new ways to make up for any lost sales (which probably don’t amount to much, anyway.)
Hi Folks, here’s my 2 cents:
Wine competitions are just one of a few valuable tools for consumers to use in making their purchasing decisions in a marketplace that is increasingly filled with a sea of wine.
When folks encounter an aisle or 10 aisles full of wine from around the world, the choices can be overwhelming. A piece of smart POS with a gold medal or a score, or even a hand sell from knowledgeable (we hope) staff can prevent this from being a needle in a haystack proposition.
We all agree that in the realm of the subjective, evaluating wine is an imperfect endeavor and anecdotal mistakes are very easy to find, whether it’s gold for a mediocre wine or Parker whiffing on a bretty or “hot” Super Tuscan.
I fail to see how the reviewers of the NY Cork Report are any different from a panel of judges at a wine competition or even someone who reviews opera for the NY Times. Anyone can have an opinion on wine, and if a consumer takes a recommendation from a particular reviewer or wine competition or blogger and they are disappointed, then they will choose to look elsewhere for recommendations.
I would like to give 95 points to Joe and Tish for their recommendation that the NY Cork Report become re-engaged in wine competitions to improve the process, or… start your own wine competition.
Blake: We certainly aren’t trying to prevent anyone from selling anything — I’m surprised you’d even say that.
Instead we hope making this stance brings some attention to this issue — which it is doing on some level — and eventually helps people understand the reality here as we see it.
Let’s face it, while competitions can become expensive, they are easy, mindless marketing. It doesn’t take any sort of creativity or marketing savvy to slap a “GOLD MEDAL” on a tasting room sheet or a website. Enter enough competitions with even a decent wine and you’ll get your gold. But remember that just about every other winery is doing the same thing.
Christopher: You said “valuable tools for consumers to use in making their purchasing decisions”
Part of our point is that they are not valuable — not in any real way.
Sure, if confronted with two similarly priced wines, a consumer might buy the one with the gold medal sticker — but are they getting the better of the two wines? I say it’s 50-50. If they are getting the better of the two wines, great. But if they end up taking the lesser of the two wines home, then the medal has just done that consumer a disservice.
The same can be said of scores of course but even in lazy stores, a score is often at least accompanied by a tasting note. That’s not true of a gold medal sticker slapped on a bottle.
Can the same thing happen with scores? Absolutely, and that’s why people who score wine without any sort of tasting note tied to it are just as guilty.
But I keep coming back to these questions:
Which competitions are run well vs. not?
Who gave this wine this medal?
What are their credentials and track record?
How many wines did they taste during a session and where did this wine fall in the lineup?
With most scoring outlets, it’s easy to answer at least most of these questions and then I can decide whether or not to put much value in the ratings.
For instance, I know that my palate is very different from Robert Parker’s, so some WA reviews are meaningless to me really. On the other hand, David Schildknecht and I have agreed on many wines, so I follow him with more interest. The same is true on the WS side — James Molesworth and I have far more similar palates than Thomas Matthews and I have.
You simply cannot gain that level of understanding with these black box competitions where wine goes in and medals come out.
“…black box competitions where wine goes in and medals come out.”
Permission to use that phrase someday?
An interesting post right after having judged a local wine tasting by amateur winemakers myself. I can understand your point. I like to select and drink wines for the education, taste and enjoyment - not their medals. It’s a nice pat on the back to a winery to get medals in competitions against their peers, but does it say “We’re the best”? After a dozen or so wines as a judge your tastebuds will be pretty close to toast. Sip, slosh, spit, water and bread or crackers can help you last longer and somewhat keep the palate clean, but if you are tasting one big oak monster, high alc cab after another your tongue will lose! I understand the reference to having some residual sugar as a relief! As I said, I drink wine not medals. Shelf talkers and neck sleeves can give medal counts and points ratings, but when I ask folks looking for wines in the store what they are looking for they often don’t know. “Something I will like” is a common answer. Will a medal help with that?
Tom - Thanks for your honest assessment, particularly in light of the fact that you just finished judging a competition.
Let me ask you: There is a lot of talk about separating the best competitions from the lesser lights; what makes for a “better” competition? How can they improve?
I’m hoping Joe Roberts, Jamie Goode et al will also tackle that. We want to be persuaded on this count, but so far I don’t know what would separate one competition to a degree that would alleviate the concerns we’ve outlined.
I’m glad Christopher O’Gorman stopped in. He’s one of my favorite people in the wine industry; he has a great palate, he’s an outstanding judge (in my opinion!), he’s a Californian with an international understanding of wine, and he’s a hell of a lot of fun. I was fortunate to meet Chris at a judging event - one of the benefits of these competitions, to be sure.
Chris, you raise an important point here: Medals represent opinions on wine, just as scores do, or recommendations from friends. I simply worry that consumers don’t necessarily see it on those terms of parity or near-parity. The medals have been marketed as something they’re not, and marketed without context or much explanation.
That said, I always look forward to sharing a tasting experience with you, whether in a competition or otherwise!
I especially appreciate the comments by Joe Czerwinski, W. Blake Gray, and Tish.
There are three unique wine competitions with which I am involved:
Sommelier Challenge International Wine Competition
http://www.SommelierChallenge.com
Critics Challenge International Wine Competition
http://www.CriticsChallenge.com
Winemaker Challenge International Wine Competition
http://www.WinemakerChallenge.com
The judges for each are as the names suggests Sommeliers, Critics and Winemakers. The names of the judges are listed on each website. These competitions are run by Robert Whitley, nationally syndicated wine writer and publisher of http://www.WineReviewOnline.com. He is also director of San Diego International Wine Competition, which has been around for 27 years. Robert has been tweeting about this post at @WineGuru on Twitter, which is how I learned about it. This comment is my own.
Simply put, a competition medal is a recommendation, just as a high score or positive review is. It can help give the wine consumer some confidence in his purchase of an unfamiliar wine, although obviously it is not a guarantee that he will like it, since taste is subjective. Similiarly it can help the winery in marketing the wine, however untrained tasting room staff or sales people may not make best use of it.
I love wine, and feel that anything that helps bring people and wine together is a good thing, especially in these difficult times when the wine industry needs all the help it can get. Forgiven me, but I don’t see how the current trend of bashing competitions or the 100 Point Scale or any particular publication/reviewer does the wine industry or wine lovers any good.