Just a few wines to be tasted at this years Indy International Wine Competition

Photo courtesy of Dave Falchek

It’s official. We are done judging big, blind, medal-focused wine competitions.

We did not arrive at this position without much thought and discussion. Ultimately, we believe that transparency and clarity are core values that should permeate the wine world — from the creation of wine, to the marketing of wine, to the writing about wine.

Everything that happens in those areas should relate in some way to answering this question: Is this providing more transparency and clarity to the consumer, or less?

We have decided that medal-focused competitions provide less clarity and transparency to the wine consumer. We feel that medals only confuse consumers instead of educating them, and that they provide little real value.

Our position going forward will be simple: The editors and writers at the New York Cork Report will not accept invitations to judge wines at large-scale, blind-tasting events with the goal to hand out “medals” to “winning” wineries.

We want to explain, and — this is vitally important — we mean no disrespect.

The vast, vast majority of competition creators, organizers and judges perform their roles with the best of intentions. Often, we find that the wines we are best are the ones that win top honors. Anthony Road Wine Company’s 2008 Semi-Dry Riesling winning the Governor’s Cup is one example).

But that cannot and does not change the reality: There are so many medal-awarding competitions that the events have lost any sense of meaning to the average consumer, and even wine-loving consumers can’t possibly know the significance of a single bronze or silver or gold medal awarded at the many, many events. Furthermore, the very act of blind judging a wide range of wines should be viewed as a parlor game and not some official declaration of merit.

Good intentions give way to nebulous marketing

We can’t stress this enough: The organizers of wine competitions are people who constantly impress us with their enthusiasm and event planning. Collectively, we have judged at many events and have been invited to judge at many more. We admire the goal of wading through oceans of wine to sort out the very best for consumers.

The problems with judging will be addressed below.

Even the medal winners can’t explain much about the meaning of such an award. Evan recently stopped by a Finger Lakes tasting room that was drowning in medals. He was told, “Our 2006 Merlot won Silver at the So-and-So Wine Competition!” He asked the staff to explain what that meant. “Well, it probably means that the judges liked our wine very much!” they replied. He asked who the judges were. They didn’t know. He asked how many wines, by percentage, got at least a silver medal. “Oh, I don’t think it’s very many,” came one reply.

Sadly, that’s wrong, by almost any measure.

On Long Island, Lenn has been similarly regaled by tasting room staffers with stories of medals awarded — often incorrectly. He’s up on some of these things, so he often knows that they are wrong when they tell him that their riesling won gold but it actually won bronze. The average person off the street can’t possibly know; there are too many medals from too many competitions. Ultimately these medals and discussions of them have become nothing more than white noise, like static on your television.\

Medals have almost no defined meaning that the wineries themselves can even explain, let alone their consumers. Ask a consumer what a medal means — really, grab a customer in a tasting room — and there’s almost no chance they’ll be able to offer anything close to an answer describing where it comes from and why the judges awarded it.

It seems that wineries simply hope the use of medals will make their bottles more attractive. We understand the impulse. The business of wine is a competitive one, and discretionary dollars are being held tightly. But ultimately a state that is attempting to attain world-class status does itself a disservice with an over-reliance on meaningless handouts.

We can promise that almost every tasting room customer would be shocked to find out that often the standard for getting at least a bronze medal is simply to create a wine that is not mortally flawed. That’s it. That’s the baseline.

The first problem with judging: Subjectivity

At Evan’s first wine judging competition, a huge annual event that we won’t name, he remembers a debate over a flight of pinot noir. One judge refused to award a particular wine a gold medal because, in his words, “There is plenty of fruit but not nearly enough supporting oak.” Evan, understandably, was stunned. A judge demanding more oak? What next?

He didn’t have to wait long to find out. During the next flight, a judge detected a whiff of Brettanomyces in one of the wines. She decided it was a nice addition to the wine, adding character. The judge to Evan’s right was offended to the point of near-insanity. “Brett is a FLAW,” the judge declared. “And a flawed wine wins no medal.” The other judge persisted, arguing that it should be a gold medal wine. Evan thought he was about to witness a fistfight.

How can you or anyone else tell a judge how to evaluate wines? The beauty of wines is that we have the opportunity to decide for ourselves what makes a wine special. Now, that’s not to say that there aren’t clearly discernible qualities and flaws. But if I love oak and over-extraction in, say, cabernet sauvignon, and you appreciate a more restrained approach, which one of us is right? If I think the best wines are indicative of where they were made, whereas you believe the best wines are hedonistic missiles, place-be-damned, who’s correct?

If you sit in on a judge’s panel at just about any wine competition, get ready to hear the same conversations. And then ask yourself how anyone can possibly hand out medals when it’s over, as if one wine correctly identified that 7×4=28.

The second problem with judging: Blind tasting

Everyone on the NYCR team has come to love blind tasting. It is great fun. It is also a bit like a sporting event or game, not befitting the anointing of medals that ostensibly carry serious value.

There is perhaps no wine more fitting to explain this problem than Finger Lakes riesling. The best winemakers in the Finger Lakes often remind their customers that riesling is a “food wine.” It certainly is. It is versatile, ranging from dry to sweet, and pairs harmoniously with a range of dishes. Winemakers have such things in mind when crafting their products. But they are not producing rieslings designed to impress judges in sterile, blind-tasting settings.

Now try to imagine tasting dozens and dozens of these wines with hardly a bite to eat. The acids are ripping at your mouth, and in the sweeter flights the sugars seem like a welcome respite. In the cabernet flight, there is no juicy steak to accompany a rich wine, and the judges are left to consider them bereft of that partnering.

But most importantly, blind judging robs the evaluators of the most significant parts of the wine — its context.

Tell a judge he’s drinking cabernet, and he’ll immediately try to lock in and ascertain the country of origin, then the region and perhaps sub-appellation. But the mind is a funny thing. Instead of simply enjoying (or not) the wine, and thinking about it individually, the judge begins to add context where there is none provided. How did the other wines in the flight taste in comparison? What might that say about this wine? When was the last time I tasted a wine like this one? Where was it from? Should I allow myself to believe this is Bordeaux, when I’ll feel awfully silly when I’m told it’s from somewhere else?

Delving deeper, we find that judging a wine that is simply known as cabernet sauvignon is extremely constricting. We don’t want a Napa cab to taste like a Bordeaux. We expect Chile to turn out something else entirely. If we’re tasting a Bordeaux cabernet that tastes like Napa, we’re bound to be disappointed. But tasting blind, we might convince ourselves it’s from somewhere else, mistaking place and winemaker intention. Whoops.

We’ve had judges tell us that we should forget about figuring out where a wine is from and simply taste it to see if we like it. Fair enough. But in that one statement, we see exactly why wine has become so homogeneous, so dangerously banal. Judges are not required to give a damn about a wine’s sense of place.

We find it vital. With no standard, how can we expect judging to be consistent?

Ah, but see: It’s not consistent.

Not even a little.

There is ample evidence that judging is like throwing darts

When Robert T. Hodgson set out to research the reliability of judging, many of us suspected he would find that judging is inconsistent. Instead, he found that medals are awarded in a fashion that almost appears to be random. Hodgson wrote, “It is reasonable to predict that any wine earning any medal could in another competition earn any other medal, or none at all.” Indeed, he found hundreds of examples of wines that earned gold medals in one competition and no medal at all in another.

Put another way: If you make a competent wine, you can enter enough competitions and that wine will almost certainly win gold eventually.

No study is perfect, but we suspected that after this study was released, drastic changes would hit the wine judging circuit. We have yet to see any. Hodgson stated that his goal was to provide some measure of judging reliability to help these competitions improve. We see the result being supportive of the idea that these competitions ought not exist at all. After all, judging in mass competitions is putting wine into just about the least most suitable place for good evaluation and enjoyment.

And for wineries that might protest, it should be said that the little study that has been done only indicates that tasting room customers really don’t care much about medals. Why should they? As we’ve already explained, they don’t know what the medals mean.

Clarity? Consumers don’t know which wineries entered a particular competition and which didn’t, they don’t know the judges and what the judges are looking for, they don’t know how many medals were awarded, and they don’t know what a medal is supposed to signify.

That should say everything.

Our decision, and our call for others to join us

In the future we will politely decline invitations to judge at these events. That does not mean we won’t participate in wine seminars, conferences, etc. This is simply about mass judging. The wine competition circuit has become quite an industry itself, but there has to be a good explanation for the purpose it serves.

We ask our colleagues to do one of two things: Pledge to join us in this decision, or provide a suitable answer for the problems we’ve outlined above.

We’re more than willing to listen, and to change our minds if it can be proven that these competitions help the consumer.

But right now, we’re sitting them out. - The NYCR Team