Photo courtesy of Dave Falchek

By Evan Dawson, Managing Editor

"Who decided that the purpose of wine competitions was to deliver valuable information to consumers? Other than Tasters Guild, I cannot think of any." -Chris Cook, Michigan Wine Competition Superintendent, attacking our premise that wine competitions are supposed to assist consumers

"Every competition I judge begins with me or some other chief judge reminding the judges we are there to help consumers find wines to drink." -Bob Foster, director of three wine competitions and judge at many others, clearly disagreeing with Cook in the same discussion thread, after explaining that his thoughts "mirror those of Chris Cook"

"Wine competitions are not for wine writers. They're for wineries and to a lesser extent the general public." -W. Blake Gray, wine writer and judge

"I don’t see this as a loss for any wine competition. With views like they have, I wouldn’t want them in my events." -Dan Berger, highly regarded wine writer and competition director, who referred to our writing as "drivel" in his weekly wine commentary

"Guys, if Oz Clarke or Jancis Robinson or Steven Spurrier had issued such a declaration, it would be a bombshell." -New York Times writer Howard Goldberg, reminding us of our status in the world of wine writing

"I believe this is part of Evan's attempt to discredit all wine competitions." -Foster, writing in a thread on the discussion board Wine Berserkers

Whew. You can see, from the quotes above, that some wine judges and competition directors are fired up over our recent post explaining why we will no longer accept invitations to judge at wine competitions. You can also see that in their criticism of us, they occasionally contradicted one another, leaving consumers to wonder, "So are these events designed to deliver valuable information to us, or not?"

No, says Chris Cook. Yes, says Bob Foster. Perhaps, but that's not the main point, says W. Blake Gray. And these men are some of the leading lights of the wine competition circuit. It's hard to blame consumers for being confused.

Let's accept Mr. Foster's contention that wine competitions are (at least partially) designed to help consumers find wines to drink. In this post we want to look at an area that most consumers don't think about, because most consumers don't taste dozens or hundreds of wines in a single day (or half day!). 

Before we delve further, though, let's offer a little self-examination.

What did we get wrong in our original post?

Plenty! It was our intention to explain to our readers why we no longer intended to accept invitations to judge at large-scale competitions. But Howard Goldberg was right: Who the heck are we, anyway? The post should not have been about us and our decision. That was a silly mistake, and I don't blame wine judges and competition directors if they took it as perhaps a bit arrogant. We apologize for that.

Also, Dan Berger wrote, "The long, curious article had some vague criticism of all wine competitions as if they are all the same." He's wrong about the article being vague — it was highly specific, after all — but he's absolutely right to criticize us for grouping all wine competitions together. There are now so many wine competitions across the country, and for us to imply that their standards are all the same was unfair. 

With that in mind, let's do something we should have done the first time around. Let's highlight some of the best examples of wine judging competitions.

Careful judge selection, thoughtful structure, food, and helpful websites for consumers

Robert Whitley, of the excellent website Wine Review Online, is helping set new standards for wine competition websites. First, though, he deserves praise for his innovative approach to wine competitions: Whitley runs several events and each event features judges from a singular field. There is the Sommelier Challenge, the Critics Challenge and the Winemaker Challenge.

Each website has a complete list of judges. Then, each website not only lists the results, but lists which particular judge contributed which medals, including comments and some tasting notes. The wineries are listed alphabetically, allowing readers to see which of their wines won medals. Whitley has answered every question I've had and our disagreements have been spirited and pleasant. I encourage you to check out WRO regularly, too — there is a fine list of writers there including Tina Caputo, who penned this pithy dissent to our original post. 

Dan Berger runs two competitions in California, and in my communication with Dan, I can say I know of no one more meticulous about the details of their events than Dan is about his. You have to admire his insistence on four-judge panels instead of the often-used 5-person, majority rules system (he demands discussion from his judges). And who can't appreciate a wine competition that gives out a Terroir Award?

Christopher Cook demonstrates what makes for the strongest competition directors: He's careful about the order of the wines poured and tasted, taking into varieties, sugar level, and the sheer volume of wine to be tasted by his judges. He runs the Michigan Wine Competition.

Bob Foster's Mid-American Wine Competition lets the wineries decide if they'd like to have their wines tasted with food! "The wineries can choose to have their wines judged with any of six dishes prepared by chefs, or in the traditional wine vs. wine format," Foster wrote. The competition has a strong website, featuring the most comprehensive section on wine judges I've yet seen. If a consumer wants to know about the people who awarded the medals, Foster makes sure they have a catalog of information. And he gets credit for his competition's head judge, the great…

Doug Frost is one of my favorite wine people anywhere in the world. If you've somehow not heard of him, watch this video and tell me you don't love his style. Doug is the rare person who can go utterly erudite and intellectual about wine (he's one of three people in the world to have passed the Master Sommelier exam while also earning Master of Wine status) and is also capable of speaking to first-time drinkers in a real, connectible fashion. He directs the Jefferson Cup, and the website lists not only recent winners, but all winners for the past five years.

One concern we have with competitions run by this estimable group — and this seems to relate to all wine competitions, best we can tell — is that consumers can not find out which wines were submitted and judged not to be worthy of medals. Only the list of medal-winning wines are accessible, and generally more than half the wines submitted to competitions win medals.

So the list of wines deemed unworthy of a medal would not be overbearingly long to publish, but I've been told that publishing that list would be embarrassing to the wineries who made those wines. Our view is that consumers who trust judges to help them choose medal-winning wines would also want to trust judges to help them avoid spending money on unworthy wines.

Palate fatigue is not a myth, and remains a major concern for us

After Dan Berger bashed our post in his weekly wine commentary, I followed up via email with Dan. (By the way, if you're not reading Dan Berger on a regular basis, we humbly believe you should. He has long offered one of the unique perspectives on wine and we often agree with his preference for elegant, balanced wines. You can read a sample issue of his work here.)

I sent him a short list of questions, including the following: Research has indicated that palate fatigue sets in after about a dozen wines. How do judges counter that — or can it be countered?

To that question, he sent a short response, reprinted in full: "I have never seen such research, and believe strongly that palate fatigue is a myth. If such research has been done, can you cite where you found it and who did it?"

More on the research in a moment. The only other writer I had ever heard refer to palate fatigue as "a myth" was Jay Miller of the Wine Advocate. Miller has commented, "The palate fatigue argument, frankly, is total hogwash. You taste, you spit, you write a note, taste again, spit, add (or not to your note) and on to the next wine. When you’ve had practice doing this, it’s simply not difficult."

Let's frame this discussion with a single, essential question: If the last wines of the day are judged with even a small decline in palate and olfactory freshness, is that fair? Or, if you prefer: Should we expect and require judges to have the same palate and olfactory freshness throughout the duration of the judging event?

Several judges have correctly pointed out that with practice, tasters can condition themselves to last longer before feeling fatigue. And several judges have used the analogy of running a marathon: a 100-yard sprinter can't just run a marathon without training, but with preparation, he or she can become strong enough to finish.

But this analogy doesn't exactly work because even the most experienced marathon runners feel differently — more tired — toward the end of the race. The point of running a marathon is to finish the race, not to admire the scenery — having just run a half-marathon Sunday, I can relate to that!

The point of tasting a hundred wines or two hundred wines, though, isn't merely to finish tasting. You're supposed to admire the scenery, not just power through. That's why you're doing the exercise in the first place, to report back on what you saw (tasted). If you ask a marathon runner to give an architectural critique of the building he passed at the 25-mile mark (or the 1-mile mark or the 10), I doubt his answer will be as valuable as that of the person who strolled by, sat on the nearest park bench, and admired the building for an hour. So the useful question to ask is not whether it is physically possible to taste a hundred wines in one sitting, but whether tasting a hundred wines in one sitting adequately serves the original purpose of doing so: to evaluate them fairly and intelligently.

Science and research tells us that just like the marathon runner losing pace and energy as the race progresses, palates do indeed get fatigued while tasting, and even our olfactory senses are affected. Let's look at the research.

Tasting fatigue

The University of Minnesota recently performed a study on the buildup of astringency in our mouths while tasting. Tannic wines have a lot of astringency, for example. Does it build up in our mouths? Can we counteract that by eating bread or drinking water? I encourage you to read this study.

The researchers used a handful of palate cleansers to see if they could "clear up" the buildup of astringency or tannins. Those palate cleansers included water, carboxymethylcellulose, crackers, milk, chewing wax or nothing.  

What they found is that it becomes increasingly difficult to clear the mouth after repeated exposure to tannins/astringency. And while some of the palate cleansers were better than others (water and doing nothing at all worked best!), it is impossible to entirely avoid the buildup of astringency. I'd like to see olives, a popular cleanser, used in future studies.

But, in other words, palate fatigue happens. If it can be mitigated, it can't be stopped.

In a subsequent email, Dan Berger clarified that he does not believe palate fatigue is a myth, but he maintains its existence can't be proven, which is why he described it as a myth. Here are Mr. Berger's comments on the matter:

Every human is genetically different from every other human. Some people who are unskilled at tasting wine will find that they cannot distinguish much after 12 wines (as you indicated.) Others who are skilled at tasting wine can do a creditable if not exemplary job tasting 100 or more wines a day, at a very high level of acuity. It depends on the ability of the palate to be sufficiently trained as well as other factors (which I alluded to in an earlier message about saliva regeneration, for example). I, for instance, can taste well over 100 wines without a huge drop-off of skill. It depends to a great degree on how that tasting is conducted. I once did 252 Chardonnays on one day, no thanks to the organizer of the San Francisco Wine Competition, and did what I think was a good job. (But the format of the event was vital to me saying this,.) But would it have been better had I done that number of wines in a four-day period? Well, that all depends. What if I had a cold on the fourth day? What if my "biorhythms" were off on one of the days? Does the mental state of the taster have anything to do with this? Of course. And not all Mondays are equal to all Tuesdays.

Olfactory fatigue

It's not just our mouths that will be affected by large tasting formats. Research shows that what we perceive in a wine's aromatics will be impacted by what we've previously smelled.

Think of it this way: Someone with 20 cats gets accustomed to the smell of cat litter — it no longer registers with them. That's adaptation. In a flight of 12 red wines that are heavily oaked, the first wine will seem oaky. By the final wine, the impact of the oak on the nose will be much different. And that's just the extreme example.

Slate's Mike Steinberger visited the Monell Chemical Senses Center in Philadelphia, where he met Dr. Charles Wysocki, an expert on olfaction. Steinberger wrote, "Wysocki expressed admiration for the stamina, aroma-identification skills, and descriptive abilities of wine critics, but he was skeptical about some aspects of the trade. He said it's impossible to taste dozens of wines in rapid succession and not suffer olfactory fatigue and that anyone who claims otherwise is claiming to 'defy biology,' as he put it. Although a critic might think that his sense of smell is still acute after sampling 40 Cabernets, his impressions at that point are being formed less by the nose than by past experience, visual cues (such as the color of the wines), and perhaps also tactile sensations."

Wine researcher and writer Jamie Goode, writing in Questions of Taste: The Philosophy of Wine, reported on experiments by University of Oxford Prof. Edmund Rolls. Goode wrote, "Rolls was cautious about speculating (as many scientists rightly are), but agreed that this could have some effect during a wine tasting where a taster is repeatedly encountering the same sort of taste or smell. At a large trade tasting it is quite common to taste as many as 100 wines in a session. If these results of sensory-specific satiety are extrapolated to this sort of setting, then it’s likely that the brain will be processing the last wine you taste differently from that of the first, assuming that there are some components to the taste or smell in common—for example, tannins, fruit or oak." (Bold is mine.)

I also addressed this with Mr. Berger, who seemed to agree that olfactory fatigue is taking place — before diverging to talk about the value of a wine's complexity. He writes:

First let's look at The good doctor, who is quoted as saying it's "impossible to taste dozens of wines in rapid succession." I assume that that the term "rapid succession" means one wine after another. But if there is a 90-second to two-minute break between the wines, is that sufficient time for the nose to regenerate? There was a sensory specialist at UC Davis, the late Rosemary Pangborn, who did a lot of research on this and found that 90 seconds was needed between wines for the nose to regenerate all (or substantially all) of its capabilities.

I followed up to ask Mr. Berger is he instructs his judges to wait 90 seconds between wines during a judging event, as I have never seen that practice employed. I have not heard back from him about that. 

What the experts say: Large tastings and blind tasting

We don't hide the fact that we view blind tasting as an extremely useful component in evaluating wine — but only a component. Beauty is difficult to discern in large tasting formats. Add to that the challenge of dealing with palate and olfactory fatigue, and it's a concern.

As Howard Goldberg has noted, we're not the well-known names that command respect in wine writing. But there are some outstanding and respected writers who cite palate fatigue as a real concern in such formats.

Steve Heimoff, author and longtime California-based wine writer for Wine Enthusiast, says, "I used to power-taste until I realized it wasn’t fair to the wines or to me. The palate gets fatigued and can only respond to the biggest, most extracted wines. Subtlety is lost. Perhaps that’s why Parker gives his highest scores to the biggest wines. Nowadays I limit myself to about 12 wines a day, and actually have the time to think about each one, the way it deserves to be considered."

Charlie Olken, the man behind the outstanding Connoisseurs' Guide to California Wine, tells me, "Those events are, to my way of thinking, exercises in futility. I simply do not believe that most tasters, even professional tasters like Dan Berger and myself, can sit through a tasting of 120 Sauvignon Blancs or 75 Petite Sirahs, etc. and make defensible decisions. I come to that position having subjected myself to plenty of those events in the past. I wound up tired, bored, burned out and eventually at odds with the selections of wines that stood out because they were different, not better."

Joel Goldberg, Editor of MichWine.com and a judge at the Michigan Wine Competition, says, "Not only is palate fatigue an issue, so is alcohol absorption through tongue and mouth, even spitting everything."

Regarding blind tasting as its own measure of a wine's quality, Mr. Steinberger writes, "Tasting blind doesn't necessarily make for better wine criticism. If you don't know the wine's name, you also don't know its back story — how it was made and how it has tended to evolve in prior vintages. These are important considerations, particularly when appraising younger wines."

Eric Asimov, New York Times chief wine critic (and almost no introduction needed), writes, "Maybe as wine drinkers we’re all a little more grown up now and don’t need to taste blind all the time. We’re not going to set aside blind tasting. At least, not yet. But I think it’s time to consider whether it’s always the best way to judge wines. Consider the question asked."

Terry Theise, importer, rogue thinker and author of the new book Reading Between the Wines, writes, "What is the value of reducing wine to a thing without context? What game is this we're insisting wine play along with? What's the good of tasting blind? Where's the silver lining of experiencing wine in a vacuum? Yes, it can train us to focus our palates and hone our powers of concentration. Then we can discard it! It has served its purpose. If we persist in tasting blind we run a grave risk — because it is homicidal to a wine's context, and wine without context is bereft of meaning… Blind tasting will confuse you as to the purpose of drinking wine. The only genuinely professional approach to wine is to know as much about it as possible. Who made it, under what conditions, what are the track records of the site and the vintner — then and only then can a genuinely thoughtful evaluation of a wine take place in the fullness of its being."

Alder Yarrow, creator of Vinography (one of the top wine blogs), supports large blind tastings but hates the medal system. Alder has called wine competitions "one big racket" and "vaguely evil." But he recently judged in a competition with no medals that allowed the public to choose the category winners.

To be fair, Thomas Matthews, of Wine Spectator, says, "Blind tasting may not always be the best way to understand a wine, and it is often humbling, even for the most experienced tasters, but I think it is the most objective method of evaluating one."

Conclusions

Contrary to what some judges have said, we are not out to discredit wine competitions. In fact, in light of the challenges posed by factors like palate fatigue, we're impressed by the careful approaches taken by some of the best competitions in the country. These competitions deserve praise, not derision, for their efforts.

Everyone will have a different standard to view these events. If we find it unfair that not all wines judged will be evaluated with the same clear palate and nose, then it seems that these events are fundamentally unfair to the wines entered — through no fault of the hard-working judges. As Dr. Wysocki notes, it's impossible to defy biology. And no matter how much judges want to insist they can withstand the palate assault of dozens or hundreds of wines, there are undeniable effects.

But it's more nuanced than dismissing competitions because not all wines are judged with the same acuity. The best competitions build in breaks and take measures to mitigate fatigue. It's fair to say that while the judges will suffer from some fatigue, they do good work under the circumstances.

Would we judge, knowing that no matter the training, we would experience the kind of fatigue that would prevent us from bringing the same fresh approach to the final wine as we had for the first? We would not. But do we condemn judges for making the effort, for putting in the time, and for doing the job? Absolutely not.

We hope this discussion does more to educate consumers about the process and its challenges. We'd still like to see competitions release the full list of wines entered; they should have nothing to fear if they want consumers to value the full results. But make no mistake: When it comes to wine competition directors and judges, they have earned the respect for this work, and they certainly have it from us.