How important are wine medals and how much can we rely on those who assign them?

By Armando Corsi, Research Associate - Lecturer, Ehrenberg Bass Institute for Marketing Science, School of Marketing University of South Australia.
Source: November/December 2012 Wine & Viticulture Journal.

Medals. Awards. In the last 30 years the system of a rating a country's best wines has been subject to several critiques, but has also been a source of publicity and promotion for both the wineries who submit their wines to these events and for wine judges and journalists, as a never-ending source of material to taste and comment.
Medals and awards in the wine sector have certainly contributed to the popularisation and diffusion of wine, allowing consumers to quickly discern special wines from ordinary ones, thus shortening the time in choosing a wine, and reducing their risk of making a bad choice. At the same time, more knowledgeable wine drinkers and/or those who always want to 'think different' have relied little on these marketing tools, believing they can judge wines better than those that try wines almost every day.
It is true that wine shows, guides and magazines have had some bugs in their system of rating and classifying wines. Perhaps the best example of this occurred in 2008 when US journalist Robin Goldstein entered Wine Spectator's Restaurant Awards program for the reserve wine list of a Milanese restaurant, Osteria L'Intrepido. Where was the trick? The restaurant did not actually exist, never did, and, probably even worse than this, the wine list was created from a range of Italian wines which had been poorly rated by Wine Spectator judges. The scandal echoed beyond wine professionals, and together with claims about the objectivity of wine shows, it is not difficult to imagine that wine judges have a substantial amount of work to do to defend their professionalism.
What does scientific research have to say about wine shows? The topic has been analysed by researchers from two main perspectives. First, academic research has observed the role of medals and awards in driving consumers' choices. Secondly, researchers have investigated the consistency of wine judges in evaluating wine across different wine shows.

The importance of medals and awards on wine purchases
Contrary to expectations, there has been little research done on the importance of medals and awards. Interestingly, though, all these studies have been conducted using the same methodology - discrete choice experiments (DCE) - and most of the papers are relative to purchases in a retail environment.
The first study of this kind does not come from a traditional wine-consuming country, but from the Czech Republic, where Orth and Krska (2002) interviewed 69 respondents in local wine shops (vinotekas). Their answers to Likert-scale type questions indicated that medals do not appear to be very important for Czechs, but their answers to a choice experiment confirmed they gave equal importance to price and the presence of awards on labels. The respondents always preferred wines showing no awards the least, and in most cases, wines with gold medals were preferred over those with silver medals.
The first study conducted in Australia was published in 2006 by Lockshin et al. using a sample of 250 Australian regular wine drinkers. The results showed that that low-involved wine consumers tended to react more positively to gold medals on wines sold at lower price points, but the effect decreased as prices rose. Highly-involved consumers relied less on gold medals, but their purchase decisions were somewhat influenced by them at lower prices. Furthermore, for highly-involved consumers, the presence of medals only slightly increased the chance of purchase for wines from well-known regions, but they certainly helped low involved wine consumers to choose wines from less known areas.
Moving out of Australia, Atkin et al. (2007) found that when US consumers are unsure about what wine to buy, women were more likely to rely on medals and awards than men in making a wine choice.
To date, the most comprehensive study about the importance of wine labels/awards was published by Dr. Steve Goodman (2009) in the International Journal of Wine Business Research. Goodman's research was conducted with 15 other researchers from 13 different countries on the elements driving consumers' choices in a retail environment. The study applied a particular type of DCE - the best:worst (BW) method - on more than 2500 consumers, finding that medals and awards were, on average, only the eighth out of 13 elements driving consumers' choices. A few exceptions were in New Zealand - where medals and awards ranked as the third most influencing element - and China and Taiwan - where medals and awards came in fifth place.
It is more difficult to find papers on the importance of wine labels in the on-premise sector, as wine bottles are not generally shown to clients before they make a choice. The only way wineries could find it useful to promote medals and awards in on-premise venues is when restaurateurs decide to include this information in their wine lists, but we know that this is not a popular option in Australia or overseas. To the best of our knowledge, the only paper that discusses the importance of medals and awards in restaurants was published by Corsi et al. (2012) as part of a wider study on the elements influencing wine choices via menus. The study was conducted using a representative sample of 1258 Australian wine consumers. The results showed that grape varietals are key choice drivers, followed by the awards obtained by a wine and its price.

The consistency of wine judges
The second research stream focusses on the concordance between wine judges in evaluating wines submitted to competitions. Since the seminal paper by Orley Ashenfelter (2006), all the authors working on this topic have agreed on the lack of consistency in wine judgments across wine shows. Hodgson (2008) analysed wine judge performance at the California State Fair Wine Competition between 2005 and 2008. Panels of four expert judges - between 65 and 70 judges were tested each year - received a flight of 30 wines using triplicate samples poured from the same bottle. The results showed that only 10 percent of the judges were able to replicate their score within the same medal group, and another 10%, on occasion, scored the same wine bronze to gold. However, the study also found that the judges were more consistent with regard to the wines they didn't like, rather than those they did.
The author also tried to explain the reasons behind these variations, hypothesising that wine evaluation can be influenced by two main factors: 1) the quality of the wine, and 2) the bias of the judge. In 50% of the wine analysed, the variation in wine evaluations was only and exclusively determined by the quality of the wine. However, for the other half of the wines, the biases in the judges' evaluations influenced the different scores received by the wines.
One year later, Hodgson (2009) published another paper on the consistency of wine evaluations across 13 wine competitions in the US. The data were relative to 2003 and consisted of more than 4000 wines. Of the 2440 wines entered in more than three competitions, 47% received gold medals, but 84% of those same wines did not receive an award in another competition.
In Australia, the only study on the consistency of wine judges has been recently published by Allen and Germov (2011). The authors reviewed the scores received by more than 5000 wines entered in four capital city wine competitions in 2007. The results showed that large wineries are more likely to enter wines in these competitions than small wineries. Similar to Hodgson (2008, 2009), there was only a moderate degree of agreement between judges in terms of the medals awarded to wines entered into multiple competitions. Disagreement among the judges was most evident in the distinctions between different medal classes, and much less pronounced between wines that received medals and those that did not.
In conclusion, wine competitions are not evil, but they are not the solution to all wine marketing problems either. They have certainly helped the diffusion and popularisation of wine and they facilitate the choice of wine, especially among wine novices. We might also speculate on the fact that the awards received in a wine show might stimulate word-of-mouth and increase the popularity of a wine, but these aspects have not been scientifically studied yet. At the same time, neither producers nor consumers can or should completely rely on this marketing tool, as wine judges' evaluations have been found to be quite inconsistent across shows, years, and countries. It would be interesting to understand what role 'official' wine judges and critics are going to have in the next few years. In a world where websites, blogs and social media are increasingly dominating the scene, does the wine sector need wine shows as we know them now, or should wineries be more in touch with online wine writers and increase their fame and appreciation through some new forms of web-based wine awards?

Allen, M. P. and Germov, J. (2011) Judging taste and creating value - The cultural consecration of Australian wines. Journal of Sociology 47(1):35-51.
Ashenfelter, O. (2006) Tales from the crypt: Bruce Kaiser tells us about the trials and tribulations of a wine judge. Journal of Wine Economics 1(2):173-175.
Atkin, T.; Nowak, L. and Garcia, R. (2007) Women wine consumers: Information search and retailing implications. International Journal of Wine Business Research 19(4):327-339.
Corsi, A.M.; Mueller, S. and Lockshin, L. (2012) Let's see what they have… What consumers look for in a restaurant wine list? Cornell Hospitality Quarterly 53(2):110-121.
Goodman, S. (2009) An international comparison of retail consumer wine choice. International Journal of Wine Business Research 21(1):41-49.
Hodgson, R. T. (2008) Evaluation of wine judges at a major US wine competition. Journal of Wine Economics 3(2):105-113.
Hodgson, R. T. (2009) An analysis of the concordance among 13 US wine competitions. Journal of Wine Economics 4(1):1-9.
Lockshin, L.; Jarvis, W.; d'Hauteville, F. and Perrouty, J.P. (2006) Using simulations from discrete choice experiments to measure consumer sensitivity to brand, region, price and awards in wine choice. Food Quality and Preference 17(3/4):166-78.
Orth, U.R. and Krska, P. (2002) Quality signals in wine marketing: The role of exhibition awards. International Food and Agribusiness Management Review 4:385-397.
Schamel, G. (2003) A hedonic pricing model for German wine. Agrarwirtschaft 52(5):247-254.

Other feature articles

AB Mauri



WID 2017