|Grapegrower & Winemaker||Wine & Viticulture Journal||Wine Industry Directory||
||Daily Wine News||
How important are wine medals and how much can we rely on those who assign them?
By Armando Corsi, Research Associate - Lecturer, Ehrenberg Bass Institute for Marketing Science, School of Marketing University of South Australia.
Source: November/December 2012 Wine & Viticulture Journal.
Medals. Awards. In the last 30 years the system of a rating a country's best wines has been subject to several critiques, but has also been a source of publicity and promotion for both the wineries who submit their wines to these events and for wine judges and journalists, as a never-ending source of material to taste and comment.
Medals and awards in the wine sector have certainly contributed to the popularisation and diffusion of wine, allowing consumers to quickly discern special wines from ordinary ones, thus shortening the time in choosing a wine, and reducing their risk of making a bad choice. At the same time, more knowledgeable wine drinkers and/or those who always want to 'think different' have relied little on these marketing tools, believing they can judge wines better than those that try wines almost every day.
It is true that wine shows, guides and magazines have had some bugs in their system of rating and classifying wines. Perhaps the best example of this occurred in 2008 when US journalist Robin Goldstein entered Wine Spectator's Restaurant Awards program for the reserve wine list of a Milanese restaurant, Osteria L'Intrepido. Where was the trick? The restaurant did not actually exist, never did, and, probably even worse than this, the wine list was created from a range of Italian wines which had been poorly rated by Wine Spectator judges. The scandal echoed beyond wine professionals, and together with claims about the objectivity of wine shows, it is not difficult to imagine that wine judges have a substantial amount of work to do to defend their professionalism.
What does scientific research have to say about wine shows? The topic has been analysed by researchers from two main perspectives. First, academic research has observed the role of medals and awards in driving consumers' choices. Secondly, researchers have investigated the consistency of wine judges in evaluating wine across different wine shows.
The importance of medals and awards on wine purchases
Contrary to expectations, there has been little research done on the importance of medals and awards. Interestingly, though, all these studies have been conducted using the same methodology - discrete choice experiments (DCE) - and most of the papers are relative to purchases in a retail environment.
The first study of this kind does not come from a traditional wine-consuming country, but from the Czech Republic, where Orth and Krska (2002) interviewed 69 respondents in local wine shops (vinotekas). Their answers to Likert-scale type questions indicated that medals do not appear to be very important for Czechs, but their answers to a choice experiment confirmed they gave equal importance to price and the presence of awards on labels. The respondents always preferred wines showing no awards the least, and in most cases, wines with gold medals were preferred over those with silver medals.
The first study conducted in Australia was published in 2006 by Lockshin et al. using a sample of 250 Australian regular wine drinkers. The results showed that that low-involved wine consumers tended to react more positively to gold medals on wines sold at lower price points, but the effect decreased as prices rose. Highly-involved consumers relied less on gold medals, but their purchase decisions were somewhat influenced by them at lower prices. Furthermore, for highly-involved consumers, the presence of medals only slightly increased the chance of purchase for wines from well-known regions, but they certainly helped low involved wine consumers to choose wines from less known areas.
Moving out of Australia, Atkin et al. (2007) found that when US consumers are unsure about what wine to buy, women were more likely to rely on medals and awards than men in making a wine choice.
To date, the most comprehensive study about the importance of wine labels/awards was published by Dr. Steve Goodman (2009) in the International Journal of Wine Business Research. Goodman's research was conducted with 15 other researchers from 13 different countries on the elements driving consumers' choices in a retail environment. The study applied a particular type of DCE - the best:worst (BW) method - on more than 2500 consumers, finding that medals and awards were, on average, only the eighth out of 13 elements driving consumers' choices. A few exceptions were in New Zealand - where medals and awards ranked as the third most influencing element - and China and Taiwan - where medals and awards came in fifth place.
It is more difficult to find papers on the importance of wine labels in the on-premise sector, as wine bottles are not generally shown to clients before they make a choice. The only way wineries could find it useful to promote medals and awards in on-premise venues is when restaurateurs decide to include this information in their wine lists, but we know that this is not a popular option in Australia or overseas. To the best of our knowledge, the only paper that discusses the importance of medals and awards in restaurants was published by Corsi et al. (2012) as part of a wider study on the elements influencing wine choices via menus. The study was conducted using a representative sample of 1258 Australian wine consumers. The results showed that grape varietals are key choice drivers, followed by the awards obtained by a wine and its price.
The consistency of wine judges
The second research stream focusses on the concordance between wine judges in evaluating wines submitted to competitions. Since the seminal paper by Orley Ashenfelter (2006), all the authors working on this topic have agreed on the lack of consistency in wine judgments across wine shows. Hodgson (2008) analysed wine judge performance at the California State Fair Wine Competition between 2005 and 2008. Panels of four expert judges - between 65 and 70 judges were tested each year - received a flight of 30 wines using triplicate samples poured from the same bottle. The results showed that only 10 percent of the judges were able to replicate their score within the same medal group, and another 10%, on occasion, scored the same wine bronze to gold. However, the study also found that the judges were more consistent with regard to the wines they didn't like, rather than those they did.
The author also tried to explain the reasons behind these variations, hypothesising that wine evaluation can be influenced by two main factors: 1) the quality of the wine, and 2) the bias of the judge. In 50% of the wine analysed, the variation in wine evaluations was only and exclusively determined by the quality of the wine. However, for the other half of the wines, the biases in the judges' evaluations influenced the different scores received by the wines.
One year later, Hodgson (2009) published another paper on the consistency of wine evaluations across 13 wine competitions in the US. The data were relative to 2003 and consisted of more than 4000 wines. Of the 2440 wines entered in more than three competitions, 47% received gold medals, but 84% of those same wines did not receive an award in another competition.
In Australia, the only study on the consistency of wine judges has been recently published by Allen and Germov (2011). The authors reviewed the scores received by more than 5000 wines entered in four capital city wine competitions in 2007. The results showed that large wineries are more likely to enter wines in these competitions than small wineries. Similar to Hodgson (2008, 2009), there was only a moderate degree of agreement between judges in terms of the medals awarded to wines entered into multiple competitions. Disagreement among the judges was most evident in the distinctions between different medal classes, and much less pronounced between wines that received medals and those that did not.
In conclusion, wine competitions are not evil, but they are not the solution to all wine marketing problems either. They have certainly helped the diffusion and popularisation of wine and they facilitate the choice of wine, especially among wine novices. We might also speculate on the fact that the awards received in a wine show might stimulate word-of-mouth and increase the popularity of a wine, but these aspects have not been scientifically studied yet. At the same time, neither producers nor consumers can or should completely rely on this marketing tool, as wine judges' evaluations have been found to be quite inconsistent across shows, years, and countries. It would be interesting to understand what role 'official' wine judges and critics are going to have in the next few years. In a world where websites, blogs and social media are increasingly dominating the scene, does the wine sector need wine shows as we know them now, or should wineries be more in touch with online wine writers and increase their fame and appreciation through some new forms of web-based wine awards?
Allen, M. P. and Germov, J. (2011) Judging taste and creating value - The cultural consecration of Australian wines. Journal of Sociology 47(1):35-51.
Ashenfelter, O. (2006) Tales from the crypt: Bruce Kaiser tells us about the trials and tribulations of a wine judge. Journal of Wine Economics 1(2):173-175.
Atkin, T.; Nowak, L. and Garcia, R. (2007) Women wine consumers: Information search and retailing implications. International Journal of Wine Business Research 19(4):327-339.
Corsi, A.M.; Mueller, S. and Lockshin, L. (2012) Let's see what they have… What consumers look for in a restaurant wine list? Cornell Hospitality Quarterly 53(2):110-121.
Goodman, S. (2009) An international comparison of retail consumer wine choice. International Journal of Wine Business Research 21(1):41-49.
Hodgson, R. T. (2008) Evaluation of wine judges at a major US wine competition. Journal of Wine Economics 3(2):105-113.
Hodgson, R. T. (2009) An analysis of the concordance among 13 US wine competitions. Journal of Wine Economics 4(1):1-9.
Lockshin, L.; Jarvis, W.; d'Hauteville, F. and Perrouty, J.P. (2006) Using simulations from discrete choice experiments to measure consumer sensitivity to brand, region, price and awards in wine choice. Food Quality and Preference 17(3/4):166-78.
Orth, U.R. and Krska, P. (2002) Quality signals in wine marketing: The role of exhibition awards. International Food and Agribusiness Management Review 4:385-397.
Schamel, G. (2003) A hedonic pricing model for German wine. Agrarwirtschaft 52(5):247-254.
Other feature articles
- Natalie Fryar, Winemaker, Jansz Tasmania
- Angelo Puglisi, Ballandean Estate, Queensland
- Dave Cleary, West Cape Howe Wines, WA
- Kim Chalmers, Chalmers Nurseries, NSW
- Tiffany Nugan, Nugan Estate, NSW
- Doug Bowen, Bowen Estate, SA
- Drew Brent-White, Windance Estate, WA
- Ian Hollick, Hollick Wines, SA
- Andrew Nugent, Bird in Hand Winery, SA
- Jim Chatto, Pepper Tree Wines, NSW
- Tony Keys, The Key Files
- Forum in pursuit of Pinot excellence
- Andrew Naylor, Pernod Ricard, NZ
- Samantha Scarratt, Fishtail Vineyards, New Zealand
- Adam Hooper and Elena Golakova, La Curio, SA
- Kathleen Quealy, T’Gallant/Balnarring Vineyard/Quealy Wine, VIC
- Richard Smart, Tamar Ridge Wines, TAS
- Terry Lee
- Ben Glover, Wither Hills, NZ
- David Fonseca Guimaraens, Fonseca and Taylor’s Port, Portugal
- Sam Temme, Lloyd Brothers, SA
- Peter May
- Colin Kay, Kay Brothers Amery, SA
- Mark Deegenaars, Sirromet Wines, QLD
- Peter Dry, University of Adelaide, SA
- Simon Thistlewood, Bimbadgen Estate, NSW
- David Lehmann, Barossa Valley, SA
- Mark Lloyd, Coriole Vineyards, McLaren Vale, SA
- Tom Harvey, Chalk Hill, SA
- Albarino – potentially Australia’s great white hope
- Ian Hendy, Tahbilk, VIC
- Oak trials instigated to create the right balance
- Jason Conti, Paul Conti Wines, WA
- Swan Valley goes organic in its approach to wine production
- 100-year-old vines saved from destruction
- Paul Boulden, Margaret River, WA
- Pinot trophy wine a close call
- Grenache finds its place in the spotlight
- Ashley Ratcliff, Yalumba Wine Company, SA
- Lessons from a fiery day in February
- Prolific Penfolds takes a double triumph
- Great win for Tatachilla Shiraz
- Ian Long, Yarraman Estate, NSW
- Capercaillie looks to future
- Tolley leaves AWBC in good shape
- Julian Parrot, Mandala Wines, NSW
- Yarra Valley’s Sticks grows up
- Rebuilding Bianchet Winery
- Kalleskes take organic grapegrowing to heart
- Vineyards benefit from WWOOF program
- Organic producer in touch with the earth
- BackVintage adopts integrated IT solution
- Lark Hill achieves full biodynamic certification
- Joseph Gilbert of Pewsey Vale – early maker of the classic Australian blend
- Sam Statham, Rosnay Wines, NSW
- Young achiever to study in USA
- Evolving Durif at Morris Wines
- Diane Miller, The Vintage Wineworx, WA
- NZ’s star producer guided by the cosmos
- On the rise: Pinot Gris secures its place
- Killeen wins Winetitles’ scholarship
- Rebecca Wilson, Tamar Valley, TAS
- Liz Riley, Vitibit, Hunter Valley
- The Gilberts of Pewsey Vale: the next generation
- Mark Cairns, Riverside Wines, Hawke’s Bay
- Craigow wins Tasmanian Vineyard of the Year
- Cornwall’s Camel Valley sets Sparkling pace in the UK
- Fred Peacock, Bream Creek Vineyard, Tasmania
- Arneis a winner, no matter how you say it
- WA’s Vanya Cullen named ‘woman of the year’
- McWilliams Mount Pleasant Estate award-winning cellar door
- Planning eases heatwave burden for Mount Horrocks
- Supplier of the Year proves customer service goes a long way
- Meet Ken Murchison
- The journey of Tempranillo to Australia
- Successful events have wineries bursting into song
- Jim Barry Wines winemaker chooses biodynamic option
- Meet Belinda Gould
- Hard grind is paying off for Sangiovese
- Biodynamic viticulture benefits Nazaraay Estate Winery
- Monitoring the key to reducing water use
- Consistently improving the quality and reputation of Australian Cabernet
- Making sure vines are true to type
- Lessons from the drought
- A story of Cuban cigars and a good nose
- A Murray Valley winery has released one of Australia’s first 100% carbon-neutral wines
- John Casella: The brains behind the brand
- Quality Tasmanian Pinot Noir stems from varying degrees of stalk removal
- Young leader forecasts positive future for Australian viticulture businesses
- China’s light shines on Aussie export radar
- Driving the blue bus of industry exposure
- Noble wine proves sweet for Australian wine industry
- Solar energy schemes lack uniformity in Australia
- Solutions for the wine industry’s fiddlier labelling jobs
- Carbon neutrality: The new black
- An understanding of excellence: James Irvine and his life in wine
- Can Australia overcome a harsh reality in the US
- Cape Mentelle treats itself to first place
- Aussies export their expert advice to China
- Bruce Tyrell: the Don Quixote of Semillon
- Why the past could help unlock grape’s future
- Brazil opportunities beckon
- Cool wine regions to benefit from research on new pathogen
- Wineries embrace sensory analysis
- Author reveals first steps to marketing magic
- China - emerging market or competitor?
- Young Vine Decline is studied closely in NSW
- Is the Shiraz berry the biggest loser?
- Small players the big winners for tomorrow’s vineyard
- Save money and wine by choosing the right bentonite
- Significant variations in an Iconic Coonawarra vineyard lead to radical solutions
- Sustainable pest control – now and in a changing climate
- Is there value in adding tannin to wine?
- How do country of origin, closure type and label style affect purchase decisions?
- City sellers
- Selective science – from the vineyard to the winery
- Change agenda includes new thinking
- WineCloud provides future direction for winemakers
- The iron(III) tartrate photochemistry of wine: impacts of bottle colour and weight
- How important are wine medals and how much can we rely on those who assign them?
- Big rewards in fine detail
- Italian inspiration for novel Nero d’Avola making
- Oak deserves its fine environmental credentials
- Machinery maintenance is key to vintage success
- Wine: does vine age really matter?
- SA wineries make a positive and lasting impression
- Magazine pages bring history to life for Rutherglen grower
- Coles tells small wineries to ‘work with us’
- Distinguished vineyard sites are essential for quality fruit production says Petaluma
- From vine to bottle: sustainability a core value for Barossa winery
- Adapt and innovate in a challenging wine business world
- Verduzzo - a 'crazy' white
- Australia’s grapevine germplasm collections under threat
- Expo offers suppliers a chance to shine
- Wineries celebrate end of vintage
- Artisan by name and nature
- Barossa symposium delivers tips for Shiraz vineyard management
- Research to reveal best irrigation practice in dry winters
- Lake’s Folly proves its credentials over 50 years
- New research sheds light on flavour additives in wine
- It’s a three-ringed circus
- Trying to paint the world red
- Where do little winemakers come from?
- A chip off the old block
- The rise and rise of Gatt Wines
- Bizot and Croser – a marriage made in the vineyard
- Wine’s wild man rides into town
- Jeff Bond – a licence to thrill
- The Visionary – who’s laughing now
- Chilling out in Australian wine’s own Ice Age
- Quarter of our wines face Chinese ban
- Four-year research project investigates early influence of oxygen
- Australian winemakers’ views towards oak barrel alternatives matures
- Ready…set…tweet! How you can bank your social media benefits
- Wine show season: It seems not all wine shows would earn a gold medal
- January 2015 Grapegrower & Winemaker magazine out now
- Smart & Sustainable: Jana Shepherd
- Lazy bones: Jo Perry’s ironic nickname
- Can’t sit still: Bleasdale’s energiser bunny
- Meet New Zealand’s best young viti
- Suzie Muntz
- Clare Burder: Ideas are nothing without action
- Steve Baraglia: A tale of two valleys
- The terror of terroir - By Tony Keys