New DTC Report long on data tidbits, marred by errors and unsupported claims, short on context or actionable info

The the 2019 Direct-to-Consumer (DtC) Wine Shipping Report from Sovos and Wines & Vines Analytics is long on data tidbits and breathless prose,  incorrect information, misleading or unsupported claims and a lack of context, transparency and credibility. It contains no credibly actionable data and fails to disclose significant methodology, data sources or margins of error that could be used to assess accuracy and reliability.

Lack of disclosure

At a fundamental level, the report does not disclose how many wineries the data comes from, or what their share of the overall market might be so that a reader can comprehend the significance of the data set.

 

The report only says that:

 

“To produce this report, Wines Vines Analytics created an algorithm that uses its database of nearly 10,000 U.S. wineries to extrapolate all DtC shipments from millions of anonymous transactions filtered through the ShipCompliant by Sovos system in 2018. The model tracks sales by winery region, annual winery production, destination of shipments, wine type (varietal) and price points.”

 

While “nearly 10,000 U.S. wineries” sounds impressive the statement offers a misleading impression that the report represents 10,000 of the 11,496 wineries in the United States.

 

As a result, the report fails to disclose how many wineries were actually represented in the data or whether that number is statistically significant (by what margin of error?) and how it can credibly be extrapolated to the entire market.

What’s more, “millions of anonymous transactions” is equally vague. Was that 2 million or 100 million or more? The use of a meaninglessly vague number and failure to offer any degree of precision does  not inspire confidence in the conclusions.

 

In addition, failure to disclose methodology also calls into question whether the “filtered” data is statistically significant. The information may actually be correct, but that cannot be determined from the information provided.

 

An algorithm is simply a set of steps used to reach a conclusion. One does not need to disclose the proprietary aspects of an algorithm to establish the credibility of the data.

 

“This is just a black box,” one winery owner told Wine Industry Insight. “The Sovos and others involved with this cannot be unaware of the severe algorithm bias problems being widely discussed today regarding problems that range from Facebook and Google to law enforcement, resumé and hiring practices. Trusting a black box has lost favor in mainstream technology which this study is not.”

 

A third winery data executive added:

“They have an algorithm, which means they did math. But Wines & Vines data base of wineries is for sale online, and it doesn’t track revenues. The wine business is private and nobody sends their financials to Wines & Vines. They aren’t matching this to tax receipts like Gomberg.

“Their database offers ranges of case sales, meaning if they are extrapolating real compliance data and throwing that at their estimates in their own database as they say, the range of error is substantial because they have estimated case sales that have to be converted to revenue using estimates of bottle prices. So its estimates of estimates to come to a fact. Maybe it’s good and maybe not.”

Incorrect Direct Shipping Map

Another hit to the report’s credibility is the fact that their map of “open” states is incorrect, according to Wine Institute data.

 

Did Sonoma Really Crush Napa?

The news release “Wine Shipments to Consumers Hit Record $3 Billion in 2018, followed up with the reason for the increase: “Fueling this growth, the average price per bottle increased 2.4 percent, the most significant one-year price spike since 2011.”

 

 

This was followed on closely by “Sonoma County was the standout among regions analyzed, propelling ahead of Napa County, the historic leader in the DtC channel. Sonoma’s total volume of shipments increased by 19 percent and the value of shipments by 18 percent, marking a 200 percent increase since 2011.”

 

Needed Context

Despite the overall industry DTC growth being driven by price increases, Sonoma failed on that metric and managed to ship lots more wine than  Napa — because they dropped the per bottle price. Of course, the report keeps the vital information secret: what are the revenues?

 

 

In addition, the placement of the in chart caption below –“VOLUME UP…” — looks as if it refers to Sonoma.