April 2-3, 2019

Register Now Convince the Boss

Using Social Media Data to Predict New Product Success

Room 1 | 1:30-2:00 | Wednesday, April 3, 2019

Almost all of our clients have social listening programs. Those thinking “downstream” are already considering whether the social data is good enough to replace tracking data. Can it tell us, for instance, whether customers are satisfied with a product? Can it tell us whether we have a healthy brand? Can it tell us about consumer innovations in a way that can guide new product development? Inevitably it has fallen short. As Forrester encapsulates: “Listening to data gleaned from social media channels, ratings and reviews and forums has existed for a decade, yet most CI pros still cannot demonstrate the value of monitoring social data.”

Together with our partner Converseon, we have taken several steps to combine their expertise in semi-supervised machine learning for text analysis with our understanding of CPG product data to do two things: (1) Understand the requirements that must be set in semi-supervised learning to identify both the product attribute and the motivational conditions that the attribute satisfies; and (2) understand the statistical criteria for discovering a trend or level shift in either text-derived or sales data and, separately, the criteria for establishing a causal relationship (not merely correlational) between text-derived and sales data.

Using a case study in vegan food, we will demonstrate how we first extract attributes from the text chunks that are the descriptive label in syndicated data. We then use these attributes to create search filters for posts in a vast “foodie” corpus. We then refine the search procedures using the ability to detect emotion and motivation in the post. Finally, we use time series methods, including Granger causality tests, to demonstrate that the time series for an attribute derived from filtered social data can predict sales trends up to 90 days before a product attribute appears in market.

Presentation type:

  • Case Study Presentation

Subjects covered:

  • Predictive analytics


  1. High-quality, properly processed social data can predict inflection in sales trends up to 90 days before they happen.
  2. There is an important feedback loop between social and sales: in the beginning social predicts sales but after a while the sales activity itself contributes to the social trend.
  3. Special filters applied to the social data (e.g. need states, combined emotions) increase the likelihood of predicting sales.


More sessions

See all sessions

Accelerate Relevance – Making Insights Sticky in Slippery Times

We’ve all felt the pressure of more information flowing in and our capacity to absorb it steadily diminishing. How do we sort through it, make the most of it and [...]


Tuesday | 10:45-11:15 | Room 1

Emotional Connections and Brand Disruption

This research project seeks to use survey data to better understand how non-traditional brand relationship metrics (emotional connections) relate to traditional brand measures, business outcomes and consumer responses to brand [...]


Tuesday | 5:00-5:30 | Room 2

Building a Powerful Global Research Insights Platform at Kellogg’s

Kellogg’s is a global business in an extremely competitive industry, with many global and regional brands and with regional teams that need fast and ongoing access to information and insight [...]


Tuesday | 10:45-11:15 | Room 4