Archive for the ‘Uncategorized’ Category

Do you ever look at your data and say, “huh?” The Unusual Statistical Phenomena of Simpson’s Paradox

November 2nd, 2021 Comments off

Sometimes when looking at the results from survey data, we see something that makes us say “huh?” or “that doesn’t look right”.  When the odd results persist after verifying the data were processed correctly (always a good practice), there is typically still a logical answer that can be uncovered after doing some digging.  Sometimes the answer lies with something that we will call “unusual statistical phenomena.”  This is part 1 of a series that will look at some of these interesting – or confounding – effects that do pop up now and then in real survey research data.

This time we will look at Simpson’s Paradox.  And we aren’t referring to the fact that Bart Simpson never seems to age while the rest of us do.  It is actually a phenomenon first described by the statistician Edward H. Simpson in 1951.

It’s easiest to understand this phenomenon through an example.  So, let’s say that we have two ads that have been on air, ad A and ad B.  In our tracking survey among adults 18 to 65, we will ask respondents if they recognize having seen each ad on air.  Earlier in the survey we ask Purchase Intent for the product which is featured in each of the two ads.  From these results, we will compare Top Box Purchase Intent among respondents who recognized each of the two ads.  The results in the table below show somewhat higher Top Box Purchase Intent for Ad A:

However, the client is also interested in seeing the results among each of two age groups: age 18 to 39 and age 40 to 65.  When we table those results, we find something that just doesn’t make sense.  Purchase Intent is slightly higher for Ad B among both age groups – a reversal from the overall results.  How can that be!

After verifying with data processing that the data are correct, we have our team dig into the data to figure out what is going on.  Finally, an explanation is found.

Ad B was aired heavily among programming targeted to a younger audience, while Ad A was primarily aired in general interest programming – which skews to a slightly older audience.  Hence Ad B had much higher recognition among the younger age group – and as a result, a much higher proportion of young people in the set of respondents among whom purchase intent was calculated.

The table of base sizes shown below reveals this imbalance. When combined with the younger age group’s more skeptical nature (and lower results) when it comes to Purchase Intent – especially in our category – the apparent anomaly is explained.

This is an example of Simpson’s Paradox.  It is a phenomenon in which individual subgroups all show the same trend in results, but the trend reverses when the subgroups are combined.  This occurs when there is a confounding variable that causes an imbalance in base sizes such as we saw above.  In our example, the confounding variable was the differing recognition levels for the ads among the two age groups.

Simpson’s paradox shows us the importance of knowing and understanding our data and keeping a watch out for the kind of confounding factors that could end up misleading us if we don’t account for them.

Categories: Uncategorized Tags:

MSW Research, Wins Best Practitioner Paper 2021 by the JAR Editorial Board

June 30th, 2021 Comments off

Today, the Advertising Research Foundation announced that the paper “Effectiveness and Efficiency of TV’s Brand-Building Power: A Historical Review — Why the Persuasion Rating Point (PRP) Is a More Accurate Metric than the GRP” has been voted Best Practitioner Paper 2021 by the JAR Editorial Board.  Along with co-authors Frank Findley of MASB, David Stewart of Loyola Marymount University and Kelly Johnson of Disney, we at MSW Research were very honored to have our work recognized with this prestigious award.

The paper was published in the December 2020 issue of the Journal of Advertising Research.  It addresses the questions of whether television advertising is as effective as in the past and if so, how it compares with other media-platform alternatives.

The major conclusions of the study include the following:

  • On a single, quality exposure basis the television ad format is as effective now as it was in the 1980s (based on copy-testing for 30-second television ads collected within the United States for typical categories with brands advertising throughout the years 1980 to 2014 using a consistent methodology, MSW Research’s CCPersuasionTM measure).

  • The rate of delivery of an ad’s selling power per GRP has slowed, requiring approximately 25% more GRPs to deliver the same brand-building power in market as it did in the 1980s.  This implies that television-channel proliferation, time-shifting technology, and simultaneous digital-media consumption (multi-screen behavior) are having an impact on the advertising viewing experience.


  • This decline is mitigated, however, by a 45 percent increase in the number of households in the United States over the same time period.  Each rating point now represents many more households. So even though fewer households are now effectively reached by a given spending level on a percentage basis, this is not true on an absolute number of households basis.


  • Despite a potential increase in distracted viewing, television advertising still maintains an effective frequency profile that is comparable to other media channels including digital.  All examined media types – including television – can be effective within the range of average frequencies typically deployed for them.


Every media platform has its own strengths and challenges. Although the interruptive nature of television advertising may make it more susceptible to divided attention with other media, it also provides television with one of the most immersive visual and audio experiences.

Television remains an effective media platform, and television advertising should continue to be used to maintain and grow market share. By focusing attention on the development stage, brands can improve advertisement quality to such a degree as to more than compensate for the decline in the rate of ad selling power delivery per gross rating point.

Trafficking gross rating points behind advertisements on the basis of their persuasive strength allows for the diminishing returns dictated by wearout to be managed while maximizing sales power delivered in market.  This means actively managing both quantity and quality of the advertising media plan, which can be achieved through focusing on Persuasion Rating Points (GRPs weighted by a measure of persuasion), rather than GRPs alone.

Contact us for a full reprint of the article.

Categories: Ad Pre-Testing, MASB, Uncategorized Tags:

| The Brand Strength Monitor / RDE Chart of the Week | HBO MAX: Pandemic Winner

June 9th, 2021 Comments off

MSW’s RDE Analytic Framework rests on a study that found that three equity dimensions (Relevance, Differentiation, Emotion) are responsible for driving a significant portion of brand growth. The RDE composite strongly relates to – and helps explain – CCPreference which itself is a validated predictor of market share.

Comparing RDE Assessment taken before the pandemic to comparable assessment from May 2021 reveals one brand in the Subscription Video Streaming Services category that has emerged as a winner – HBO Max.

  • HBO Max was launched in May 2020 well after the COVID pandemic was underway. It essentially replaced the previous HBO streaming service, HBO Now, but with a huge amount of new content from a variety of WarnerMedia brands.
  • Despite RDE levels being generally depressed as a result of the pandemic, HBO Max RDE levels easily surpass HBO Now pre-pandemic levels.
    • HBO Max easily outstrips HBO Now on all three RDE dimensions.
    • HBO Max is particularly strong in terms of relevance meaning that SVOD consumers feel that the HBO Max service is for someone like them.

  • HBO Max also far exceeds HBO Now in terms of CCPreference – the percent of consumers preferring HBO Max over competitors in the subscription streaming services category. This suggests that HBO Max should be performing well in the marketplace. How has it done?
  • HBO Max’s strength is its large and varied TV and movie catalog. In addition, Warner Brothers has been releasing its 2021 movies on HBO Max simultaneous with the theatrical release which has proven very popular.
    • At the end of the first quarter of 2021, HBO and HBO Max totaled 44.2 million domestic subscribers – far exceeding the 33.1 million subscribers a year ago (before HBO Max).
    • A report from Reelgood indicated HBO’s share of streaming activity rose from 2.0% in Q1 of 2020 (pre-HBO Max) to 12% in Q4 2020, placing HBO Max ahead of Disney+ and behind only Netflix, Prime Video and Hulu.
  • This success – mirrored by extremely strong CCPreference growth – has come despite a soft launch which slowly added support for many popular streaming devices, the COVID pandemic and accompanying economic downturn, and ever-increasing competition in the streaming video space.

The RDE composite is strongly related to brand preference in this category, with a correlation of 0.96