Optimizing paywall intercept rates for total revenue

In a recent research project, Mather Economics analyzed the performance of 118 websites for one year after they had modified their paywall strategies beginning in March of 2023. One cohort of news media sites allowed fewer free articles, the “closed” cohort, so a higher percentage of their site visitors encountered a paywall. The other cohort allowed more free content, the “open” cohort, with fewer visitors reaching a paywall.

This was not a randomized test. There were 91 sites in the data observed to have tightened their paywall (closed cohort) and 27 that loosened their paywalls (open cohort). While not a statistically valid comparison of markets, this natural experiment offers valuable insights into the factors that affect news media’s sites approach to optimizing their paywall settings.

To be included in the study, publishers must meet the following criteria:

• The market must offer a paid subscription.
• The market must have at least one form of standard content restriction (registration wall, premium paywall, metered paywall).
• Market must pass all data QA and validation for March 2023-March 2024, including accurately differentiating between the three types of restrictions.

Mather compared several performance metrics for each of these cohorts one year after they changed their settings, including monthly users, monthly pageviews, conversions per million users and the paywall conversion rate. We then applied these metrics in an economic model to identify the effect of their paywall strategies on subscription volumes, retention rates and revenue. We can also observe how these paywall strategies would affect advertising revenue through changes in page views.

Chart: Paywall hit rate index for one year:  This chart reflects the effect of the paywall changes on the intercept rate for the two cohorts from March of 2023 to March of 2024:

As seen in the chart of paywall hit rates, the closed paywall markets saw a greater paywall hit rate, more than a 100% increase. The more open markets experienced a decrease in paywall hits with about 30% fewer intercepts. This result is consistent with our expectations, given users' distribution of article views. Most website visitors only read one story per month, with a small share, typically about 5%, reading more than three articles. As a result, lowering the paywall from three to two articles per month will affect more visitors than increasing the number of free articles from three to four.

The charts below illustrate the changes in monthly users and page views by month for one year following the paywall changes. Within each market the metric is indexed vs March 2023. Data points reflect the median indexed value within each cohort.

Observations regarding these charts of monthly users and page views:

• Traffic declined for both cohorts YOY due to overall news industry trends over this time period.
• The Closed Cohort experienced larger declines in Users and Pageviews than the Open Cohort.
• The divergence in Pageviews doesn’t begin until August 2023, and the divergence in Users begins in October 2023.

The following charts illustrate the changes in the paywall conversion rate and the conversions per one million monthly users.

Note: the closed cohort is the blue line in these charts. They experienced lower conversion rates but higher conversions per million users.

The trends in conversion rates and conversions per million users between the two cohorts follow expected patterns. The cohort with the more open paywalls had a higher conversion rate, reflecting the greater engagement of those users who encountered the paywall after consuming more free content, while the tighter paywall cohort had a decrease in conversion rates as it intercepted, on average, fewer engaged users.

The net effect of the changes in the intercept rate and the conversion rate is observed in the conversions per million users for each cohort. The tighter paywall cohort realized a greater increase in conversions per million users than the open paywall cohort because the number of sales attempts (+100%) outweighed the reduction in conversion rate (-20%).

Both cohorts experienced higher conversion rates per million users over the year due to several factors, including improving conversion rates and changes in audience levels.

Some observations regarding the change in paywall settings on audience behavior:

• Tighter content restrictions result in fewer articles read but no net decrease in daily users or pageviews in the short term (about six months.)
• Users don’t immediately know a change has been made and continue to visit the site.
• Users refresh pages or try multiple articles when encountering a wall.
• Tighter content restrictions begin to affect page views and monthly users several months after the changes in paywall settings are implemented.

Applying metrics in an economic model

The cohort of sites with a tighter paywall strategy realized 46% more starts on average than those with a more open strategy. One tradeoff in tighter paywalls is that the acquired subscribers will have higher churn rates as the publisher implicitly trades lower quality for a higher quantity of new customers. Quality in this context is reflected in expected retention of new subscribers. The key question is, what level of retention is a “breakeven threshold” for the paywall strategy?  There is also an advertising revenue element to this analysis that is related to the change in page views.

Indexed performance metrics (March 2024 vs March 2023) for the two cohorts:

If we start with the closed paywall as the base case, since that is what most sites adopted during this time frame, we can calculate the breakeven retention needed by open paywall sites to achieve the same level of subscriber-months from new subscriptions (the number of months of active individual subscriptions). This breakeven rate depends on the time horizon. In this illustration, we have used one and two years using an index of 100 starts for the open sites and 146 starts for the closed sites.

If the closed paywall retention is 40% in year one, the open paywall customers would need an 85% annual retention to offset the lower start volume in the first year. A two-year breakeven annualized retention rate is 63%.

Revenue from these new subscribers will depend on the new offers and end of promotion pricing tactics employed by these news media companies. For this analysis, we use volume as the breakeven metric due to the limited space available in this blog post. The results of related pricing tests that change new offer promotion prices, promotional terms and renewal rates indicate significant benefits of strategic pricing tactics.

Chart: Subscriber months from the two cohorts showing “breakeven” subscriber volumes and retention from a two-year breakeven time horizon.

A few takeaways from this analysis and model:

• The growth in new starts outweighs the reduction in retention from tighter paywall strategies from a subscription volume perspective.
• The reduction in page views and visitors from a tightening of a paywall usually happens several months after the change is implemented.
• The required increase in retention needed to “breakeven” from the fewer number of starts generated by an open paywall is often difficult to achieve.
• More open paywall or registration wall policies should be accompanied by retention-improving new offer characteristics, such as longer terms.
• Advertising revenue effects are not material for the first few months given the lag in customer behaviors in response to paywall changes.
• A “pulsed” paywall strategy with alternating open and closed phases may mitigate long-term traffic losses, suggesting that active management of paywall strategies is needed to maintain an optimal revenue mix.

Conclusion

This analysis of 118 paywalled news media sites yielded valuable insights into the dynamic factors involved in optimal revenue strategies across subscriptions and advertising. The time lag from customer response to new paywall behavior is an important factor to consider. Also, the retention lift required to offset lower starts from a more open paywall is a difficult threshold to achieve. This analysis identifies areas for future research, including how to design new subscription offers that align with the customer segments that reach paywall intercepts under alternative paywall strategies and subscriber-only content.

Matt Lindsay is president at Mather Economics, based in Atlanta, Georgia. He can be reached at matt@mathereconomics.com.