Pushing the boundaries of experimentation and measurement with innovative new modeling techniques

In our measurement framework, we have three methodologies: attribution, incremental testing, and MMM. However, these methods currently face constraints preventing seamless collaboration.

The attribution model, for example, needs to include the incorporation of constraints related to the baseline, a critical factor in understanding the overall impact of marketing activities.

Simultaneously, incremental testing is often tightly controlled by major tech platforms. They often require significant media spend for execution and are often limited to paid digital media alone. This necessitates advanced techniques like Bayesian causal inference and the potential outcomes framework; methodologies that have been pioneered and open-sourced by entities like Amazon Science and Microsoft Research.

And in the area of MMMs, there are solid opportunities to leverage the latest advances in causal models, again thanks to the pioneering work of Amazon Science, whose breakthrough innovation is indeed open source and available for the data science community to build upon.

The current methodologies offer a more nuanced understanding of marketing activities within the MMM framework. However, they resist going into the granularity necessary for marketers to take swift and targeted actions aligned with specific business needs. To address this, there’s a surge in the development of hierarchical Bayesian models and more importantly, structured causal models, which allow a more granular perspective and more frequent execution of MMMs.

Since 2021, there’s been a causal AI revolution. Companies, led by Microsoft Research and Amazon Science, are now emphasizing understanding the “why” behind data, transcending the traditional focus on the “what.” Open source solutions in structured causal models enable businesses to estimate the impact of a single event within the broader context of concurrent and confounding events.

Marketing and data teams are an ideal pairing, with marketers driven by a desire to push creative boundaries finding a match in data teams that thrive on solving challenges. Breaking away from institutionalized approaches is crucial for fostering an innovative culture. Encouraging data scientists to solve problems that arise from pushing creative boundaries ensures the company remains agile and relevant.

In the Netherlands, we’re working with an online e-commerce player that conducts hundreds, if not thousands of experiments annually, a major shift from having no measurement practices just two years ago. This robust culture of innovation and experimentation has set them apart, improving marketing efficiency and ROI by an impressive 20%.

In summary, the synergy between advanced measurement practices, a focus on causality, and a culture of innovation and experimentation is reshaping the marketing analytics landscape and ensuring that companies remain agile and relevant in an ever-evolving business landscape.