#139 Causal Inference with Carlos Avello - Marketing Science Lead
We have Carlos Avello, Marketing Science Lead at Amazon (Director of Marketing Analytics at eBay at the time of recording) joining us for an insightful chat on Causal Inference and how it can help businesses understand causes and impacts for better decision making.
Even before ecommerce started, Carlos was already working within an extremely data-driven business model. He started his journey in direct response marketing, tracking data from coupons and later analyzing it.
In his own words, he explains causal inference is based on evidence you collect of the impact a certain marketing effort had on your overall sales. Experimentation is the core of causal inference, and specifically A/B tests are used very often.
During his time on eBay, the main thing they tested was marketing impressions to help calculate ROI. Carlos also worked on figuring out how much incrementality was coming from paid marketing.
He points out it’s important for the advertiser and advertising platform to work together during the experimentation process for the right data to be available and later be analyzed. Carlos goes into detail about the two types of testing, user-based and geo-based, and the pros and cons of each one.
Stay tuned to learn more about causal inference and the insights you can unlock with it.
Enjoy the show!
We speak about:
[04:45] Carlos’ journey and how he ended up working in causal inference.
[09:00] What is causal inference?
[12:15] Can you tell us about your journey with experimentation on eBay and causal inference is tied to experimentation?
[18:00] What type of things do you test?
[25:25] How do you approach and address the saturation question?
[29:50] How did you move to a scalable experimentation platform, what are some of the components of the platform?
[35:30] Do you use data from past experiments?
[40:00] What are your thoughts on causal inference and the insights you can unlock with it?
[43:30] Resources on causal inference.
[46:50] What about experiments where you cannot restrain certain users from treatment, like in a healthcare setting for example?
[53:05] With everything you’ve done in your career, what are you most proud of?
Resources:
Carlos’s LinkedIn: https://www.linkedin.com/in/carlos-avello-5aa56327/
Amazon on LinkedIn: https://www.linkedin.com/company/amazon/
Books recommended by Carlos
Large‑Scale Inference: Empirical Bayes Methods for Estimation, Testing, and by Bradley Efron.
Statistical Rethinking: A Bayesian Course with Examples in R and Stan by Richard McElreath.
Quotes:
"Causal inference is based on evidence you collect of the impact a certain marketing effort had on your overall sales."
"Experimentation is at the core of it. You cannot get into causal inference if you do not compare two different populations with different treatments."
"Marketing is the product of a partnership between the advertiser and the advertising platform. Two different companies and there are things that we are ready to share and some others that are tricky to share, like customer data. If you want to setup a test where you are suppressing treatment to some people and allowing others to be exposed to the treatment and then compare the final results on sales you need to find a solution were you are saving enough information so the advertiser knows who has been exposed and who has not and later they can compare the sales."
Thanks to our sponsor:
And as always, we appreciate your Reviews, Follows, Likes, Shares and Ratings. It really helps new data scientists find us.
Thank you so much, and enjoy the show!