on 14-Jun-2022 (Tue)

𝔼[𝑌 | do(𝑡), 𝑍 = 𝑧] vs 𝔼[𝑌 | 𝑍 = 𝑧]

𝔼[𝑌 | 𝑍 = 𝑧] simply refers to the expected value in the (pre-intervention) population where individuals take whatever treatment they would normally take ( 𝑇 ). This distinction will become important when we get to counterfactuals...

In this book, we stress the need to take the causal question seriously enough to articulate it, and to delineate the separate roles of data and assumptions for causal inference. Once these foundations are in place, causal inferences become necessarily less casual, which helps prevent confusion. The book describes various data analysis approaches to estimate the causal effect of interest under a particular set of assumptions when data are collected on each individual in a population. A key message of the book is that causal inference cannot be reduced to a collection of recipes for data analysi

This is not a philosophy book. We remain agnostic about metaphysical concepts like causality and cause. Instead, we focus on the identification and estimation of causal effects in populations, that is, numerical quantities that measure changes in the distribution of an outcome under different interventions.

Consider the following general kind of path, where → · · · → denotes a directed path: 𝑇 → · · · → 𝑍 ← · · · ← 𝑌 . Conditioning on 𝑍 , or any descendant of 𝑍 in a path like this, will induce collider bias

Machine Learning Engineering in Action

I had some successes but many failures, and generally left a trail of unmaintainable code in my wake as I moved from job to job. It’s not something that I’m particularly proud of. I’ve been contacted by former colleagues, years after leaving a position, to have them tell me that my code is still running every day. When I’ve asked each one of them why, I’ve gotten the same demoralizing answer that has made me regret my implementations: “No one can figure it out to make changes to it, and it’s too important to turn off.” I’ve been a bad data scientist. I’ve been an even worse ML engineer. It took me years to learn why that is. That stubbornness and resistance to solving problems in the simplest way created a lot of headaches for others, both in the sheer number of cancelled projects while I was at companies and in the unmaintainable technical debt that I left in my wake. It wasn’t until my most recent job, working as a resident solutions architect at Data- bricks (essentially a vendor field consultant), that I started to learn where I had gone wrong and to change how I approached solving problems.

Part 1 (chapters 1–8) is focused primarily on the management of ML projects from the perspective of a team lead, manager, or project lead. It lays out a blue- print for scoping, experimentation, prototyping, and inclusive feedback to help you avoid falling into solution-building traps

Part 2 (chapters 9–13) covers the development process of ML projects. With examples (both good and bad) of ML solution development, this section carries you through proven methods of building, tuning, logging, and evaluating an ML solution to ensure that you’re building the simplest and most maintainable code possible

Part 3 (chapters 14–16) focuses on “the after”: specifically, considerations related to streamlining production release, retraining, monitoring, and attribution for a project. With examples focused on A/B testing, feature stores, and a passive retraining system, you’ll be shown how to implement systems and architectures that can ensure that you’re building the minimally complex solution to solve a business problem with ML

The vast majority of the time, a project that fails to make its way to production for sustained utility has issues that are rooted in the very early phases. Before even a single line of code is written, before a serving architecture is selected and built out, and long before a decision on scalable training is made, a project is doomed to either cancellation or unused obscurity if planning, scoping, and experimentation is not done properly.