Skip to main content

The Game and its Rules

The Game here is to see if and how the Observed Association can be used to recover the Average Causal Effect.

E[Y∣A=1]−E[Y∣A=0]⏟Observed Association→Under what assumptions? Can I recover?E[Y1]−E[Y0]⏟Average Causal Effect (ACE)\boxed{ \underbrace{ \mathbb{E}[Y\mid A=1] - \mathbb{E}[Y\mid A=0] }_{\text{Observed Association}} } \xrightarrow[\text{Under what assumptions?}]{\ \text{Can I recover?}} \boxed{ \underbrace{ \mathbb{E}[Y^1] - \mathbb{E}[Y^0] }_{\text{Average Causal Effect (ACE)}} }

Seems almost impossible without access to a Multiverse eh? But some very bright people have come up with very bright ideas on how to accomplish this.

But first, can you even play this game?

Identification​

This is bloody critical and worth spending a ton of time on. It determines what causal statements you can make. It’s a lot of chin-scratching and mental torture but a lot of things drop out from this (and it makes you go “wow” too.)

To expand on the intro to this page: It is about whether you can (a) determine the causal effect using (b) the distribution of the observed data (c) based on certain assumptions.

It's not always about more data

Suppose we had unlimited data. Would we be able to determine the causal quantity we want?

You may be tempted to exclaim “HELL YEAH MORE DATA” but you’d be kinda wrong. The goal of that question is not about the size of the data. It is about the assumptions you’re making. A million patients in your study might not solve the causal effect estimation problems you have with a thousand.

  • More data will not fix a failure of Identification.
  • Identification depends on the assumptions, not on some fancy estimator(s)

Given infinite data and our assumptions, there is only one possible value for the causal effect. It is identified.

So what are your assumptions?

Assumptions​

TL;DR:

  • Consistency: The observed outcome matches the potential outcome under the treatment actually received.
  • SUTVA: Treatment is well-defined, and one person’s treatment does not affect another person’s outcome.
  • Exchangeability: Treated and untreated groups are comparable in terms of their potential outcomes (possibly after adjusting for covariates).
  • Positivity: For every relevant type of person, both treatment and no treatment are possible.

They’re all pretty commonsensical when you think about them. I’ll start with the most confusing one.

Consistency​

A=a  ⟹  Y=YaAAlice=a  ⟹  YAlice=YAlicea\begin{align*} A = a &\implies Y = Y^a \\ A_{\text{Alice}} = a &\implies Y_{\text{Alice}} = Y_{\text{Alice}}^a \end{align*}

Here’s another way:

Y=AY1+(1−A)Y0Y = AY^1 + (1 - A)Y^0

The key idea here is that we need to clearly define our interventions. This sounds “duh” and simple but there’s a lot of nuance here and I can only do examples to try and cut through the fog.

Imagine you want to examine the effect of sleeping 8 hours before a test on the text scores the next day. Alice and Bob both sleep for 8 (both assigned A=1A=1). But Alice sleeps uninterrupted and Bob wakes up a few times. Is this a problem?

Now imagine you want to examine the effect of 500mg Aspirin on headaches. Alice takes 500mg, Bob takes 150mg, Rita takes baby Aspirin three times over the day, once with coffee in the morning. But we don’t know this and record Bob and Rita as taking the full dose. Is this a problem? Could you have defined the intervention better?

This is about the assigned treatment versus what actually happened. When we assume Consistency, we’re saying the treatment is well-defined and what people actually did matches it.

I’ve read “The treatment people actually received must correspond to the treatment used to define their potential outcomes” or “If you actually took aspirin, your observed outcome must equal your potential outcome under aspirin” but this is still confusing.

TODO: Finish this.

It links to SUTVA which is easier to follow.

SUTVA​

Stable Unit Treatment Value Assumption. Wonder who came up with this lovely name.

It’s a two-fer:

  1. Well-defined treatment/Consistency: There is only one clear version of each treatment, and the mechanism used to deliver it does not change the result1.
  2. No Interference: One person’s treatment cannot affect another’s outcome. Think studying vaccines in a college dorm.

Exchangeability​

(Y1,Y0)⊥A\boxed{(Y^1, Y^0) \perp A}

TLDR: “Assignment is independent of potential outcomes”.

This is really really important. What are we saying?

  • Knowing who was assigned the treatment does not give us any more information on the potential outcomes.
  • That is, both treated and untreated groups have the same distribution of potential outcomes regardless of which treatment they received.

Here’s a trivial example. Can you conclude that the Drug is OMG AMAZING?

AliceBobAge3070HealthPretty healthySuper sickTreatmentDrug X (A=1)No drug (A=0)OutcomeHealthyStill sick :/\begin{array}{l|l|l} & \textbf{Alice} & \textbf{Bob} \\ \hline \text{Age} & 30 & 70 \\ \text{Health} & \text{Pretty healthy} & \text{Super sick} \\ \text{Treatment} & \text{Drug X } (A=1) & \text{No drug } (A=0) \\ \text{Outcome} & \text{Healthy} & \text{Still sick :/} \end{array}

The Perfect Doctor​

An example that comes up a lot. Our Doctor only assigns treatment to the sicker patients because she knows how sick her patients are.

Treated patientsUntreated patientsBefore treatmentVery sickRelatively healthyAfter treatment40% recover80% recover\begin{array}{l|l|l} & \textbf{Treated patients} & \textbf{Untreated patients} \\ \hline \text{Before treatment} & \text{Very sick} & \text{Relatively healthy} \\ \text{After treatment} & 40\% \text{ recover} & 80\% \text{ recover} \end{array}

The treatment assignment tells us something about their potential outcomes. This violates Exchangeability.

Conditional Exchangeability​

We introduce a new guy, CC here. For confounder (or set of coufounders.)

(Y1,Y0)⊥A∣C\boxed{(Y^1, Y^0) \perp A \mid C}

Here you “adjust for things” (covariates) and claim that Exchangeability holds. We’re saying that once we compare people who have the same relevant baseline characteristics, treatment assignment no longer tells us about their potential outcomes.

This is critical in Observational Studies

“All observational studies are plagued with the problem that we can only correct for known and measured covariates but not for the unknown or unmeasured covariates. Randomized studies do not have that problem and any imbalance in unobserved covariates is only coincidental and not systematic!”

— Source

Positivity​

0<P(A=1)<10<P(A=1∣C=c)<1\boxed{0 < P(A=1) < 1}\\ \boxed{0 < P(A=1|C=c) < 1}

This one’s rather easy. Everyone has a decent chance of being assigned a treatment or not. That last part is the nuance! If you have a drug to everyone over the age of 75, this violates Positivity since at least a few of them should be given no drug.

Now what if the chance of treatment or no treatment is very low? Say 1 person gets the drug and 99 don’t. Then estimating the Causal Effect becomes harder because it depends on the outcome of that one person. There may be ways around this:

  • Larger sample/more data!
  • You would just remove the group with poor “overlap”2
  • Use Propensity Scores (estimate P(A=1∣C)P(A=1 \mid C))… more on that later.

Note that from a ‘mechanical’ standpoint, expressions like E[Y∣A=1]E[Y|A=1] are meaningless unless there’s some assigment.

Assignment​

TL;DR of the last section
  • Exchangeability says the groups are comparable.
  • Positivity says both groups exist.
  • Consistency says the observed outcomes correspond to the potential outcomes we think they do.

This is where you put those assumptions into action. There’s four of them and they’reare easy if you understand the above.

Is the Assignment…Simple questionConnected concept
Individualistic?Does Alice’s assignment depend on Bob?Assignment independence across units
Probabilistic?Can Alice receive either treatment?
0<P(A=a)<10 < P(A=a) < 1		Positivity
Unconfounded?Does assignment depend on potential outcomes?
P(A∣C,Y1,Y0)=P(A∣C)P(A \mid C,Y^1,Y^0) = P(A \mid C)		Exchangeability
Known/controlled?Does the researcher control the assignment process?Experimental design

(Also, a well-defined intervention →\rightarrow Consistency.)

All classical Randomized Experiments have these four properties. This is why they’re Gold Standards ✨

Putting things together​

Where the magic happens! The assumptions allow us to recover the Causal Effect from the Observed Association. Here’s how.

Remember the goal:

E[Y∣A=1]−E[Y∣A=0]⏟Observed Association→Under what assumptions? Can I recover?E[Y1]−E[Y0]⏟Average Causal Effect (ACE)\boxed{ \underbrace{ \mathbb{E}[Y\mid A=1] - \mathbb{E}[Y\mid A=0] }_{\text{Observed Association}} } \xrightarrow[\text{Under what assumptions?}]{\ \text{Can I recover?}} \boxed{ \underbrace{ \mathbb{E}[Y^1] - \mathbb{E}[Y^0] }_{\text{Average Causal Effect (ACE)}} }

Preflight Math Setup​

Some mathy reminders of where stuff is so far. We have Potential Outcomes as

Y1=outcome under treatmentY0=outcome under no treatment\begin{align*} Y^1 &= \text{outcome under treatment} \\ Y^0 &= \text{outcome under no treatment} \end{align*}

Exchangeability means that the Expected Value of the potential outcome is unfazed by the assignment. But then we can have covariates. We’ll put these covariate groups in a vector C={c1,…,cK}C = \{c_1,\ldots,c_K\}. Let’s write btoh the ‘regular’ and conditional exchangeability like this:

Ya⊥A  ⟹  E[Ya∣A=a]=E[Ya](Marginal)Ya⊥A∣C  ⟹  E[Ya∣A=a,C=c]=E[Ya∣C=c](Conditional)\begin{align*} Y^a \perp A & \implies E[Y^a \mid A=a] = E[Y^a] & & \text{(Marginal)} \\ Y^a \perp A \mid C & \implies E[Y^a \mid A=a, C=c] = E[Y^a \mid C=c] & & \text{(Conditional)} \end{align*}

With Consistency we are saying A=a  ⟹  Y=YaA=a \implies Y=Y^a, so

E[Ya∣A=a]=E[Y∣A=a]E[Ya∣A=a,C=c]=E[Y∣A=a,C=c]\begin{align*} E[Y^a \mid A=a] & = E[Y \mid A=a] \\ E[Y^a \mid A=a,C=c] & = E[Y \mid A=a,C=c] \end{align*}

Let’s get some Positivity up in here with a nice

P(A=a∣C=c)>0for all relevant a,cP(A=a \mid C=c)>0 \quad \text{for all relevant } a,c

And finally, we have the Law of Total Expectation. If QQ partitions the sample space then:

E[X]=∑q∈QE[X∣Q=q]P(Q=q)E[X]=\sum_{q\in Q} E[X\mid Q=q]P(Q=q)

We are now ready.

Randomized Experient​

This one’s super easy because RCTs are awesome. No confounders here.

ACE=E[Y1]−E[Y0]=E[Y1∣A=1]−E[Y0∣A=0](by Exchangeability)=E[Y∣A=1]−E[Y∣A=0](by Consistency)\begin{align*} ACE & = E[Y^1] - E[Y^0] \\ & = E[Y^1 \mid A=1] - E[Y^0 \mid A=0] & & \text{(by Exchangeability)} \\ & = E[Y \mid A=1] - E[Y \mid A=0] & & \text{(by Consistency)} \end{align*}

Et voila!

E[Y1]−E[Y0]⏟causal=E[Y∣A=1]−E[Y∣A=0]⏟observed association\boxed{ \underbrace{E[Y^1] - E[Y^0]}_{\text{causal}} = \underbrace{E[Y \mid A=1] - E[Y \mid A=0]}_{\text{observed association}} }

Easy-peasy!

Observational Study​

Little more faffing around because confounders here but it’s not too bad. This derivation is important, it’s the g-Formula!

In Observational Studies, we cannot claim Exchangeability (potential outcomes are independent of assigment):

Ya⊥AY^a \perp A

But we can (hopefully) say via Conditional Exchangeability,

Ya⊥A∣CY^a \perp A \mid C

The game plan here is

Law of Total Expectation→Exchangeability→Consistency\text{Law of Total Expectation} \rightarrow \text{Exchangeability} \rightarrow \text{Consistency}

We sum the potential outcome random variable YaY^a over the covariate vector CC that partitions the space using the Law of Total Expectation. Then we sneak in the assignment AA using exchangeability. We finally ‘transform’ the potential outcome into the actual outcome using Consistency and will rely on Positivity to make sure we don’t get absurd results. Let’s go!

E[Ya]=∑c∈CE[Ya∣C=c]P(C=c)(Total Expectation)=∑c∈CE[Ya∣A=a,C=c]P(C=c)(Exchangeability)=∑c∈CE[Y∣A=a,C=c]P(C=c)(Consistency)\begin{align*} E[Y^a] & = \sum_{c\in C} E[Y^a\mid C=c]P(C=c) & & \text{(Total Expectation)} \\ & = \sum_{c\in C} E[Y^a\mid A=a,C=c]P(C=c) & & \text{(Exchangeability)} \\ & = \sum_{c\in C} E[Y\mid A=a,C=c]P(C=c) & & \text{(Consistency)} \end{align*}

Behold the g-formula! So now,

ACE=E[Y1−Y0]=E[Y1]−E[Y0]=∑c∈CE[Y∣A=1,C=c]P(C=c)−∑c∈CE[Y∣A=0,C=c]P(C=c)=∑c∈CE[Y∣A=1,C=c]P(C=c)−∑c∈CE[Y∣A=0,C=c]P(C=c)=∑c∈C{E[Y∣A=1,C=c]−E[Y∣A=0,C=c]}⋅P(C=c)\begin{align*} \text{ACE} & = E[Y^1 - Y^0] \\ & = E[Y^1] - E[Y^0] \\ & = \sum_{c\in C} E[Y\mid A=1,C=c]P(C=c) - \sum_{c\in C} E[Y\mid A=0,C=c]P(C=c) \\ & = \sum_{c\in C} E[Y\mid A=1,C=c]P(C=c)- \sum_{c\in C} E[Y\mid A=0,C=c]P(C=c) \\ & = \boxed{\sum_{c\in C} \left\{ E[Y\mid A=1,C=c] - E[Y\mid A=0,C=c] \right\} \cdot P(C=c)} \end{align*}

BOOM. How lovely is that?

ACE=∑c∈CE[Y∣A=1,C=c]P(C=c)−∑c∈CE[Y∣A=0,C=c]P(C=c)\begin{aligned} ACE &= \sum_{c\in\mathcal C} E[Y\mid A=1,C=c]P(C=c)\\ &\quad-\sum_{c\in\mathcal C} E[Y\mid A=0,C=c]P(C=c) \end{aligned}

Footnotes​

  1. Why is it mentioned again? Good question. Professor thought it was important enough to separate out first. It’s really part of SUTVA. ↩

  2. A confusing word for the 1/99 drug example above. Say you had 50/50 Healthy group, 20/80 Moderately healthy group, and 1/99 Super Sick grou. You’d consider trimming that last group. ↩