Effects

Effect = “What Happened?” We assign numbers to things that can happen. Some are binary (e.g. 0 = didn’t work, 1 = worked), some are categorical, some are continuous.

For a given person in the hypothetical space (you give and don’t give treatment to the same person lol) the Individual Causal Effect ($\tau_i$) is simple:

$$\begin{align*} \tau_i &= \text{Outcome with treatment} - \text{Outcome without} \\ &= Y_i^1 - Y_i^0 \end{align*}$$

With a binary $\{0,1\}$ outcome that $\tau_i$ could be $\{-1,0,1\}$. Check out this example (stick with Aspirin and headaches):

Person	$Y_i^0$	$Y_i^1$	$\tau_i = Y_i^1 - Y_i^0$
Alice	0	1	+1
Ravi	1	0	−1
Mei	1	1	0
Omar	0	0	0
Greta	1	1	0
Kofi	0	0	0
Lucía	1	1	0
Tomas	0	1	+1
Total	$\sum_i Y_i^0 = 4$	$\sum_i Y_i^1 = 5$	$\sum_i \tau_i = 1$

This is God Mode!

You know what happened and what would have happened to the same person with and without the treatment 👼 As a mortal, you will never know what would have happened to Alice in both cases! Hypothetical space!

So that’s for each individual. Like good statisticians, we want some averages. Why? Well you can’t do much by just asking “How did Aspirin help Alice?” Ask “How did taking Aspirin help people on average?”

$$\frac{1}{n}\sum_{i=1}^8{\tau_i} = \frac{1}{8}$$

What does this tell us? It tells us that administering Aspirin increases the average value of the outcome $Y$ by 12.5 points.

TODO: Explain why points and not percent. Has to do with absolute and relative…

Average Causal Effect

You are still in God Mode!

We’ll speak of Individual, Sample, and Population-level Causal Effects. These are The Truth™ that you do not have access to outside our hypothetical playground! These are things you’re trying to estimate using some lovely statistics. These are estimands.

Remember that $Y^A$ is the Population-level outcome where $A \in \{0,1\}$ is the assignment (whether or not treatment was given.) That average above really can and should be written as:

$$E[Y^1 - Y^0] = E[Y^1] - E[Y^0] = \frac{5}{8} - \frac{4}{8} = \boxed{\frac{1}{8}}$$

This is the Average Causal Effect (ACE).¹

Sample Average Causal Effect (SACE)

This is exactly what we computed before.

$$\text{SACE} = \frac{1}{N}\sum_i^N(\tau_i) = \frac{1}{N}\sum_i^N(Y_i^1 - Y_i^0)$$

Population Average Causal Effect (PACE)

This is ACE.

$$\text{PACE} = E[Y^1 - Y^0] = E[Y^1] - E[Y^0]$$

Are they equal? I am a God and know Everything.

Nope. Well maybe in three cases:

1. Pure luck! 🍀
2. When you decree that your sample is the entire population of interest.
3. When everyone has the same individual causal effect $\tau_i$ (haha)

I think we can write $E_{\text{Sampling}}[\text{SACE}] = \text{PACE}$ but I could be wrong. TODO: ask prof.

Which has more variance?

Think about it: with SACE, you have differences between people. With PACE you deal with that and who was sampled. Which do you think?

Why even SACE?

Why not be ‘just’ interested in PACE? Because it depends on what you’re trying to do. It depends on which group of people you are studying.

• Sometimes you are only interested in what happens to people in your study and nothing more. Examples: you’re trying to study something at the hospital you work at, or you’re studying people with rare conditions and have managed to enroll most of them. You decree with some justification that your sample is your entire population of interest.
• The people you recruit for a study might be the types of people who sign up for studies. Can you say that this representative of the population?
• SACE is nice when you want to study statistical uncertainty (Fisher and Neyman, coming up!) With SACE you can answer narrower questions with fewer assumptions.

Assignment and Sampling

Seems like a duhburger but worth mentioning: Random Assignment is not the same as Random Sampling.

SACE (remember, this is the estimand; God Mode!) remains the same even if you assign randomly and rerun the experiement. PACE is also an estimand.

---

You are now a Mere Mortal

Let’s exit hypothetical space. This table is Real Life™ because you can only know one potential outcome for the person you assigned the treatment to.

Person	$Y_i^0$	$Y_i^1$	$A$
Alice	?	1	1
Ravi	?	0	1
Mei	1	?	0
Omar	?	0	1
Greta	1	?	0
Kofi	0	?	0
Lucía	1	?	0
Tomas	?	1	1

So what do you do now? What can you do? This is the only thing:

$$E[Y|A=1] - E[Y|A=0] = \frac{2}{4} - \frac{3}{4} = \boxed{-\frac{1}{4}}$$

In English: The difference between the average observed outcome among those treated and average observed outcome among those not treated.

Now that number is pretty different than the S/ACE we got earlier. The difference in group means and apparent effect is lower. Why? Could be that some people assigned the treatment ($A=1$) are healthier? Some other funky shit could be going on since you partitioned the people into two separate groups.

Unlike earlier, you not dealing with the same person under two different circumstances (with/out treatment). Add up this loss of information and it affects your average, your estimate of the effect (more on this later.)

It would be Very Nice Indeed if you could make Observed Association and Average Causal Effect correspond with each other if possible.

---

Causation and Association

Association: Average Effect Size across people who happened to be treated versus people who happened to be untreated (again: differnt groups!)

$$\text{Observed Association} = E[Y|A=1] - E[Y|A=0]$$

Causation: Average Effect Size across everybody under treatment versus everybody under no treatment.

$$ACE = E[Y^1 - Y^0] = E[Y^1] - E[Y^0]$$

These are different as you’ve seen with our small example. You want ACE but you only see the Observed Association. Would be nice if you could make these two correspond with each other…

They’re not the same

Consider this: A hospital has 200 patients: 100 are very sick and 100 are mildly sick. Doctors give the medicine mostly to the very sick ones. The outcome $Y = 1$ means the patient died :/

Suppose we are in God Mode we know the medicine actually works and cuts the death rate by 10 points in both groups.

Group	Death rate without medicine	Death rate with medicine
Very sick	60%	50%
Mildly sick	20%	10%

Now here’s who actually got it and who died:

Group	Got medicine ($A=1$)	Deaths	No medicine ($A=0$)	Deaths
Very sick	90	45	10	6
Mildly sick	10	1	90	18
Total	100	46	100	24

Association (what you observe): 46% of medicine users died vs 24% of non-users.

$$E[Y|A=1] - E[Y|A=0] = 0.46 - 0.24 = +0.22$$

Medicine users die way more often. So the medicine kills people? Nope.

Causation (what is true): Give the medicine to everybody and 30% die (50% of the very sick, 10% of the mildly sick). Give it to nobody and 40% die (60% and 20%).

$$E[Y^1] - E[Y^0] = 0.30 - 0.40 = -0.10$$

The medicine saves lives. So why’d the sign flip? What happened?

How sick you are motivates two separate things:

1. Whether you get the medicine. Doctors hand it to the very sick (90 of 100) and mostly skip the mildly sick (10 of 100). So knowing someone took the medicine tells you something: they were probably very sick.
2. Whether you die. Very sick people die more often (50–60%) than mildly sick people (10–20%), medicine or not.

So “took medicine” and “died” show up together a lot, but mostly because both are caused by being very sick. The medicine’s own arrow ($A \to Y$) actually lowers deaths. It’s just drowned out by sickness pushing up both.

The key thing is that the two groups were different before anyone took anything. The association is mostly measuring how sick people were, not what the medicine did.

The Sickness here is a confounder.

---

Footnotes

1. This is linearity of expectation: $E[X - Z] = E[X] - E[Z]$, always. No independence needed, which is good, because $Y^1$ and $Y^0$ belong to the same person and are definitely not independent. Average of differences = Difference of averages. Sums only: this lovely magic doesn’t work for products or rations. Calibrate your expectations (lol) ↩