Neyman

Here we focus on the average causal effect size. Fisher was a smoke test: “Is anything there?” This is a “How much is there and how confident are you?”

The Null Hypothesis here is “The average effect is zero.” Let’s use SACE. The null here is:

$$\text{SACE} = \frac{1}{N} \sum_{i=1}^N(Y_i^1 - Y_i^0) = 0$$

Remember that this is a God Mode claim. We cannot observe both potential outcomes (the Fundamental Problem of Causal Inference) and have to deal with that. We do this by putting a hat on SACE, the estimand, and estimating it as $\widehat{SACE}$, the estimator. A number will pop out. It is the estimate.

The Old Example

Take the old example from Fisher. Same table.

Student	Treatment $A$	$Y^0$	$Y^1$
Alice	1	?	8
Ben	0	6	?
Carol	1	?	4
David	0	2	?
$\bar{Y}^a$		4	6

Now with Fisher, we were able to fill in in those ? with the corresponding values because of the Sharpass Null ($Y^1 = Y^0$). We are not doing that here.

Instead, we will go by column, leave the ? in place, and average across everyone for each potential outcome. Then the average effect is $6 - 4 = +2$. Let’s call this $T_{\text{Observed}}$

$$T_{\text{Observed}} = \bar{Y^1}_{\text{Observed}} - \bar{Y^0}_{\text{Observed}}$$

Now what about shuffling assignments like we did with Fisher? There are 5 more left! No FOMO. We don’t do that. Instead, we focus on