Types of Studies

Not meant to be exhaustive.

From various class notes. There will be a lot of stuff missing here. I’ll finish this one day lol.

You can zoom and pan this.

Broad Classes

Quantitative and Qualitative

This is about what data is collected.

Quantitative focuses on the measurable (statistics involved of course).

Qualitative is about observation (description, explanation) that produces artifacts (e.g. interviews, photos, sketches of spaces, notes) and may use some quantitative data. It can be exploratory research (see and refine questions, understand opportunities.) Can be used for evaluation (why are informatics interventions used or not used?)

Prospective and Retrospective

This is about the temporality of the data that is collected. Look forward or backward in time. One is cheaper because things have already happened. One gives you more control.

Observational and Experimental

Did you change anything in the world? Just observed it? Observational Study. Did you assign an exposure? Experimental.

Measurement and Demonstration

Measurement Study is where you evaluate the properties of a measurement instrument or method itself: these are the objects of investigation.

You typically use these later in Demonstration Studies where you want to show that an intervention, system, or technology works. The instrument or system is now the tool, and you’re investigating its impact.

Experimental and Quasi-Experimental Studies

Key distinction is randomization. Experimental studies randomly allocate subjects to exposure groups, which (with sufficient N) removes confounding. Quasi-experimental studies assign exposures without randomization — common when randomization is infeasible or unethical.

---

Observational

No exposure assigned (i.e. no influence on subjects). Descriptive. Analytical: give associations through comparisons. Can infer cause-effect relationships. Less common in interventions research unless you’re studying existing technologies/interventions.

Cross-Sectional Studies

Slice in time. There is no temporality in cross-sectional studies. It’s a slice in time. Because of this, there cannot be causal assertions. You can only speak of associations.

You can measure prevalence. You cannot measure incidence (incidence is about new cases, which implies temporality). Think of how timelines (e)------(o) get smushed into a spreadsheet — you just get (e)s and (o)s, the lines vanish.

                    ┌─── Single Time Point ───┐
                    │                         │
                    ▼                         ▼

   Population ──► Sample ──► Measure simultaneously:
                                ├── Exposure status (E+ / E−)
                                └── Outcome status  (O+ / O−)

                    Resulting 2×2:
                            O+      O−
                        ┌────────┬────────┐
                     E+ │   a    │   b    │
                        ├────────┼────────┤
                     E− │   c    │   d    │
                        └────────┴────────┘

   ├──────────────── time = t₀ ─────────────────┤
                  (no follow-up)

Cohort Studies

Select on exposure, follow for outcome. The key move: enforce temporality by design. These have the most information about time of any observational design.

                                    ┌──► Outcome+
                       ┌─ Exposed ──┤
                       │            └──► Outcome−
   Population ──►  Cohort
                       │            ┌──► Outcome+
                       └ Unexposed ─┤
                                    └──► Outcome−

   t₀ ─────────────────────── follow-up ───────────────────► t₁
   (define exposure)                              (ascertain outcome)

   Risk in exposed   = a / (a+b)
   Risk in unexposed = c / (c+d)
   Relative Risk     = [a/(a+b)] / [c/(c+d)]

Prospective: Study begins at point of exposure. You have a lot of control. You cannot always assume causality. Kinda costly for rare outcomes (gotta recruit a lot of people!). Loss-to-follow-up is a big problem.

Retrospective: Study begins at point of outcome. No control (lol). Inexpensive (already happened, someone else paid the bill). But you’re limited in terms of sampling and quality (already happened). And you cannot always assume causality. More prone to selection, information, and recall bias.

Ambidirectional (“Callback Studies”): Rare. “Weld” two timelines together — retrospective data combined with prospective follow-up. Data inconsistency is a headache. Inherits the bummers of both designs.

Multiple/Double Cohort: You have several cohorts with different levels of the same exposure. Lots of potential confounding risk here.

info

Cohort studies can start with cross-sectional studies where you can establish prevalence.

Case-Control Studies

Select on outcome, look back at exposure. The logical inversion of a cohort study. Always retrospective.

                            ┌── Exposed (a) ◄──┐
              Cases (O+) ───┤                  │
                            └── Unexposed (c) ◄┤
                                               │ recall /
                                               │ records
                            ┌── Exposed (b) ◄──┤
           Controls (O−) ───┤                  │
                            └── Unexposed (d) ◄┘

   ◄──────── look backward in time ────────  t₀ (sampling)

   Odds Ratio = (a·d) / (b·c)

You can study just one outcome. Big challenge is bias: you recruit your cases separately from controls (you can match, for example, to get over this problem). Doesn’t have the temporal ordering that cohort studies do.

Controls approximate what the cases would have looked like had they not developed the disease — it’s a bit counterfactual-y. Cases and controls must come from the same source population (don’t break links). This is why matching matters: cases are straightforward but controls are hard — you don’t want confounding by covariates or time.

When to use: Rare diseases, long latency periods. Think glioblastoma — a cohort study would require following participants for decades.

Metrics: You cannot compute Risk Ratios! You’re fixing the outcome distribution (setting the case/control ratio), so you don’t know $(a + b)$ and $(c + d)$. You can compute the Odds Ratio. For rare diseases, $RR \approx OR$.

Nested Case-Control

Say you follow 50,000 people over 20 years to see if an exposure caused a disease. Testing all of them would be expensive. What if you only tested people who didn’t get sick?

THE COHORT (already exists, already followed)

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
                    50,000 people
                        │
                        ▼
              follow them for 20 years
                        │
                        ▼
┌──────────────────────────────────────────────────┐
│  500 people develop the disease  =  CASES        │
│  49,500 people don't             =  potential    │
│                                     controls     │
└──────────────────────────────────────────────────┘
                        │
                        ▼
    For each case, randomly pick a few controls
    from people who were still disease-free at
    the moment that case got sick.
                        │
                        ▼
┌──────────────────────────────────────────────────┐
│  500 cases  +  ~2,000 controls (4 per case)      │
│                                                  │
│  NOW measure exposure on these 2,500 people.     │
│  Skip the other 47,500.                          │
└──────────────────────────────────────────────────┘

Sampling Strategies

The hardest part. The control group should be a snapshot of the baseline population.

Strategy	When Sampled	Notes
Case-Base / Case-Cohort	At start	Controls = random sample from source population at beginning. Use for very rare outcomes. Controls may develop the outcome.
Cumulative Density / Survivor	At end	Controls = people who survived without outcome. Problem: survivor bias.
Incidence Density / Risk Set	Each time a case occurs	For every incident case, randomly sample from risk set at that time. Preserves underlying risk process. Good for short-term or fluctuating exposures.

Crossover Studies

Cases serve as their own controls. Reduces random error and confounding. Think seasonal allergies — within-person comparison across time periods.

---

Experimental

Exposure is assigned by the researcher. Randomization is the dividing line between experimental and quasi-experimental.

Randomized Controlled Trial (RCT)

The gold standard. You assign exposure randomly and follow for outcome. Establishes temporality and (with sufficient N) removes confounding.

Key features:

• Stringent inclusion criteria
• Typically one primary predictor and one primary outcome — keep it simple
• Randomization only works with large N. Small samples may not remove all confounders.

                          ╔═══════════════╗
                          ║ RANDOMIZATION ║
                          ╚═══════╤═══════╝
                                  │
                ┌─────────────────┴─────────────────┐
                ▼                                   ▼
         ┌────────────┐                      ┌────────────┐
         │ Treatment  │                      │  Control   │
         │   (n=N/2)  │                      │  (n=N/2)   │
         └─────┬──────┘                      └─────┬──────┘
               │                                   │
               ▼                                   ▼
         ┌────────────┐                      ┌────────────┐
         │  Outcome   │  ◄─── compare ────►  │  Outcome   │
         └────────────┘                      └────────────┘

   t₀ ─── allocation ─── follow-up ─── outcome assessment ── t₁

Factorial Design

Assess the impact of multiple interventions (all permutations) without creating a separate arm for each. More efficient for multi-intervention testing.

                      Factor B: Behavioral coaching
                          B−                B+
                    ┌────────────────┬────────────────┐
   Factor A:    A−  │   Group 1      │   Group 2      │
   Drug             │   Placebo +    │   Placebo +    │
                    │   No coaching  │   Coaching     │
                    ├────────────────┼────────────────┤
                A+  │   Group 3      │   Group 4      │
                    │   Drug +       │   Drug +       │
                    │   No coaching  │   Coaching     │
                    └────────────────┴────────────────┘

   Main effect of A  =  (G3+G4)/2  −  (G1+G2)/2
   Main effect of B  =  (G2+G4)/2  −  (G1+G3)/2
   A×B interaction   =  (G4−G3) − (G2−G1)

Crossover Design

Each subject receives both treatment and control in sequence (order randomized). Subjects serve as their own controls. Reduces inter-subject variability. Washout period between treatments needed.

                    ┌── RANDOMIZE SEQUENCE ──┐
                    ▼                        ▼
        Sequence 1: A ──► [washout] ──► B
                    │                        │
                    └── outcomes Y₁ ─────────┴── outcomes Y₂

        Sequence 2: B ──► [washout] ──► A
                    │                        │
                    └── outcomes Y₁ ─────────┴── outcomes Y₂

   Period 1            Period 2            Period 3
   ├──── Tx ────┤├── washout ──┤├──── Tx ────┤
                 (clears carryover)

   Within-subject contrast:  Yₐ − Y_b  for each participant

Cluster Randomized Trial

Randomize at the level of groups (clinics, schools, communities) rather than individuals. Use when individual randomization is infeasible or risks contamination.

Population of clusters:  ◯ ◯ ◯ ◯ ◯ ◯ ◯ ◯
                            │
                            ▼
                   ╔═══════════════════╗
                   ║  RANDOMIZE CLUSTERS║
                   ╚═════════╤═════════╝
                             │
              ┌──────────────┴──────────────┐
              ▼                             ▼
       Treatment clusters             Control clusters
       ┌───────────────┐              ┌───────────────┐
       │  ◯ Clinic A   │              │  ◯ Clinic C   │
       │  ├ pt pt pt   │              │  ├ pt pt pt   │
       │  ◯ Clinic B   │              │  ◯ Clinic D   │
       │  ├ pt pt pt   │              │  ├ pt pt pt   │
       └───────────────┘              └───────────────┘
              │                             │
              ▼                             ▼
        Outcomes in pts  ◄── compare ──► Outcomes in pts

   Note: individuals within a cluster are correlated → ICC

Microrandomized Trials (MRT)

Used a lot in behavioural change interventions. Idea is to not split participant into intervention and control group. Each participant, whenever there is an opportunity for an intervention, is randomized.

Smaller sample sizes (think half) than RCTs. Learning effects mess this up, perhaps. However, in CDS, you are either receive information or not so this effect is not as profound here.

  Participant i:

   t₁     t₂     t₃     t₄     t₅     t₆     t₇    ...
    │      │      │      │      │      │      │
    ▼      ▼      ▼      ▼      ▼      ▼      ▼
   [R]    [R]    [R]    [R]    [R]    [R]    [R]      ← randomize at each
    │      │      │      │      │      │      │         decision point
    A      Ø      A      A      Ø      A      Ø       ← prompt / no prompt
    │      │      │      │      │      │      │
    Y₁     Y₂     Y₃     Y₄     Y₅     Y₆     Y₇      ← proximal outcome

   Estimand: causal excursion effect
            = E[Yₜ | Aₜ=1, Hₜ] − E[Yₜ | Aₜ=0, Hₜ]
   averaged over decision points and participants.

Sequential, Multiple Assignment, Randomized Trial (SMART)

Adaptive treatment design: multiple decision points with re-randomization based on response. Useful for building adaptive treatment strategies.

Here’s a more colorful picture.

                ╔═══════════════╗
                ║ RANDOMIZE  R₁ ║
                ╚═══════╤═══════╝
                        │
            ┌───────────┴───────────┐
            ▼                       ▼
        Treatment A             Treatment B
            │                       │
       evaluate response       evaluate response
       at week 8               at week 8
            │                       │
       ┌────┴────┐              ┌───┴────┐
       ▼         ▼              ▼        ▼
   Responder  Non-resp.     Responder  Non-resp.
       │         │              │        │
   Continue   ╔══════╗      Continue   ╔══════╗
      A       ║  R₂  ║          B      ║  R₂  ║
              ╚══╤═══╝                 ╚══╤═══╝
                 │                        │
              ┌──┴──┐                  ┌──┴──┐
              ▼     ▼                  ▼     ▼
            Aug.  Switch             Aug.  Switch
             A    to B                B    to A

   Builds dynamic treatment regimes:
     "Start A; if non-responder by wk 8, switch to B"

---

Quasi-Experimental

Exposure is assigned but not randomly. Common when randomization is infeasible or unethical. Ranked roughly from weakest to strongest:

Pre-Post Design

Weakest. Single measurement before and after intervention. Major threat: regression to the mean.

              ┌────────────┐         ┌────────────┐
              │  Pretest   │         │  Posttest  │
              │     O₁     │   ──►   │     O₂     │
              └────────────┘         └────────────┘
                    │                       ▲
                    │   Intervention X      │
                    └──────────►────────────┘

   Estimate of effect = O₂ − O₁

   ⚠ Threats: history, maturation, regression to mean,
              testing effects (no counterfactual)

Posttest Only

Measure only after intervention. No baseline — hard to attribute change to the intervention.

      Group ──► Intervention X ──► Posttest O₁

   t₀ ─────────── X ─────────────────────────► t₁

   ⚠ No baseline, no comparison group.
     Cannot establish change or causation; descriptive only.

Posttest Only with Controls

Add a concurrent control group but still no pre-test. Better than posttest only but you can’t verify group equivalence at baseline.

              Group A (self-selected) ──► X ──► O₁
                                                 │
                                                 │ compare
                                                 ▼
              Group B (self-selected) ────────► O₂

   ⚠ Selection bias is the dominant threat — groups may have
     differed before X. No pretest to verify equivalence.

Pre-Post with Concurrent Controls

Very common. Also called Interrupted Time Series. Sample → Treat → Measure → No Treat → Measure → …

  Treatment grp:   O₁ ──── X ────► O₂
                    │              │
                    │              │ Δ_T = O₂ − O₁
                    │              │
   Control grp:     O₃ ──────────► O₄
                                   │
                                   │ Δ_C = O₄ − O₃

                Difference-in-differences:
                   DiD = Δ_T − Δ_C
                       = (O₂−O₁) − (O₄−O₃)

   t₀ ──── pretest ──── X ──── posttest ──► t₁

   Controls for: secular trends, history, maturation
   Does NOT control for: differential selection on trajectory

Innate characteristics are eliminated as confounders. But: regression to the mean and learning effects remain. Not appropriate for drug efficacy.

Removed Treatment Design

Single group acts as its own control. Over equal time periods, you add and remove the intervention. Stronger than pre-post because the pattern of improvement/reversal supports causal inference.

  Phase:        Baseline    Treatment    Removal
                    │            │            │
   Time:    t₀ ────► t₁ ────────► t₂ ────────► t₃

   Group:    O₁ ─────► O₂ ──X──► O₃ ──×──► O₄
                       ▲         ▲         ▲
                       │         │         │
                  pre-Tx     during Tx   after Tx
                                        withdrawn

   Expected pattern if X works:
                            O₂          O₄
              O₁     ───►   ●           ●
              ●            /  \        /
                          /    \      /
                         /      \    /
              ────────●          ●●●
                     baseline   treatment   removal

   Effect = (O₃ − O₂) and reversal at (O₄ − O₃)

Double Pretest Pre-Post Design

Measure → wait → Measure again → Introduce intervention → Measure. Measurements must be equally spaced. The double pretest lets you detect pre-existing trends (“something funky going on”) before attributing change to the intervention.

              ┌──── pretests ────┐
   Group:    O₁ ────────► O₂ ──── X ────► O₃
              │            │              │
              │  baseline  │  intervention│
              │   trend    │   effect     │
              └────────────┘              │
                           └──────────────┘

   t₋₁ ─────── t₀ ─────────── X ─────────► t₁

   Uses (O₂ − O₁) to estimate the underlying trend,
   then asks: does (O₃ − O₂) exceed that trend?

   Helps rule out: maturation, regression to the mean,
                   pre-existing time trends

---

Bias

Think of the contingency table. Bias is systematic/structural distortion between exposure and outcome.

	O+	O-
E+	a	b
E-	c	d

Two things can go wrong: (a) who gets into the table (selection bias) and (b) how/where they are placed in the table (information bias).

Bias	What it is	Most prone design
Selection	Who gets into the study. Loss-to-follow-up is a subtype.	Retrospective; prospective (attrition)
Information	How subjects are classified in cells.	Retrospective
Recall	Differential accuracy of remembered exposures.	Retrospective, case-control
Survivor	Only survivors available for sampling.	Cumulative/survivor sampling in case-control
Regression to the mean	Extreme measurements tend toward average on re-test.	Pre-post designs

Misclassification

When subjects move between cells (e.g. $a \to b$, $c \to d$, or vertically).

• Non-differential: Misclassification is the same across groups. Biases toward the null (slightly better — you underestimate the effect).
• Differential: Misclassification rate differs between groups. Can distort in any direction.

---

Key Measures

Measure	Computable in…	Notes
Prevalence	Cross-sectional, case-control	Snapshot of who has what
Incidence	Cohort, RCT	Requires temporality
Risk Ratio (RR)	Cohort, RCT	$\frac{a/(a+b)}{c/(c+d)}$
Odds Ratio (OR)	Case-control (primary), all others	$\frac{a \cdot d}{b \cdot c}$; for rare diseases $RR \approx OR$
Incidence Rate	Cohort	Cases / person-time

---

Validity

• Internal Validity: How sound your statistical treatment is. Did your study measure what it set out to measure?
• External Validity: Generalizability. Do findings apply beyond your study population?

Also distinct from test validity, which is about instrumentation/measurement tools.

---

Simpler Diagrams

Using the amazing MonoDraw.

OBSERVATIONAL
-------------

Cross-Sectional
Population ──► Measure exposure + outcome once
                (single time point)

Cohort
Population ──► Exposed ───────► Follow over time ───────► Outcome?
           └─► Unexposed ─────► Follow over time ───────► Outcome?

Case-Control
Cases   = outcome present ─────┐
                               ├──► Look backward for exposure
Controls = outcome absent ─────┘

EXPERIMENTAL
------------

Randomized Controlled Trial (RCT)
Eligible participants ──► Randomize
                         ├──► Intervention ──► Outcome
                         └──► Control ───────► Outcome

Factorial Design
Eligible participants ──► Randomize
                         ├──► A only ────────► Outcome
                         ├──► B only ────────► Outcome
                         ├──► A + B ─────────► Outcome
                         └──► Neither ───────► Outcome

Crossover Design
Participants ──► Randomize
                ├──► Treatment A ─► Washout ─► Treatment B ─► Outcome
                └──► Treatment B ─► Washout ─► Treatment A ─► Outcome

Cluster Randomized Trial
Groups/clusters ──► Randomize
                  ├──► Cluster intervention ─► Outcome
                  └──► Cluster control ──────► Outcome

Microrandomized Trial (MRT)
Participant timeline:
T1 ── R ──► Intervention? ──► Response
T2 ── R ──► Intervention? ──► Response
T3 ── R ──► Intervention? ──► Response
T4 ── R ──► Intervention? ──► Response

SMART
Participants ──► Randomize
                ├──► Treatment A ─► Respond? ─┬──► Continue A
                │                             └──► Switch/intensify
                └──► Treatment B ─► Respond? ─┬──► Continue B
                                              └──► Switch/intensify

QUASI-EXPERIMENTAL
------------------

Pre-Post Design
O1 ──► X ──► O2

Posttest Only
X ──► O

Posttest Only with Controls
X  ──► O1
No X ─► O2

Pre-Post with Concurrent Controls
O1 ──► X    ──► O2
O1 ──► No X ──► O2

Removed Treatment Design
O1 ──► X on ──► O2 ──► X removed ──► O3

Double Pretest Pre-Post Design
O0 ──► O1 ──► X ──► O2