Skip to main content

Miscellaneous

On Precision and Recall​

Our model is saying “I can predict sick people 96% of the time”. However, it is doing the opposite. It is predicting the people who will not get sick with 96% accuracy while the sick are spreading the virus!

Do you think this is a correct metric for our model given the seriousness of the issue? Shouldn’t we be measuring how many positive cases we can predict correctly to arrest the spread of the contagious virus? Or maybe, out of the correctly predicted cases, how many are positive cases to check the reliability of our model?

This is where we come across the dual concept of Precision and Recall.

Precision tells us how many of the correctly predicted cases actually turned out to be positive.

Recall tells us how many of the actual positive cases we were able to predict correctly with our model.

Precision is a useful metric in cases where False Positive is a higher concern than False Negatives.

Recall is a useful metric in cases where False Negative trumps False Positive.

Recall is important in medical cases where it doesn’t matter whether we raise a false alarm but the actual positive cases should not go undetected!

On Fitting Parameters​

“When Dyson met Fermi, he quickly put aside the graphs he was being shown indicating agreement between theory and experiment.

His verdict, as Dyson remembered, was “There are two ways of doing calculations in theoretical physics. One way, and this is the way I prefer, is to have a clear physical picture of the process you are calculating. The other way is to have a precise and self-consistent mathematical formalism. You have neither.”

When a stunned Dyson tried to counter by emphasizing the agreement between experiment and the calculations, Fermi asked him how many free parameters he had used to obtain the fit. Smiling after being told “Four,” Fermi remarked, “I remember my old friend Johnny von Neumann used to say, with four parameters I can fit an elephant, and with five I can make him wiggle his trunk.” There was little to add.”

-- Segre G., Hoerlin B. The Pope of Physics: Enrico Fermi and the Birth of the Atomic Age

Simple R Code that Illustrates the Inverse Transform Theorem​

by Michael Kuehn, grad student extraordinaire at Georgia Tech 🐝

set.seed(1)
n <- 1e6

# Exponential
X <- rexp(n, rate = 1)
hist(X)
FX <- 1- exp(-X)
hist(FX)

# Using built-in CDF
FX <- pexp(X, rate = 1)
hist(FX)

# Normal
X <- rnorm(n, mean = 0, sd = 1)
hist(X)
FX <- pnorm(X, mean = 0, sd = 1)
hist(FX)

# t
X <- rt(n, df = 10)
hist(X)
FX <- pt(X, df = 10)
hist(FX)

# Cauchy
X <- rcauchy(n, location = 0, scale = 1)
hist(X, freq = F, breaks = "FD", xlim=c(-15,15))
FX <- pcauchy(X, location = 0, scale = 1)
hist(FX)

# Weibull
X <- rweibull(n, shape = 2, scale = 2)
hist(X)
Fx <- pweibull(X, shape = 2, scale = 2)
hist(FX)

# Gamma
X <- rgamma(n, shape = 3)
hist(X)
FX <- pgamma(X, shape = 3)
hist(FX)