Keyword Search

About as simple and stupid as it gets. You look for fragments of text. Fast, cheap, reproducible, explainable.

Examples

Take these notes:

note	text
N1	Hives after amoxicillin in 2019. Penicillin allergy documented.
N2	No known drug allergies. Denies penicillin allergy.
N3	Allergic to PCN. Rash.
N4	Penicillin G started for syphilis.
N5	Annual physical. Labs pending.

In SQL

SELECT note_id, text FROM notes WHERE text ILIKE '%penicillin%';

ILIKE is case-insensitive LIKE. % means “anything (or nothing) here”. So this returns every note with “penicillin” somewhere in it: N1, N2, N4.

On the Command Line

grep -i "penicillin" notes.txt

Same shit. Good ‘ol grep has been doing this since 1973.

Multiple Words

SELECT note_id FROM notes
WHERE text ILIKE '%penicillin%' AND text ILIKE '%allergy%';

Returns N1, N2. Use OR to get anything with either word.

Why It Fails

Question: “Which patients are allergic to penicillin?” The right answer is N1 and N3.

note	matches %penicillin%?	actually allergic?	verdict
N1	✅	✅	correct
N2	✅	❌	false positive
N3	❌	✅	false negative
N4	✅	❌	false positive
N5	❌	❌	correct

Three of five wrong. Lol.

• N2 contains the word but denies the allergy. Keyword search has no idea what “denies” or “no” means.
• N3 is allergic but says “PCN”. Different string, no match. Same for “amoxicillin” (which is a penicillin), typos (“penicilin”), and other languages.
• N4 is just a prescription. Mentioning a word $\neq$ being about it.

And it gets worse with short fragments. Search for %pen% to catch abbreviations and you also get “pending” (N5), “appendectomy”, “penile”… Substrings don’t care about word boundaries.

Pros and Cons

Pros	Cons
Dead simple. Works in every database and every terminal	No meaning: misses synonyms, abbreviations (“PCN”), and misspellings
Fast and cheap. No model, no GPU, no upfront index	No idea about negation (“denies penicillin allergy” still matches)
Exact. Great for IDs, drug codes, MRNs, error messages	No ranking: a note either matches or it doesn’t
Fully explainable: “it matched because the word is in there”	Substrings match inside other words (%pen% → “pending”)
Nothing to go stale. Search the raw text directly	%word% can’t use a normal index, so it scans every row on big tables

Peeking ahead

The “no ranking” problem is what BM25 fixes.
The “no meaning” problem is what Vector Search fixes (sort of).