Sep 15 - EHR Audit Logs
by Thomas Kannampallil, WUSTL Medicine.
This is what a deidentified EHR Audit Log looks like. It's a table of events. A clinican can generate a 1,000 of these events a day.
| ACCESS_TIME | USER_ID | PAT_ID | METRIC_NAME |
|---|---|---|---|
| 12/21/19 7:21 | M58530 | Inpatient Patient Lists list loaded | |
| 12/21/19 7:21 | M58530 | Inpatient system list accessed | |
| 12/21/19 7:21 | M58530 | A | Storyboard viewed |
| 12/21/19 7:21 | M58530 | A | SmartSets activity selected |
| 12/21/19 7:21 | M58530 | A | Visit Navigator template loaded |
| 12/21/19 7:21 | M58530 | A | Problem List accessed |
| 12/21/19 7:21 | M58530 | A | Chart Review Encounters tab selected |
| 12/21/19 7:21 | M58530 | A | Chart Review Notes tab selected |
| 12/21/19 7:21 | M58530 | A | Chart Review Note report viewed |
| 12/21/19 7:22 | M58530 | A | Report with patient data viewed |
| 12/21/19 7:23 | M58530 | A | Notes viewed |
| 12/21/19 7:23 | M58530 | A | Visit Navigator template loaded |
| 12/21/19 7:25 | M58530 | A | In Basket message created |
| 12/21/19 7:25 | M58530 | A | Pend clinical note |
A very early paper on clinical log analysis was done in 2003 by students in the department.
You can certainly use these logs to get a lot of information but they can have severe limitations and it depends a lot on the problem you're trying to solve. In the speaker's example, they were trying to predict burnout in clinicians and used survey scores which had AUROC scores of 0.8+ compared to 'typical' models like Random Forest, etc that lingered around 0.5. Even deep learning approaches did not work as well. Theirs were a very complex problem. Might require causal modeling.
"APPs" → Advance-Practice Providers. "Physician assistants, nurse practitioners, nurse anesthetists, nurse midwives and clinical nurse specialists all fall under the category of APPs [#]".
In a lovely irony, the increase in the use of (secure) messaging with all its blessings (async, 'efficient') didn't really help with burnout or the amount of time clinicians spent on the phone actually talking to each other; it increased! Speaker notes that this is associative and not causative.
Grand Challenges with this kind of informatics
- Large-scale multicenter studies. Regulatory issues around large-scale data sharing.
- Need for data models and standards.
- Building standard data pipelines.
- Adapting behavioral theories from cognitive sciences. Action-as-a-language framework and the use of LLMs. Measure for clinician behaviors.
TODO: Review Prospective versus Retrospective studies.
TODO: Get a 10,000ft overview of Large Action Models.
TODO: You've never done a Mixed-Effects Regression Model before.