As most readers of this blog are aware, I’m an amateur student of the psychology of decision-making (in fact, there a tag for this over on the right). Having just finished Nate Silver’s excellent book, The Signal and the Noise, I’ve got Bayes’ Theorem on my mind and have been thinking about how it can be applied in practice.
Composite case: A 72 year old man presented with acute left sided numbness and weakness, last known well 2 hours before. He has all of the usual vascular risk factors, and these are poorly-controlled. The exam, however, is quite confusing. He’s agitated, combative, and disoriented. There is no clear neglect. Visual field testing is inconsistent across numerous attempts, sometimes suggesting a monocular problem on the left, sometimes a left sided field cut, and sometimes no clear abnormality at all. Other cranial nerve functions are normal. He says that he can’t move his left side, but with encouragement he produces 3/5 strength consistently, and can occasionally generate 4/5 strength . He reports left sided numbness. There are no reflex asymmetries or Babinski signs.
Non-contrast head CT shows extensive vascular calcifications, but no acute abnormalities.
Do we give tPA?
It’s common in neurology to obtain a confusing exam. Sometimes this is because we don’t recognize the pattern of deficits. I’m told that multiple sclerosis was frequently misdiagnosed as psychogenic in etiology, since its multifocality cuts against our usual imperative to “localize the [singular] lesion”. Sometimes difficulty arises because the patient overlays psychologically-mediated behaviors on top of organic deficits. And of course sometimes all of the exam findings are psychogenic.
If we interpret the exam above as suggesting a combination of organic and psychogenic deficits, we might withhold a definitive stroke diagnosis and tPA treatment. Can a Bayesian approach help us here?
The first step in a Bayesian approach is to estimate the prior probability of the diagnosis. If the patient were young and healthy, the prior probability would be very low. When the patient is old and has a plethora of poorly-controlled vascular risk factors, however, the prior probability is high. So, in the example above we have an elderly man with extensive vascular risk factors who presents with acute, lateralized neurological symptoms. And he has cerebrovascular calcifications seen on CT. I think it’s fair to estimate the prior probability of stroke at 90%.
Then, we then modify our assessment in light of each new piece of information. Unlike many blood tests, we don’t have sensitivity and specificity data for every component of the clinical evaluation. The Gill paper references some older literature on the clinical evaluation of ascites, abdominal aortic aneurysm, etc. The main conclusion is that clinical examination is far from perfect. “Test” characteristics (sensitivity, specificity, etc.) for various exam maneuvers vary according to the severity of disease and other factors, but can be as low as 28% and as high as 90%.
The figure below illustrates the critical relationships among the prior probability of the disease, the test characteristics of the neurological exam, and the negative predictive value of a “negative” neurological exam (since in this example we’re trying to rule out stroke with our exam).
On the x-axis are the sensitivity and specificity of the neurological exam, which I’ve combined into one number as a simplification. (In reality, there is a trade-off between the sensitivity and specificity of a test, but here I’m more concerned with the basic relationships than the precision of the numbers. Which is another way of saying that I’m not a statistician, but would welcome any corrections to this that are more precise). If the sensitivity and specificity are 95% it means that if a person has a stroke, we’re 95% likely to find it on exam and if he doesn’t have a stroke, we’re 95% likely to correctly rule it out. If these are 70%, then we’ll only make a clinical diagnosis of stroke 70% of the time that the patient actually has one, and we’ll incorrectly say he didn’t have a stroke 30% of the time.
The y-axis is the negative predictive value of the exam. If this is 90%, it means that when our exam is “negative” for stroke, it is 90% likely that the patient in fact does not have a stroke and 10% likely that he does. If it is 20%, it means that it is 20% likely that he doesn’t have a stroke at 80% likely that he does.
The blue line shows the relationship between sensitivity/specificity and negative predictive value when the prior probability of stroke is 90%. The red line is for a prior probability of 50%. You can see that if our prior probability is high (blue line), the negative predictive value of our exam drops off sharply as the sensitivity and specificity of our exam goes down. When the prior probability is lower (red line), then even a less-than-perfect exam provides accurate results.
So it seems to me that a key question that we must ask ourselves is what are the sensitivity and specificity of our exam maneuvers? We’re always taught that the exam reigns supreme–it’s the sine qua non of neurology, the queen of specialties. I’ll submit, however, that the accuracy of our exams likely depends on the situation. There are cases that are very clear and the exam is likely to be highly reliable–think of a case of suspected myelopathy where the exam shows spastic paralysis of both legs with marked hyperreflexia, bilateral Babinski signs, and a sensory level at the umbilicus. Then there are cases that are not clear, like the example above, where the exam is hard to interpret. I suspect that we often fall into the trap of believing that our beloved neurological exam always has very high sensitivity and specificity when in fact there are situations where the findings are ambiguous and the accuracy is consequently lower.
Getting back to our example, let’s stubbornly insist that we’re 95% sensitive and specific in detecting and ruling out stroke on the basis of our exam. The negative predictive value of our negative exam would be 68%. That means that there’s a one in three chance that we’re wrong and the patient has a stroke after all. That might not be enough to convince us to give tPA, but I think it’s certainly humbling.
However, let’s indeed be more humble and estimate that in this situation, with an agitated, combative patient showing us some focal signs but a lot of inconsistent behaviors, our sensitivity and specificity are only 70%. Now, our negative predictive value is only 21%. Because of the powerful effect of the high prior probability, there’s a 79% chance that we’ve incorrectly excluded stroke as the diagnosis.
So, who wants to give tPA now? It’s very disconcerting to think about administering a potentially dangerous medication to someone whom you’ve just examined and don’t have a clear understanding of where their lesion lies. Nonetheless, if the history and ancillary data available at the time are strongly suggestive (i.e., if the prior probability is very high), it may indeed be appropriate to render a stroke diagnosis anyway and administer the drug. Providing further reassurance is evidence suggesting that it is safe to administer tPA to patients with stroke mimics.