Tuesday, September 30, 2008
Wednesday, July 30, 2008
Reliability does not equal predictive value
I guess because we're in the midst of Lyme season here in the Lyme rampant states today I googled "lyme disease" + test + research. The second "scholarly article" return was this one which I found interesting enough to read in full text (and to follow several linked references to and from).
Klempner, M. S., C. H. Schmid, L. Hu, A. C. Steere, G. Johnson, B. McCloud, R. Noring, and A. Weinstein. 2001. Intralaboratory reliability of serologic and urine testing for Lyme disease. Am. J. Med. 110:217-219.
Article Outline
Laboratory testing for Lyme disease is controversial because of problems with test sensitivity and specificity, the lack of standardized reagents, and interlaboratory and intralaboratory variability [see article for hyper-refs]. We determined the reliability of a serologic test and a urine test for Lyme disease, each performed in a reference laboratory, in control subjects and patients with Lyme disease who had posttreatment symptoms.
Material and Methods
Study Subjects
We studied 10 healthy control subjects who had never had Lyme disease and 21 patients with a history of acute Lyme disease, as defined by the Centers for Disease Control and Prevention [see article for hyper-refs], who had chronic (>6 month’s duration) fatigue, musculoskeletal pain, or neurocognitive impairment despite treatment with recommended antibiotics.
Sample Collection
Serum samples were obtained from all 21 patients and the 10 control subjects. One aliquot was immediately analyzed; duplicate aliquots were frozen at −70°C and tested within 6 months after collection.
...
Results
Serologic Test
In all 10 control subjects, the initial western blot analysis yielded negative results. In three of four duplicate specimens analyzed, the same immunoreactive bands seen in the original aliquot were present; 1 duplicate specimen contained a 41-kDa band that was not present in the original aliquot.
In the 21 patients with Lyme disease, the results of the initial western blot analysis were positive in 14 cases and negative in 7. Analysis of duplicate specimens yielded identical results in all 21 patients (κ = 1.0, Table 1). The same immunoreactive bands identified in the first analysis were present in 7 of the 14 seropositive duplicate samples; 5 samples had 1 additional band, and 2 samples had 2 additional bands. Repeat testing of the 7 seronegative samples showed fewer than 5 reactive bands in all samples.
...
To sum, using the "five band" criteria, the test-retest outcome was great when we consider just the 10 control subjects. Moreover, one might conclude that a negative blood test reliably predicts/detects the absence of disease in a person who does not have the disease. Further, a positive blood test reliably predicts/detects the presence of disease in people with it. However, a negative blood test is not so great a diagnostic tool for someone with the disease. That is, there are false negatives among the 21 experimental subjects, 7 false negatives in both the test and retest conditions (- reliably). That's one third of experimental condition subjects who tested negative twice despite being diagnosed with Lyme. How do our authors interpret these findings? In explicit terms, very narrowly.
Discussion
Our study showed that testing of duplicate serum specimens from 21 patients with Lyme disease and 10 healthy controls by a single reference laboratory using a commercially available immunoglobulin G western blot kit gave 100% concordant results for seroreactivity and highly reproducible results for the identification of individual bands.
Yes, the results were "reproducible", thus the test is deemed reliable. What does this mean for practice? That is, should we extrapolate that the western blot is a good test for confirming (or disconfirming) a suspected diagnosis of Lyme Disease? The authors make a cited claim that "In patients with chronic symptoms of at least 6 months’ duration, the most appropriate serologic test for prior infection with B. burgdorferi is the immunoglobulin G western blot, which is recommended by the Centers for Disease Control as the final basis for determination of seroreactivity." which, given its context in the publication, serves to link the reliability of the western blot with it being a good tool for diagnosis.
So I guess if you think you have Lyme, you'd better hope you're not a member of portion of the population who may get a false negative test result.
For more (and quite interesting) reading on predictive value of tests as diagnostic tools (which is different from the reliability of such a test), search terms "sensitivity" and "reliability". A nice overview of the topics as they relate to diagnostic testing is given in this piece by Tze-Wey Loong titled "Understanding sensitivity and specificity with the right side of the brain".
Labels: Infectious Disease, Lyme Disease, research
Wednesday, December 19, 2007
fuel for the fire
Another Lyme headline. The part that spooks me is the third paragraph.
(excerpted from the Stamford Advocate)
Medical board approves probation, fine for controversial doctor
Associated Press
December 18 2007
HARTFORD, Conn. -- A New Haven pediatrician who has been praised by patients but criticized by the medical establishment for the way he treats Lyme disease was reprimanded, fined $10,000 and placed on two years probation by state regulators Tuesday.
The Connecticut Medical Examining Board voted unanimously to impose the sanctions after concluding that Dr. Charles Ray Jones violated care standards by diagnosing Lyme disease in a boy and his sister and prescribing antibiotics based on a phone conversation with their mother, months before he examined them in May 2004.
The board also found that Jones broke standards by failing to reconsider his diagnoses of the children after lab tests came up negative for the tick-borne disease, which can cause painful arthritis, meningitis and other serious illnesses if not treated promptly.
Board members further concluded that Jones was wrong to prescribe antibiotics for nearly a year without repeat exams and without any arrangement with another doctor, because the children lived in Nevada, to monitor for any side effects of long-term antibiotic therapy.
Hartford lawyer Elliott Pollack, who is representing the 77-year-old Jones, said he will appeal the board's decision.
...
Other things aside for a moment, I know it's hard but try, consider that part of the board's ruling was based on the finding that Dr. Jones "broke standards by failing to reconsider his diagnoses of the children after lab tests came up negative for the tick-borne disease". I thought that a Lyme diagnosis was supposed to be more based on clinical presentation than lab values. What crazy Lyme-loving website did I get that from? The CDC, among others.
Lyme disease is diagnosed based on symptoms, objective physical findings (such as erythema migrans, facial palsy, or arthritis), and a history of possible exposure to infected ticks. Validated laboratory tests can be very helpful but are not generally recommended when a patient has erythema migrans.
On the CDC Lyme website, this passage continues with a helpful link to the current, detailed recommendations on serological testing for Lyme. If you do click there, what you get is a report called "Recommendations for Test Performance and Interpretation from the Second National Conference on Serologic Diagnosis of Lyme Disease".
The Second National Conference on Serological Diagnosis of Lyme Disease was in October of 1994.
I'm going to let you all do the math. You'll do it faster than me. I nearly had to count it out on my fingers...good thing it wasn't much older or I'd have run out of fingers!
My point. There are two, or one somewhat bifurcated one depending on how you look at it. My point is that the CT board's finding engenders and perpetuates too high a level of confidence in serological testing for the clinician dealing with the realities of Lyme presentation. The board's level of confidence would be well founded if the testing had improved much in the over 13 years since the "current" test performance and interpretation recommendations were made. It has not. I wonder now as I so often do when I read the latest Lyme fueled controversy story (or maybe more appropriately, the latest controversy fueled Lyme story) - Can't we do better than this?
Labels: Infectious Disease, legal, Lyme Disease, Meds, News
Monday, October 15, 2007
hmosis
From Harvard Pilgrim Health Care's "Why Does Your Drug Require Prior Authorization?" page.
Why does Zyvox (linezolid) require prior authorization?
Zyvox is a very strong and costly antibiotic. It has been shown to be just as effective as other antibiotics currently available to treat infection, however because it is one of the few antibiotics shown to be effective against resistant gram positive infections, infectious disease experts caution that this drug should be used conservatively. The Harvard Pilgrim Prior Authorization Program helps to ensure that this drug is used only when medically necessary.
It's tangential to why I'm quoting Harvard Pilgrim Health Care's FAQs, but I have to say the statement that Zyvox "has been shown to be just as effective as other antibiotics currently available to treat infection" is so flagrantly misleading it's insulting to anyone with more than two neurons to rub together.
The relevance though is that my brother was denied his medication (zyvox) tonight when he went to pick it up from the pharmacy on his way home from the hospital. Two weeks inpatient and he's not sick enough for the HMO.
"Prior authorization" means he will spend hours on the phone tomorrow trying to get this cleared up. He'll have to do it himself because if someone tries to advocate for a patient, even a very sick patient, the bureaucrats bust out the HIPAA gags before you can even say "health care proxy".
So if my brother's got enough luck, tolerance, and energy to fend for himself with the HMO tomorrow, he might get the drug approved in time for his next dose.
I totally get not wanting people to hand out Zyvox like it's candy, but a vancomycin resistant MRSA pneumonia really looks like one of those situations for which Zyvox is the best thing going (1, 2) - better than the other antibiotics Harvard Pilgrim's website nebulously refers to.
Also, I've got to wonder...does it really need to be this hard for the doctor, the pharmacy, and the various departments (or rather, databases) at the HMO to get in the same chapter if not on the same page? It's not like we have to wait for the pony express to deliver the document with the diagnosis on it from the hospital way out East to the HMO head office....er, way out East. We have electronic databases, intranet, and internet. Even the slower connections and programs on poorly maintained 3 years out of date systems (what I hypothesize are used in many if not most medical contexts) are usually faster still than a bike courier at 3 on a Sunday morning in Boston. Point is, I'll bet if an insurance company had the proper motivation (i.e. business related) to facilitate communication between it's departments (e.g., that which handles inpatient care claims and that which handles prescription coverage), we'd see that info moving so fast you'd think it was magic.
Update 10/16/07, 7:12 PM. Hospital has faxed "the form" and the HMO is doing whatever it does which we would consider mulling it over. And my brother is going without antibiotics for two doses and counting.
Labels: Infectious Disease, Insurance, Meds