Reliability does not equal predictive value

I guess because we're in the midst of Lyme season here in the Lyme rampant states today I googled "lyme disease" + test + research. The second "scholarly article" return was this one which I found interesting enough to read in full text (and to follow several linked references to and from).

Klempner, M. S., C. H. Schmid, L. Hu, A. C. Steere, G. Johnson, B. McCloud, R. Noring, and A. Weinstein. 2001. Intralaboratory reliability of serologic and urine testing for Lyme disease. Am. J. Med. 110:217-219.
Article Outline
Laboratory testing for Lyme disease is controversial because of problems with test sensitivity and specificity, the lack of standardized reagents, and interlaboratory and intralaboratory variability [see article for hyper-refs]. We determined the reliability of a serologic test and a urine test for Lyme disease, each performed in a reference laboratory, in control subjects and patients with Lyme disease who had posttreatment symptoms.

Material and Methods
Study Subjects
We studied 10 healthy control subjects who had never had Lyme disease and 21 patients with a history of acute Lyme disease, as defined by the Centers for Disease Control and Prevention [see article for hyper-refs], who had chronic (>6 month’s duration) fatigue, musculoskeletal pain, or neurocognitive impairment despite treatment with recommended antibiotics.

Sample Collection
Serum samples were obtained from all 21 patients and the 10 control subjects. One aliquot was immediately analyzed; duplicate aliquots were frozen at −70°C and tested within 6 months after collection.
...
Results
Serologic Test
In all 10 control subjects, the initial western blot analysis yielded negative results. In three of four duplicate specimens analyzed, the same immunoreactive bands seen in the original aliquot were present; 1 duplicate specimen contained a 41-kDa band that was not present in the original aliquot.

In the 21 patients with Lyme disease, the results of the initial western blot analysis were positive in 14 cases and negative in 7. Analysis of duplicate specimens yielded identical results in all 21 patients (κ = 1.0, Table 1). The same immunoreactive bands identified in the first analysis were present in 7 of the 14 seropositive duplicate samples; 5 samples had 1 additional band, and 2 samples had 2 additional bands. Repeat testing of the 7 seronegative samples showed fewer than 5 reactive bands in all samples.
...
To sum, using the "five band" criteria, the test-retest outcome was great when we consider just the 10 control subjects. Moreover, one might conclude that a negative blood test reliably predicts/detects the absence of disease in a person who does not have the disease. Further, a positive blood test reliably predicts/detects the presence of disease in people with it. However, a negative blood test is not so great a diagnostic tool for someone with the disease. That is, there are false negatives among the 21 experimental subjects, 7 false negatives in both the test and retest conditions (- reliably). That's one third of experimental condition subjects who tested negative twice despite being diagnosed with Lyme. How do our authors interpret these findings? In explicit terms, very narrowly.

Discussion
Our study showed that testing of duplicate serum specimens from 21 patients with Lyme disease and 10 healthy controls by a single reference laboratory using a commercially available immunoglobulin G western blot kit gave 100% concordant results for seroreactivity and highly reproducible results for the identification of individual bands.

Yes, the results were "reproducible", thus the test is deemed reliable. What does this mean for practice? That is, should we extrapolate that the western blot is a good test for confirming (or disconfirming) a suspected diagnosis of Lyme Disease? The authors make a cited claim that "In patients with chronic symptoms of at least 6 months’ duration, the most appropriate serologic test for prior infection with B. burgdorferi is the immunoglobulin G western blot, which is recommended by the Centers for Disease Control as the final basis for determination of seroreactivity." which, given its context in the publication, serves to link the reliability of the western blot with it being a good tool for diagnosis.

So I guess if you think you have Lyme, you'd better hope you're not a member of portion of the population who may get a false negative test result.

For more (and quite interesting) reading on predictive value of tests as diagnostic tools (which is different from the reliability of such a test), search terms "sensitivity" and "reliability". A nice overview of the topics as they relate to diagnostic testing is given in this piece by Tze-Wey Loong titled "Understanding sensitivity and specificity with the right side of the brain".