Evaluation of three serum antibody enzyme-linked immunosorbent assays for Mycoplasma hyopneumoniae

Evaluación de tres pruebas de ELISA de anticuerpos en suero para Mycoplasma hyopneumoniae

Évaluation de trois épreuves d'anticorps de sérum ELISA pour le Mycoplasma hyopneumoniae

Keith R. Erlandson, DVM, MS; Richard B. Evans, PhD; Brad J. Thacker, DVM, PhD, MBA, Diplomate ABVP; Matthew W. Wegner, DVM, MS; Eileen L. Thacker, DVM, PhD, Diplomate ACVM

KRE: Carthage Veterinary Service, Ltd, Carthage, Illinois. RBE, BJT: Department of Veterinary Diagnostic and Production Animal Medicine, College of Veterinary Medicine, Iowa State University, Ames, Iowa. MWW, ELT: Department of Veterinary Microbiology and Preventive Medicine, College of Veterinary Medicine, Iowa State University, Ames, Iowa. Corresponding author: Dr Eileen L. Thacker, 2118 Vet Med Building, Dept VMPM, College of Veterinary Medicine, Iowa State University, Ames, IA 50011; Tel: 515-294-5097; Fax: 515-294-8500; E-mail: ethacker@iastate.edu.

Cite as: Erlandson KR, Evans RB, Thacker BJ, et al. Evaluation of three serum antibody enzyme-linked immunosorbent assays for Mycoplasma hyopneumoniae. J Swine Health Prod. 2005;13(4):198-203.
Also available as a PDF.

Keywords: swine, Mycoplasma hyopneumoniae, ELISA, diagnostic tests, sensitivity
Search the AASV web site for pages with similar keywords.

Received: June 26, 2003
Accepted: January 5, 2004

Mycoplasma hyopneumoniae is an economically important pathogen affecting swine production worldwide.¹ Alone, M hyopneumoniae causes mild bronchopneumonia, but may exacerbate pneumonia caused by viral pathogens such as porcine reproductive and respiratory syndrome virus (PRRSV).² Infection with M hyopneumoniae enables invasion by secondary bacterial pathogens such as Pasteurella multocida.³ For these reasons, there is much interest in surveillance and testing for M hyopneumoniae.

There are currently several commercially available enzyme-linked immunosorbent assays (ELISAs) based on detection of antibodies to M hyopneumoniae in swine serum. Unfortunately, because M hyopneumoniae attaches to the ciliated respiratory epithelium and is not invasive, the serum antibody response to the bacteria may be variable. This variable antibody response leads to problems with assay interpretation, especially due to false-negative results.

The relationship between the results of the various diagnostic assays and the presence or absence of M hyopneumoniae in swine herds has been controversial. Although ELISAs for detection of M hyopneumoniae antibodies are extensively used, there is little information in the scientific literature as to the epidemiological usefulness of these tests.^4-7 The objective of this study was to compare the performance of three ELISAs used in the United States in detecting M hyopneumoniae antibodies in serum samples from pigs confirmed to be negative for M hyopneumoniae serologically, by culture, and by polymerase chain reaction (PCR), and from vaccinated and nonvaccinated experimentally inoculated pigs.

Materials and methods

Serum samples

Archived serum samples from 51 experimentally infected pigs (serum obtained from 10 unvaccinated pigs <= 21 days postinfection and 41 vaccinated pigs > 21 days postinfection) and 17 known-negative serum samples from unvaccinated pigs, were tested using three ELISAs. The experimentally infected pigs had been inoculated intratracheally with M hyopneumoniae strain 232 (a derivative of strain 11) during previous studies.^8,9

Serological tests

The three antibody assays evaluated included a Tween-20 ELISA⁴ and two commercially available ELISA tests: the HerdChek Mycoplasma hyopneumoniae (Idexx Laboratories, Westbrook, Maine) and the DAKO Mycoplasma hyopneumoniae ELISA (DAKO Corporation, Carpenteria, California). The Tween-20 and HerdChek tests are indirect ELISAs, whereas the DAKO Mycoplasma hyopneumoniae test is a blocking ELISA. The DAKO test available in the United States differs from the test available in other countries. A different substrate is used, and the test stop solution is not included and must be prepared by the laboratory performing the test. This may be problematic to the accuracy of the DAKO ELISA, which may vary, especially between laboratories performing the test.

Classification of the M hyopneumoniae serostatus of a sample for the Tween-20 ELISA was based on the optical density (OD) value of the sample, as is commonly done.⁴ All samples were performed in duplicate, and the average OD of the two wells was used to determine M hyopneumoniae serostatus. The positive control was "normalized" to an OD of 0.4, and a formula was used to adjust the sample OD on the basis of the positive sample adjustment. A positive result with the Tween-20 was defined as an OD of >= 0.24, ODs of 0.20 to 0.23 were classified as suspect, and ODs of < 0.20 were classified as negative.

It is also possible to use a sample-to-positive (S:P) ratio for classification of samples with the Tween-20 ELISA. Sample-to-positive ratios of >= 0.5 were considered positive, S:P ratios of < 0.5 to >= 0.4 were classified as suspect, and S:P ratios of < 0.4 were considered negative.

The HerdChek test bases the sample classification on the S:P ratio. The S:P ratio is defined as (sample OD - negative control OD) (positive control OD - negative control OD). Sample-to-positive ratios of >= 0.4 were considered positive, S:P ratios of < 0.4 but >= 0.3 were classified as suspect, and S:P ratios < 0.3 were classified as negative. All samples were performed in duplicate, and the average of the two wells was used to calculate the S:P ratio.

The DAKO ELISA test result is based on the comparison of the sample OD to the OD of the buffer control, yielding a percent inhibition value. A sample was classified as positive if the percent inhibition of the sample was <= 50% of the buffer control. Samples with an OD > 50% of the buffer control were classified as negative. The DAKO test does not classify samples as suspect, but the manufacturer's instructions include the caveat that when ODs of serum samples are between 50% and 65% of the buffer control, animals should be retested in 2 weeks. All samples were performed in duplicate, and the average OD of the two wells was used to calculate the percentage of the buffer control OD.

Analysis of test properties

Kappa. The kappa coefficient is a method of determining agreement between observers (tests in this case) beyond that of chance. It should be noted that the kappa coefficient measures the strength of agreement between tests: it does not provide a comparison to a "gold standard" or known disease status. Therefore, two assays may have a high level of agreement (high kappa score), but be equally poor at detection of disease or correctly classifying negative animals. The three assays were evaluated with the kappa coefficient using published benchmarks,¹⁰ where a kappa value = 0 is classified as poor agreement; kappa values from 0.01 to 0.20 constitute slight agreement; kappa values from 0.21 to 0.40 constitute fair agreement; kappa values from 0.41 to 0.60 constitute moderate agreement; kappa values from 0.61 to 0.80 constitute substantial agreement; and kappa values >= 0.81 constitute perfect agreement.

Receiver operating characteristic curves. The receiver operating characteristic (ROC) curve is a method of test evaluation and cutoff selection. The sensitivity of a test (true-positive rate) is plotted on the x-axis while 1- specificity (false-positive rate) is plotted on the y-axis at multiple possible cutoff points. A curve is then drawn connecting these points and the area under the curve (AUC) is calculated. The ROC curve can be used to select a cutoff point that represents the objectives of the tester. If a high specificity is desired, a cutoff can be selected to maximize specificity, but some sensitivity may be sacrificed. Likewise, a cutoff can be selected to maximize sensitivity, usually at the expense of some specificity, or a cutoff can be selected for maximum sensitivity and specificity. The area under the ROC curve for each test can be compared to determine the relative performance of the tests.

ROC curve analysis was performed using the freeware program ROCKIT (University of Chicago, Chicago, Illinois). ROCKIT uses the Alf-Dorfman maximum likelihood method for estimation of AUC¹¹ and the procedure described by Hanley and McNeil¹² for comparison of AUCs.

Covariance of tests. When tests evaluating disease status are based on the same or similar biologic processes, such as serum antibody levels in the case of M hyopneumoniae ELISAs, it is logical to assume that they are correlated to some degree. This correlation is known as the test dependence.¹³ A test of covariance can be used to determine the degree to which the tests are dependant. In this study, covariance of the tests was estimated using the procedure described by Gardner et al.¹³ The covariance value is directly related to the magnitude of the dependence, so covariance is expressed as the percentage of the maximum possible value of the covariance.¹³

Combination tests. The performance of test combinations was evaluated using the method described by Gardner et al.¹³ When using tests in parallel ("OR" testing scheme), one positive test result classifies the animal as positive. When testing in series ("AND" testing scheme), all tests must be positive for the animal to be classified as positive.¹³

The dependence of the tests is important when multiple tests are used. If tests are highly dependent (eg, two ELISAs testing for serum antibody), there may be little information to be gained by multiple tests. However, if two tests have a low dependence (eg, ELISA for serum antibody and PCR for detection of the organism), more information can be gained from using multiple tests.

Statistical analysis

For statistical analysis in this study, suspect samples were considered negative, because we consider suspect samples to be a subset of negative samples, and the suspect class-ification is useful for identification of animals for further testing. Statistics were computed using SAS version 8.02 (SAS Institute, Cary, North Carolina), with the exception of ROC curves, which were evaluated using the freeware program ROCKIT. A parametric (cumulative Gaussian) model was used, with a univariate z-test to compare areas under the ROC curve. The Bonferroni correction for multiple test comparisons indicated that statistical differences were identified at P < .02.

Results

Test properties

Table 1 provides a summary of test results for the three ELISAs on the basis of the M hyopneumoniae status of the pigs. All three assays correctly identified known-negative samples, resulting in a specificity of 1 for all three tests. Table 2 provides a summary of the properties of the three assays.

Table 1: Summary of serum positive (+) and negative (-) test results for three Mycoplasma hyopneumoniae ELISA assays performed on serum from naive and experimentally inoculated pigs grouped by infection status1 1 Naive pigs were negative for M hyopneumoniae serologically, by culture, and by polymerase chain reaction. Serum from challenged groups was obtained from unvaccinated pigs <= 21 days postinfection (early infection) and from vaccinated pigs > 21 days postinfection (late infection). 2 Tween-20 Mycoplasma hyopneumoniae ELISA; positive sample, optical density >= 0.24 or sample-to positive (S:P) ratio >= 0.5. 3 HerdChek Mycoplasma hyopneumoniae (Idexx Laboratories, Westbrook, Maine); positive sample, S:P ratio >= 0.4. 4 DAKO Mycoplasma hyopneumoniae (ELISA; DAKO Corporation, Carpenteria, California); positive sample, % inhibition <= 50%.

Table 2: Summary of the sensitivity (%) and specificity (%) results for three Mycoplasma hyopneumoniae ELISA assays performed on serum from experimentally inoculated and naive pigs grouped by infection status1 1 Serum obtained from 10 unvaccinated pigs <= 21 days postinfection (early infection) and 41 vaccinated pigs > 21 days postinfection (late infection), plus 17 known-negative serum samples from unvaccinated, unchallenged pigs negative for M hyopneumoniae serologically, by culture, and by polymerase chain reaction. 2 Tween-20 Mycoplasma hyopneumoniae ELISA. 3 HerdChek Mycoplasma hyopneumoniae; Idexx Laboratories, Westbrook, Maine. 4 DAKO Mycoplasma hyopneumoniae ELISA; DAKO Corporation, Carpenteria, California.

Table 3: Summary of kappa coefficient for agreement between tests for three Mycoplasma hyopneumoniae ELISA assays performed on serum from experimentally inoculated pigs grouped by infection status1 1 Serum obtained from 10 unvaccinated pigs <= 21 days postinfection (early infection) and 41 vaccinated pigs > 21 days postinfection (late infection). 2 Tween-20 Mycoplasma hyopneumoniae ELISA. 3 HerdChek Mycoplasma hyopneumoniae; Idexx Laboratories, Westbrook, Maine. 4 NA (not applicable) and NS (nonsignificant): agreement between tests was no better than chance. 5 DAKO Mycoplasma hyopneumoniae ELISA; DAKO Corporation, Carpenteria, California.

Kappa. Table 3 shows the kappa coefficients for the tests. Comparison test results for all sera from experimentally challenged pigs (unvaccinated and vaccinated) resulted in substantial agreement between the Tween-20 and the HerdChek tests, moderate agreement between the HerdChek and DAKO tests, and fair agreement between the Tween-20 and DAKO tests.

Agreement between the HerdChek and DAKO tests for early-positive samples was moderate. For early-positive samples, agreement between the Tween-20 and the HerdChek and between the Tween-20 and the DAKO was no better than that expected by chance. Agreement between tests was largest when the results for vaccinated late-positive samples were compared. There was substantial agreement between the Tween-20 and the HerdChek tests, moderate agreement between the HerdChek and DAKO tests, and fair agreement between the Tween-20 and DAKO tests.

ROC curve analysis. Table 4 provides a summary of the Wilcoxon estimate of the AUC, as calculated by ROCKIT, for the three tests on all serum samples from experimentally infected pigs. Table 5 summarizes the maximum likelihood estimation of the AUC. The maximum likelihood estimates differ due to the numerical accuracy of the algorithm ROCKIT uses to compute them.

Table 4: Summary comparison of the Wilcoxon estimate of area under the curve (AUC) test evaluation for three Mycoplasma hyopneumoniae ELISA assays performed on serum from experimentally inoculated pigs1 1 Serum obtained from 10 unvaccinated pigs <= 21 days postinfection and 41 vaccinated pigs > 21 days postinfection. Wilcoxon estimate calculated by ROCKIT (University of Chicago, Chicago, Illinois). 2 Tween-20 Mycoplasma hyopneumoniae ELISA. 3 HerdChek Mycoplasma hyopneumoniae; Idexx Laboratories, Westbrook, Maine. 4 DAKO Mycoplasma hyopneumoniae ELISA; DAKO Corporation, Carpenteria, California. ab Values with different superscripts are significantly different (P = .01)

Table 5: The maximum likelihood estimation of the area under the curve (AUC), A(z) (an estimation of the area under the fitted smooth curve), and the 95% asymmetric confidence interval (CI) for three Mycoplasma hyopneumoniae ELISA assays performed on serum from 51 experimentally inoculated pigs1 1 Serum obtained from 10 unvaccinated pigs <= 21 days postinfection (early infection) and 41 vaccinated pigs > 21 days postinfection (late infection). Wilcoxon estimate of the AUC was calculated by ROCKIT (University of Chicago, Chicago, Illinois). 2 Tween-20 Mycoplasma hyopneumoniae ELISA. 3 NA = not applicable. 4 HerdChek Mycoplasma hyopneumoniae; Idexx Laboratories, Westbrook, Maine. 5 DAKO Mycoplasma hyopneumoniae ELISA; DAKO Corporation, Carpenteria, California.

The AUC for the Tween-20 test was significantly smaller than that for the DAKO and the HerdChek (Table 4), indicating that the Tween-20 ELISA has the poorest potential performance of the three tests. The AUC of the HerdChek was not significantly different from the AUC of the DAKO, indicating that the optimum performance for these two tests is equal.

Covariance of tests and interpretation of multiple tests

Covariance value and percentage of maximum covariance are shown in Table 6.

Table 6: Comparison of covariances measuring assay correlation for three Mycoplasma hyopneumoniae ELISA assays performed on serum from experimentally inoculated pigs1 1 Serum obtained from 10 unvaccinated pigs <= 21 days postinfection (early infection) and 41 vaccinated pigs > 21 days postinfection (late infection). 2 Tween-20 Mycoplasma hyopneumoniae ELISA. 3 HerdChek Mycoplasma hyopneumoniae; Idexx Laboratories, Westbrook, Maine. 4 DAKO Mycoplasma hyopneumoniae ELISA; DAKO Corporation, Carpenteria, California.

In this study, all known negative samples were correctly classified as negative by all three tests, resulting in a specificity of 1. For this reason, the covariances of the test specificities were not calculated, since the specificities of tests with perfect specificities are independent by definition.¹³ Table 7 provides a summary of the properties of the tests when used in combination.

Table 7: Estimation of sensitivity (%) for three Mycoplasma hyopneumoniae ELISA assays when performed in combination on serum from experimentally inoculated pigs1 1 Serum was obtained from 10 unvaccinated pigs <= 21 days postinfection (early infection) and 41 vaccinated pigs > 21 days postinfection (late infection). 2 When tests are performed in parallel, one positive test result classifies the animal as positive. When test are performed in series, all tests must be positive for the animal to be classified as positive. 3 Tween-20 Mycoplasma hyopneumoniae ELISA. 4 HerdChek Mycoplasma hyopneumoniae; Idexx Laboratories, Westbrook, Maine. 5 DAKO Mycoplasma hyopneumoniae ELISA; DAKO Corporation, Carpenteria, California.

Discussion

The performance of M hyopneumoniae ELISA tests is not well documented in the current literature, especially concerning performance on samples from vaccinated animals and animals in the early stages of infection. The objective of this study was to determine the relative ability of the three ELISA tests to detect infection by M hyopneumoniae using serum from naive and experimentally inoculated pigs. Use of samples from experimentally infected animals allowed for evaluation of the test's ability to detect infection during the period of maximal pneumonia.

All three of the assays evaluated had excellent specificities, correctly identifying negative samples, as no false-positives were detected in the samples assayed in this study. Sensitivity, which is based on the number of positive samples detected by each assay compared to the gold standard of being experimentally challenged, was much lower, ranging from 35% seropositive in all challenged pigs as detected by the Tween-20 assay to 63% seropositive as detected by the HerdChek ELISA. The sensitivities of the assays determined in this study were lower than or equivalent to those in other published reports.^4-7 The low sensitivities of these assays may be the result of the vaccinated status of some animals, and the early infection of others. It should be noted that the low sensitivities of these assays are not necessarily indicative of poor quality of the tests, but rather the variable and often delayed immune response generated by M hyopneumoniae.

ROC analysis showed that the HerdChek and DAKO ELISAs were not statistically different from each other and both were superior to the Tween-20 ELISA. It should be noted that positive and negative classifications in this study were based on the standard cutoffs of the ELISAs. It may be useful to modify cutoff points for classification of positive samples to improve the sensitivity of the assays tested.

The Tween-20 and the HerdChek tests had both the largest agreement as measured by kappa and the largest sensitivity covariance. This is logical, as these two tests are the same type of indirect ELISA, while the DAKO is a blocking ELISA.

The Tween-20 and DAKO tests had the smallest covariance (ie, they were less dependent), and had the highest sensitivity when used in a parallel interpretation scheme. Thus, if multiple ELISAs are to be used, the DAKO monoclonal blocking ELISA should be used in combination with either the HerdChek ELISA or the Tween-20 indirect ELISA to maximize sensitivity.

This study was limited by the small sample size, especially for evaluation of test properties on true-negative animals. Additionally, most pigs were necropsied at or near 28 days postinfection; thus, there was no opportunity to follow their serological status over time. In addition, there are differences in the DAKO assays performed in the United States and Canada; however, the Canadian assay was not investigated in this study. Further studies with more animals may be able to detect statistically significant differences in the three assays. Studies of longer duration would be beneficial to further quantify the serological reaction to M hyopneumoniae and test properties for detection of infection by serum testing.

Implications

• As M hyopneumoniae ELISA assays have a low sensitivity early in infection, care must be used when interpreting the results.
• A combination of serological assays may increase testing sensitivity and assist in the accurate diagnosis of M hyopneumoniae infection in a herd.

References

1. Ross R. Mycoplasmal diseases. In: Leman AD, Straw B, Mengeling WL, eds. Diseases of Swine. 7th ed. Ames, IA: Iowa State University Press; 1992:537-551.

2. Thacker EL, Halbur PG, Ross RF, Thangawongnuwech R, Thacker, BJ. Mycoplasma hyopneumoniae potentiation of porcine reproductive and respiratory syndrome virus-induced pneumonia. J Clin Microbiol. 1999;37:620-627.

3. Ciprian A, Cruz TA, de la Garza M. Mycoplasma hyopneumoniae interaction with other agents in pigs, and evaluation of immunogens. Arch Med Res. 1994;25:235-239.

4. Bereiter M, Young TF, Joo HS, Ross RF. Evaluation of the ELISA and comparison to the complement fixation test and radial immunodiffusion enzyme assay for detection of antibodies against Mycoplasma hyopneumoniae in swine serum. Vet Microbiol. 1990;25:177-192.

5. Nicolet J, Paroz P, Bruggmann S. Tween 20 soluble proteins of Mycoplasma hyopneumoniae as antigen for an enzyme linked immunosorbent assay. Res Vet Sci. 1980;29:305-309.

6. Sorensen KJ, Botner A, Madsen ES, Strandbygaard B, Nielsen J. Evaluation of a blocking Elisa for screening of antibodies against porcine reproductive and respiratory syndrome (PRRS) virus. Vet Microbiol. 1997;56:1-8.

*7. Sorensen V, Barfod K, Ahrens P, Friis NF, Feenstra AA, Pedersen MW, Feld NC, Jensen NE. Comparison of four different methods for demonstration of Mycoplasma hyopneumoniae in lungs of experimentally inoculated pigs. Proc IPVS Cong. Bangkok, Thailand. 1994:188.

8. Thacker EL, Thacker BJ, Boettcher TB, Jayappa H. Comparison of antibody production, lymphocyte stimulation, and protection induced by four commercial Mycoplasma hyopneumoniae bacterins. Swine Health Prod. 1998;6:107-112.

9. Thacker EL, Thacker BJ, Kuhn M, Hawkins PA, Waters WR. Mucosal and systemic characteristics of protective activity of a Mycoplasma hyopneumoniae bacterin. AJVR. 2000;61:1384-1389.

10. Landis JR, Koch G. An application of hierarchiacal kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977;23:363-374.

11. Dorfmann DD, Alf EJ. Maximum likelihood estimation of parameters of signal detection theory - a direct solution. Psychometrika. 1968;33:117-124.

12. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29-36.

13. Gardner IA, Stryhn H, Lind P, Collins, MT. Conditional dependence between tests affects the diagnosis and surveillance of animal diseases. Prev Vet Med. 2000;45:107-122.

* Non-refereed reference.