From the Editor |
This is the first in a series of articles in which I'll discuss how to critically evaluate and interpret articles you read in the scientific literature. In this column, I'd like to offer you an alternative paradigm for approaching scientific articles.
Articles that appear in a refereed journal such as Swine Health and Production are intended to inform. The scientific literature is the place where we communicate and archive new knowledge. Once the results of a trial are in print, they are a permanent source of information. However, scientific articles are really also an argument--a document whose ultimate aim is to persuade you to adopt the viewpoint of the author.
It is the author's job to establish a research design that will allow him or her to make observations that are free from confounders and invalidating conditions. But, there is no perfect study. Therefore, these observations, made in the context of an original research study, should be regarded as inherently subject to the biases that impinge on even the most careful and conscientious researcher and the most carefully planned research project.
The scientific method was established precisely to correct for those inevitable biases and to ensure validity. A well-designed study will control confounding factors so that the only difference between the treated pigs and the control pigs is the one factor of specific interest. This is why it is so crucial to approach the Materials and Methods section of every article with a skeptical eye. The criteria used to determine the influence of the "treatment" is statistical inference. Are you confident that the author has used the appropriate statistical test and controlled for confounders? Remember, any weaknesses in the Materials and Methods limit the strength of the claims an author can make in the Results. The Results section needs to convey all of the observations on every animal that began the study.
Once you are convinced that the Materials and Methods provide the conditions to support the claims made in the results, and the results are complete, you are ready to critically read the Discussion. Here the authors are engaged in a fundamentally different sort of task than their recounting of their study design (Materials and Methods) and their display of their observations (Results). When creating their Discussion, authors switch from an "observation mode" into an "interpretation mode." The Discussion is the combination of "science" and "art." Each statement in the Dicussion section should be regarded as an argumentative claim, and your job is to assess whether you believe they have provided adequate evidence in the rest of their article to back up that claim.
Here are some questions to ask yourself:
- Is the claim that an author is making in the Discussion grounded by something s/he measured and therefore could have observed in the course of his/her study?
- If the claim cannot be founded on something the author herself observed, then does the author cite a reference for another study in which such an observation could legitimately have been made?
Critical evaluation of the literature is conducted by asking pertinent questions in a systematic manner. These questions will depend on the type of study that was conducted. In future articles we will discuss these in detail.
--Cate Dewey
Executive Editor, Swine Health and Production