Research is the cornerstone of evidence-based decision-making. Clinical trials are an essential part of the research process; trials provide the highest evidentiary value of primary research studies for addressing intervention questions where it is feasible and ethical to allocate animals to intervention groups.1 However, information is only valuable if it is available. There is empirical evidence in human healthcare that inaccessible research is impacting its value and leading to research wastage.2 As an example, only half of the human health studies funded in the European Union between 1998 and 2006 resulted in identifiable research reports.3 Is availability a concern for swine veterinarians and researchers? If so, what can we do to improve research availability, and therefore increase the value of swine trial research? This article will explore two aspects of this issue: publication of results and access to research reports. Although we will focus on clinical trials, this discussion has applicability to other study designs.
Publication of trial results
The utility of research results to end users, including veterinarians, producers, and other researchers, requires the methods and results of the research to be available. A 2011 survey of 2137 veterinarians found that journals were the most common source of information for both clinicians (65.8%) and nonclinicians (75.6%).4 Several studies have evaluated the proportion of livestock research presented at conferences that is subsequently published as a journal article with publication rates ranging from 7.1% to 45.0%.5-7 Specific to swine research, Brace et al5 reported that only 5.6% (5 of 89) swine vaccine trials presented at the American Association of Swine Veterinarians (AASV) Annual Meeting between 1988 and 2003 were subsequently published as journal articles.
Presentation of research at conferences is important, as this provides a means of early dissemination of results, and as a forum to obtain input on findings and generate awareness. However, failure to subsequently publish full research results is problematic; in many instances, conference proceedings are restricted in length such that key details required to critically appraise the methodological rigor of the study are not provided. The results presented at conferences may represent preliminary, rather than final, results.
There is also empirical evidence that results may differ between conference proceedings and the subsequent journal article. This includes a tendency for trials with beneficial treatment effects to be published more often and more quickly than studies not showing beneficial treatment effects.8 In an evaluation of food safety trials in livestock species, trials with at least one positive outcome (ie, intervention benefit) were more likely to be published.6 Specific to swine, Brace et al5 found that 64% (57 of 89) of swine vaccine trials in conference proceedings reported that the vaccine was efficacious, compared to 80% (4 of 5) of trials reported in journal articles. Due to the low publication level, it was not possible to conclude that these percentages differ.
There is also evidence that, for the same study, details of the study differ between what is reported in a conference proceeding and what is reported in the subsequent journal article. Although there is no empirical evidence specific to swine, several studies in the broader veterinary literature have compared the methods and results of studies as reported in a conference proceeding to the journal article for the same study. In an evaluation of over 700 studies originally presented at the American College of Veterinary Surgeons Annual Meeting, and subsequently published as journal articles, the study outcome measures changed for 10% of the studies, including omission and addition of outcome measures.9 The study design changed between the conference abstract and the journal article for 6% of studies, most frequently because of the addition or omission of a control or experimental group. In some cases, the study results changed because of sample size. However, the study results also changed for 12% of studies when there was no change in sample size, intervention, outcome, or study design between the conference abstract and the journal article.9 In a study of 59 preharvest food safety trials which were subsequently published as journal articles, of the 231 outcome measures reported in both the proceedings and the article, different results were reported for 77 (33.3%), with 32 outcomes having a different direction of effect reported in the journal article.10 The overall conclusion on the efficacy of the intervention changed between the conference abstract and the journal publication for 10.7% of the trials. In a comparison of 384 studies reported at veterinary anesthesia conferences and subsequently published as journal articles, the overall conclusion as to whether the primary outcome was significant changed in 29 (7.6%) studies.11
There are several reasons why a study presented at a conference would not be subsequently published as a journal article. Not all manuscripts that are submitted to a peer-review journal are accepted; veterinary journals have a mean of 47% acceptance for articles submitted, with a mean of only 3% acceptance without the need for revisions.12 Thus, submission is not a guarantee of publication and authors need to be willing to commit time and effort to advance a manuscript to publication even after a manuscript has been submitted.
The most common reason for rejection of manuscripts submitted to human medical journals was problems with the study design, with the methods section containing the most flaws.13 Thus, it seems probable that at least some studies presented at conferences may lack the scientific rigor necessary for publication. Most scientific journals have a peer-review process in place to evaluate the methodological rigor of articles that are submitted. In this process, two or more individuals with expertise in the area evaluate the manuscript and provide comments, which the author can then use to modify the manuscript prior to acceptance by the journal. In some instances, peer reviewers may recommend that the manuscript be rejected due to major flaws. However, conference abstracts are not evaluated with the same rigor. Abstracts submitted for conferences may be evaluated based on fit with the conference themes as well as quality, and there is not a forum for back and forth between reviewers and authors for clarifications or modifications. Also, conference abstracts often have word count limits which preclude the comprehensive reporting of study methods that would be necessary for an evaluation of study validity.
Nonetheless, the main reason for studies not being published is that authors do not submit their research for publication.14,15 The most common reason for not submitting their research is lack of time.14,15 In a review of 6 studies on nonpublication, fear of rejection was a more common reason for nonpublication than journal rejection.14 Authors may also be hesitant to submit the results of trials where the results were not statistically significant.14,15 Not submitting study results for publication is problematic because it may lead to unnecessary duplication of efforts, waste limited resources, and potentially result in a loss of trust in the integrity of research conducted. Additionally, there are ethical concerns related to not publishing trial results. Sir Iain Chalmers, a champion of research quality who is one of the founders of the Cochrane Collaboration and a coordinator of the James Lind Initiative, has stated that not publishing research is scientific misconduct.16 In clinical trials, animals are allocated to treatment groups by the investigator. This means that some animals may receive an inferior treatment. This is justified on the assumption that the findings increase our knowledge; an assumption that is not met if the full results of a trial are not publicly available. In two recent systematic reviews evaluating the efficacy of preventive antibiotics to reduce respiratory disease in swine17 and vaccines targeted to bacterial respiratory pathogens,18 there were 105 (of 182) trials reported only in conference proceedings.
An issue related to publication of swine trials, and potentially other livestock and poultry industries, is the issue of private research. Although not explicitly documented, considerable research is undertaken by pharmaceutical companies on private farms or within large production systems, where the results deliberately are not published for proprietary reasons or because the results are intended to provide a competitive advantage to those conducting the research. This is a concept that does not really have an equivalent in human medicine, where clinical trials generally are conducted for the public good. In swine, the argument that publication is for the public good may hold true for research involving zoonotic diseases but is less obvious for research on production-limiting diseases and productivity. The argument also has been made that research funded with public monies should be published regardless of results, but again, this is not the case for in-house research. So, the dilemma is whether research results should be made available for the good of the industry or whether it is justified to not publish for competitive advantage. The answer is not in the scope of this article but may be an issue that swine veterinarians and the research community should consider.
Thus, some key messages are apparent related to publication of research. First, the empirical evidence illustrates that a substantive proportion of trials conducted in swine populations are not subsequently published in journals, and that there may be differences between trial results in conference proceedings compared to subsequent journal publications. This highlights the importance of submitting research results for publication. The evidence also provides important caveats for using trial results presented in conference proceedings. Publication is time consuming for researchers; however, without public dissemination of final results and a full presentation of the methodology, it is not possible to build a scientifically defensible body of knowledge to make evidence-based clinical decisions.
Access to research reports
Access to research reports is pertinent to two groups: researchers need to ensure that their work is accessible to those who need the information for clinical decision-making, and readers of research need to know how to access the results to make evidence-based decisions. There are several aspects of access, including knowing how to effectively search for publications, language of publication, and whether the research report (conference proceeding, industry report, or journal article) is freely available online or available via a charge or subscription.
Large volumes of articles are published every year, and it can be a challenge to find all the literature on a specific subject. There are tools available to help with searching the literature, including online databases which catalogue citations of research reports from journals (and other sources to some extent). However, not all articles are available through each database, and not all databases are freely available online. Therefore, searching for the literature can be complex. Grindlay et al19 evaluated the journal coverage of databases in veterinary literature and found that CAB abstracts (http://www.cabdirect.org) provide the highest coverage. Once databases have been identified, searches consist of identifying key words or phrases related to the topic of interest and combining those words in a search string using “AND”, “OR”, or “NOT” operators. Guidelines for searching the veterinary literature are available20,21 and certainly can be applied by academic researchers and those with access to a wide range of journals. However, searching and finding publications can be challenging for those without extensive journal access. It has been reported that approximately half of North American swine veterinarians interested in infectious disease research have access to 2 or fewer journals.22
Some research may not be available to end users because of the language in which the report was written; this may be because English-speaking individuals cannot read non-English publications or because non-English-speaking individuals cannot read English publications. However, English is recognized as the lingua franca of scientific publications. This is the case even for non-English speaking scientists; based on the results presented in 4 recent systematic reviews of trials addressing swine health topics, a substantial proportion of the trials were conducted in non-English speaking countries but published in English (7 of 20 trials in a review of preventive antibiotics for respiratory disease17; 27 of 142 trials in a review of bacterial vaccines to prevent respiratory disease18; 16 of 34 trials in a review of antibiotics to treat respiratory disease23; and 23 of 44 trials in a review on vaccines to prevent Salmonella24). However, language of publication still may be a barrier. Based on the reasons for full text exclusions from 4 systematic reviews, the number of trials excluded because of the language of publication was 0 of 190 full texts evaluated,17 41 of 536 full texts evaluated,18 8 of 90 full texts examined,23 and 54 of 126 full texts examined.24
Another issue is whether research reports can be found. Notwithstanding the caveats for using conference proceedings for decision-making, they still may provide useful information on what is being researched. To explore the availability of conference proceedings, we used 2 recent systematic reviews which evaluated the efficacy of preventive antibiotics to reduce respiratory disease in swine17 and vaccines targeted to bacterial respiratory pathogens.18 Of the 182 articles included in those reviews, 105 were published in conference proceedings. As the eligibility criteria for these reviews would preferentially include a journal article over a conference proceeding, it is assumed that these represent studies reported only at a conference venue. There were 7 organizations represented: AASV Annual Meeting, Asian Pig Veterinary Society Congress, International Pig Veterinary Society Congress, International Society for Veterinary Epidemiology and Economics, International Symposium on Emerging and Re-emerging Pig Diseases, European Symposium of Porcine Health Management, and World Association of Veterinary Laboratory Diagnosticians and OIE Seminar on Biotechnology. Of these 7 organizations, it was only possible to access the conference proceedings for 4 organizations, with 2 unavailable online and 1 being password protected and available to members only. These results represent conference proceedings availability for only 2 topic areas and may not represent the availability of swine conference proceedings in general. Nonetheless, these results illustrate that some, but not all, conference proceedings can be freely accessed via the internet.
Journal articles are an important source of information for veterinarians and researchers,4 although not all journals are freely accessible. When an article is published in a journal, it is common for the researcher(s) to transfer their copyright to the publisher, who controls further access. There are several access options: publishers may require a subscription to access the journal or a singular article, the journal may be open access and thus full-text articles are freely available to all, or the journal may be a hybrid where researchers can pay a fee to have their article made open access.
A study of veterinary research articles published between 2000 and 2014 found that over half (62%) of the articles were freely available.25 Specifically for swine and based on trials included the systematic reviews as described previously,17,18 we evaluated the accessibility of these published trials. There were 77 articles published in 35 journals included in the 2 reviews. Of the 35 journals, 16 were fully open access, 9 were hybrid, 4 were available to association members, and 6 were no longer active journals. Open access provides a way for potential users of research to have access to the full study results. However, it is not always without cost; the publication fees, as billed to the article authors, for the open access and hybrid journals identified in the two systematic reviews ranged from $0 to $4200 per article, with a median cost of $1935.
There are other ways researchers can make their work freely available. These tend to include preprints (the researcher’s own write-up of results and analysis that has not been peer reviewed, nor had any other “value added” by a publisher) posted on faculty or departmental websites, government websites,25 or profiles on sites such as ImpactStory or ORCiD. Institutional or subject-based digital repositories are of growing importance in the research community, especially as government mandated open access policies such as those put forth by the Tri-Agency (Canada) and UKRI (United Kingdom) are introduced and begin to be implemented. These sites, which tend to be maintained by research centers or academic libraries, provide permanent and stable access to various types of research outputs including articles, theses, dissertations, data, diagrams, posters, and other items.26 Outputs are assigned appropriate metadata (researcher name[s], title, abstract, keywords, and copyright or licensing information) as well as a digital object identifier (DOI) or permalink ensuring perpetual access at the same digital location. Institutional repositories (IRs) tend to be set up in hierarchical structures. For instance, the University of Guelph IR (“the Atrium”; https://atrium.lib.uoguelph.ca/) has collections within existing faculties (eg, Ontario Veterinary College), and departments (eg, Department of Population Medicine), and then within topic areas (eg, theses and dissertations, systematic review protocols, and study protocols for research involving animals). Some repositories also house collections for outputs related to conferences, projects, research units, or researchers.
While there are many advantages to using an institutional or subject-based repository, such as they are free to use, often maintained by staff with preservation expertise, equipped with functionality that reveals basic or advanced usage metrics, and facilitate wider impact, they tend not to have the same popularity as tools such as ResearchGate or Academia.edu.27,28 These sites are social networking tools for academics, with some of the same problematic approaches to user privacy and data monetization as their nonacademic counterparts.29,30 Since much of the perceived value of these sites is discoverability, it has become a focal point for copyright violations, with many researchers uploading the published versions of their research to the site in an infringement of copyright.31,32 To avoid such infringement, researchers should be seeking to self-archive an appropriate version of their published research in a repository. Such action is often permitted by journal publishers, so long as particular conditions are met: usually only preprints (the version of the article submitted to the journal prior to being peer-reviewed) and postprints (the version of the article that has been through peer-review, has been accepted for publication, but lacks “value-added” services of the publisher such as formatting) can be archived, though there may be a particular length of time that must pass before the researcher can do so. Digital repositories can easily accommodate these embargoes, putting in place a “dark deposit,” whereby the full text is not openly available until a predetermined date.33 The metadata associated with the work is still public, allowing the record to remain discoverable both through the repository as well as aggregated search tools such as Digital Commons Network (https://network.bepress.com/) and Google Scholar.
Another interesting situation related to access to research is for emergency situations, where it is imperative that research results be made available quickly for rapid decision-making, even if the results are not final or there is not time to complete a highly polished manuscript as one would expect for peer-review. An example of this was the recent emergence of porcine epidemic diarrhea (PED), where funding organizations, such as the National Pork Board, publicly promoted titles of funded projects to increase awareness of pending research helping to identify remaining knowledge gaps and avoid unnecessary duplication of studies (see https://www.nationalhogfarmer.com/health/pork-board-funds-eight-ped-virus-projects). Their requirement for updates on the results of research they had funded meant they could make these reports freely available to help producers to quickly use knowledge gained to deal with the crisis. Journals can assist in this situation by being flexible with allowing subsequent publication of full research results, even when early results have been made publicly available.
These examples serve to illustrate not only the magnitude of the accessibility issue in swine research, using clinical trials as an example, but also the serious consequences of not making research available. The onus is largely on researchers to ensure that they complete research using animals and submit full reports of that research to journals. Although paying for open access may not be an option for all researchers, there are increasingly other ways that researchers can ensure that knowledge users can access their findings. Researchers who wish their work to be used for clinical decision-making should take advantage of emerging options for wider accessibility of their research results.
Proposed solutions to increase research availability and accessibility
To provide utility to the swine industry, research must be available and accessible. Researchers employed in academia have received advanced training in research methodologies and are incentivized to publish research. However, this may not be the case for those employed in other types of organizations; publication takes time and may therefore be a low priority. One possible solution to increase publication would be to increase collaborative opportunities between academics and others in the design, conduct, and dissemination of research. There is a role for academia in teaching not only graduate students but student veterinarians on the appropriate conduct of research and critical appraisal. Incentives to publish also may come from the consumers of research; as evidence-based medicine continues to evolve, veterinarians and practitioners may expect a higher standard for research availability. Funding agencies could assist by linking funding to publication of results or, if publication is not possible, posting of full methods and results of their research on a publicly accessible site. Organizations involved in research should promote open-access publication and researchers should include possible open-access fees into grant applications. While being aware of copyright obligations, researchers should take advantage of new options for publicly disseminating research articles free of charge. Improving availability and access to research will benefit the entire swine industry and help to maximize the value of the research investment.
Implications
- Accessible swine research results may positively impact the swine industry.
- Results must be available to avoid waste and understand intervention efficacy.
- Opportunities exist to enhance research availability and benefit the swine industry.
Acknowledgments
The authors were responsible for developing the ideas presented in this commentary. Partial funding support was obtained from the University of Guelph Research Leadership Chair (Sargeant).
Conflict of interest
None reported.
Disclaimer
Scientific manuscripts published in the Journal of Swine Health and Production are peer reviewed. However, information on medications, feed, and management techniques may be specific to the research or commercial situation presented in the manuscript. It is the responsibility of the reader to use information responsibly and in accordance with the rules and regulations governing research or the practice of veterinary medicine in their country or region.
References
1. Sargeant JM, Kelton DF, O’Connor AM. Study designs and systematic reviews of interventions: Building evidence across study designs. Zoonoses Public Health. 2014;61(Suppl 1):10-17. https://doi.org/10.1111/zph.12127
2. Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, Krumholz HM, Ghersi D, van der Worp HB. Increasing value and reducing waste: Addressing inaccessible research. Lancet. 2014;383:257-266. https://doi.org/10.1016/S0140-6736(13)62296-5
3. Galsworthy MJ, Hristovski D, Lusa L, Ernst K, Irwin R, Charlesworth K, Wismar M, McKee M. Academic output of 9 years of EU investment into health research. Lancet. 2012;380(9846):971-972. https://doi.org/10.1016/S0140-6736(12)61528-1
4. Huntley SJ, Dean RS, Massey A, Brennan ML. International evidence-based medicine survey of the veterinary profession: Information sources used by veterinarians. PLoS One. 2016;11:e0159732. https://doi.org/10.1371/journal.pone.0159732
5. Brace S, Taylor D, O’Connor AM. The quality of reporting and publication status of vaccines trials presented at veterinary conferences from 1988 to 2003. Vaccine. 2010:2;5306-5314. https://doi.org/10.1016/j.vaccine.2010.05.041
6. Snedeker KG, Totton SC, Sargeant JM. Analysis of trends in the full publication of papers from conference abstracts involving pre-harvest or abattoir-level interventions against foodborne pathogens. Prev Vet Med. 2010;95:1-9. https://doi.org/10.1016/j.prevetmed.2010.02.015
7. Bentley E, Koester M, Bdolah-Abram T, Yair N, Ofri R. Factors affecting publication in peer-reviewed journals of abstracts presented from 2008 to 2012 ACVO meetings. Vet Ophthalmol. 2018;14:147-152. https://doi.org/10.1111/vop.12573
8. Dickersin K, Min YI. Publication bias: The problem that won’t go away. Ann N Y Acad Sci. 1993;703:135-148. https://doi.org/10.1111/j.1749-6632.1993.tb26343.x
9. Meyers KE, Lindem MJ, Giuffrida MA. An observational study of abstracts presented at the American College of Veterinary Surgeon Annual Meetings (2001-2008) and their subsequent full-text publication. Vet Surg. 2016;45:672-678. https://doi.org/10.1111/vsu.12484
10. Snedeker KG, Campbell M, Totton SC, Guthrie A, Sargeant JM. Comparison of outcomes and other variables between conference abstracts and subsequent peer-reviewed papers involving pre-harvest or abattoir-level interventions against foodborne pathogens. Prev Vet Med. 2010;97:67-76. https://doi.org/10.1016/j.prevetmed.2010.07.012
11. Wieser M, Braun C, Moens Y. Publication rate of studies presented at veterinary anaesthesia specialty meetings during the years 2003-2008. Vet Anaesth Analg. 2016;43:118-127. https://doi.org/10.1111/vaa.12295
12. Lamb CR, Adams CA. Acceptance rates for manuscripts submitted to veterinary peer-reviewed journals in 2012. Equine Vet J. 2015;47:736-740. https://doi.org/10.1111/evj.12376
13. Byrne DW. Common reasons for rejecting manuscripts at medical journals: A survey of editors and peer reviewers. Sci Ed. 2000;23:39-44.
14. Song F, Loke Y, Hooper L. Why are medical and health-related studies not being published? A systematic review of reasons given by investigators. PLoS One. 2014;15:9(10):e110418. https://doi.org/10.1371/journal.pone.0110418
15. Scherer RW, Ugarte-Gil C, Schmucker C, Meerpohl JJ. Authors report lack of time as main reason for unpublished research presented at biomedical conferences: A systematic review. J Clin Epidemiol. 2015;68:803-810. https://doi.org/10.1016/j.jclinepi.2015.01.027
16. Chalmers I. Underreporting research is scientific misconduct. JAMA. 2010;263:1405-1408. https://doi.org/10.1001/jama.1990.03440100121018
17. Sargeant JM, Bergevin MD, Churchill K, Dawkins K, Deb B, Dunn J, Hu D, Moody C, O’Connor AM, O’Sullivan TL, Reist M, Wang C, Wilhelm B, Winder CB. A systematic review of the efficacy of antibiotics for the prevention of swine respiratory disease. Anim Health Res Rev. 2019;20:291-304. https://doi.org/10.1017/S1466252319000185
18. Sargeant JM, Deb B, Bergevin MD, Churchill K, Dawkins K, Dunn J, Hu D, Moody C, O’Connor AM, O’Sullivan TL, Reist M, Wang C, Wilhelm B, Winder CB. Efficacy of bacterial vaccines to prevent respiratory disease in swine: A systematic review and network meta-analysis. Anim Health Res Rev. 2019;20:274-290. https://doi.org/10.1017/S1466252319000173
19. Grindlay DJC, Brennan ML, Dean RS. Searching the veterinary literature: A comparison of the coverage of veterinary journals by nine bibliographic databases. J Vet Med Educ. 2012;39:404-412. https://doi.org/10.3138/jvme.1111.109R
20. O’Connor AM, Anderson KM, Goodell CK, Sargeant JM. Conducting systematic reviews of intervention questions I: Writing the review protocol, formulating the question and searching the literature. Zoonoses Public Health. 2014;61(Suppl 1):28-38. https://doi.org/10.1111/zph.12125
21. Murphy SA. Searching for veterinary evidence: Strategies and resources for locating clinical research. Vet Clin North Am Small Anim Pract. 2007;37:433-445. https://doi.org/10.1016/j.cvsm.2007.01.003
22. Keay S, Sargeant JM, O’Connor A, Friendship R, O’Sullivan T, Poljak Z. Veterinarian barriers to knowledge translation (KT) within the context of swine infectious disease research: An international survey of swine veterinarians. BMC Vet Res. 2020;16:416. https://doi.org/10.1186/s12917-020-02617-8
23. O’Connor AM, Totton SC, Shane D. A systematic review and network meta-analysis of injectable antibiotic treatment options for naturally occurring swine respiratory disease. J Swine Health Prod. 2019;27:133-149.
24. de la Cruz ML, Conrado I, Nault A, Perez A, Dominguez L, Alvarez J. Vaccination as a control strategy against Salmonella infection in pigs: A systematic review and meta-analysis of the literature. Res Vet Sci. 2017;114:86-94. https://doi.org/10.1016/j.rvsc.2017.03.005
25. Page JR. Accessibility of published research to practicing veterinarians. J Med Libr Assoc. 2018;106:330-339. https://doi.org/10.5195/jmla.2018.196
*26. The Canadian Association of Research Libraries (CARL) and the Scholarly Publishing and Academic Resources Coalition (SPARC). Greater reach for your research. 2008. Accessed December 20, 2021. https://www.carl-abrc.ca/doc/sparc_repositories.pdf
27. Laakso M, Lindman J, Shen C, Nyman L, Björk B-C. Research output availability on academic social networks: Implications for stakeholders in academic publishing. Electronic Markets. 2017;27(2):125-133. https://doi.org/10.1007/s12525-016-0242-1
28. Borrego A. Institutional repositories versus ResearchGate: The depositing habits of Spanish researchers. Learn Publ. 2017;30:185-192. https://doi.org/10.1002/leap.1099
*29. Fortney K, Gonder J. A social networking site is not an open access repository. Office of Scholarly Communication, University of California. December 1, 2015. Accessed October 20, 2021. https://osc.universityofcalifornia.edu/2015/12/a-social-networking-site-is-not-an-open-access-repository/
*30. Fitzpatrick K. Academia, not edu. Kathleen Fitzpatrick. October 26, 2015. Accessed October 20, 2021. https://kfitz.info/academia-not-edu
*31. Van Noorden R. Publishers threaten to remove millions of papers from ResearchGate. Nature. 2017. https://doi.org/10.1038/nature.2017.22793
*32. McKenzie L. Publishers escalate legal battle against ResearchGate. Inside Higher Ed. 2018. Accessed December 20, 2021. https://www.insidehighered.com/news/2018/10/04/publishers-accuse-researchgate-mass-copyright-infringement
*33. European University Association. EUA’s open access checklist for universities: A practical guide on implementation. 2015. Accessed December 20, 2021. https://eua.eu/downloads/publications/euas%20open%20access%20checklist%20for%20universities%20a%20practical%20guide%20on%20implementation.pdf
*Non-refereed references.
PDF version
RIS
citation