Reference Rot in Medical Publications

 C. O’Connor 1, A. O’Connor 2

  1. RCSI Library, Royal College of Surgeons in Ireland, Dublin, Ireland
  2. University of Adelaide, Director of Emergency Medicine, Riverland General Hospital, South Australia

 Dear Sir,

The sourcing of the original references of scholarly articles is an integral part of academic endeavour. A contemporary phenomenon  that has arisen as a result of  the increased use of online resourcing, and the referencing of same, is the issue known as  “reference rot“. Reference rot is where the article or webpage resource identified by a URL no longer exists or has moved to another site. As a result, when searching for the original paper or source, one is met with the ubiquitous “404 not found” error message. Reference Rot also refers to the case whereby the resource identified has changed over time and evolved into a resource that bears no resemblance to the content originally referenced.

As a result, we postulate that URLs are not a robust method of referencing, and that this can be demonstrated using Ireland’s leading medical journal as an example. Such a hypothesis has been upheld in several studies of other medical journals of various specialties. The problem of reference rot is a serious one. Academic scholarship relies on references to support the claims made by authors; reference rot makes academic papers vulnerable to references which no longer support these claims. For this study all URL references  within ‘Original Papers’ in the Irish Medical Journal between 2013 and 2017 were checked manually by the authors for reference rot.

In total there were 50 issues and 296 Original Papers in the editions of the Irish Medical Journal published between 2013 and 2017 with a total of 5078 references. Of these, 487 (9.6%) were URL references. Overall 165 URL references (34.0%) suffered from reference rot.  3.2% of total references (URL + non-URL references) between 2013 and 2017 were inaccessible due to reference rot. Our findings demonstrate that reference rot is a significant problem in the Irish Medical Journal and is on a par with that in other medical journals. An analysis of URL links in the journal Emergency Medicine Australasia found that 34.1% of URL references between 2010 and 2014 were no longer accessible.1 A 2013 Study which looked at URL references in the Lancet and the BMJ between January 2006 and June 2013 found that 39% of URL references from that period were inaccessible by mid-2013.2

Some solutions to the problem of reference rot have been suggested. Burnhill et al suggests journals incorporate an archival step in their submission process, thus ensuring a record is kept of references at the time of submission and so ensuring there is ‘appropriate and sufficient evidence for what is stated in scholarly statement’.3 . Archiving in WebCite, for example, allows an author to use a reference which contains a link to an archived copy of the material, exactly as the citing author saw it when he or she accessed the cited material.4  In addition author’s should try and use persistent identifiers, such as DOIs where possible and should also avoid linking directly to documents such as PDFs as these tend to be less stable than webpages. 

Correspondence:
Colm O’Connor, MA, MLIS,
Medical Librarian,
RCSI Library,
Royal College of Surgeons in Ireland,
Dublin,
Ireland.
Email: colmoconnor@rcsi.com

References

1. O’Connor C & O’Connor AE. 'Reference rot': A developing problem in Emergency Medicine Australasia. Emergency Medicine Australasia. 2015;27(5):494
2. Habibzadeh, P. (2013). "Decay of references to Web sites in articles published in general medical journals: mainstream vs small journals." Applied Clinical Informatics.2013;4(4):455-464.
3. Burnhill, P, Mewissen, M and Wincewicz, R, Reference rot in scholarly statement: threat and remedy, Insights, 2015, 28(2):55–61; DOI: http://dx.doi.org/10.1629/uksg.237
4. WebCite Consortium. WebCite Consortium FAQ. 2007. Available at: https://web.archive.org/web/20180211164104/http://www.webcitation.org/faq Accessed 9th June 2018

P827