1 Post publication peer review in Iranian biomedical journals

Introduction

Letters to editor or the correspondence columns are one of the most popular sections of some scientific journals [1-2]. Letters to editor are written for different purposes. Some journals may use them for publishing concise articles, preliminary results of studies or brief reports, for instance reports of new cases [1-3]. In many journals, however, they provide an opportunity for readers to state their views about published articles [3]. This use of letters to editor is frequently regarded as a form of ‘post-publication peer review’.Many readers may suppose that papers published in peer-reviewed journals are valid and trustworthy [4], however; one may find errors in research methodology, statistical analyses and generalization of results in articles [5]. In fact, even in the best journals, peer review system may not be able to prevent thoroughly the publication of flawed studies [6-10]. Altman, [4] while summarizing some evidence on the prevalence of methodological problems in clinical trials, concludes that “poor methodology and reporting are widespread”. In such cases, readers can assist in post-publication peer review by writing a letter to editor to correct possible errors in articles [5, 11]. Additionally, one may write a letter to share with other readers his or her viewpoints to help clarify scientific evidence recorded in the related literature [12]. Consequently, letters can preserve the trustworthiness of the literature and make journals accountable to the scientific community [2, 12]. As Brown [2] states “through the letters, scientific articles published in a peer reviewed journal are subject to continuing scrutiny”. Research on the issue of post-publication peer review is not abundant. A review of medical literature revealed only 7 research studies in this regard. The purpose for performing these studies has been varied. One subject of interest has been the number of letters written in relation to published articles. Von Elm et al. [13] investigated letters published in ten leading medical journals and found a median number of one letter per citable article. Other researchers were interested in the percentage of articles that attracted correspondence. Caswell [14] reported that letters were written in correspondence to only 20% of papers published in the Medical Journal of Australia. Another study showed that letters raised “substantive criticism” against only one third of research papers published in BMJ [6]. Boyton and Arnold, [15] in an audit of the BMJ’s correspondence columns, found that same number of letters agreed and disagreed with original articles. Three studies aimed to explore what percentage of letters were replied by authors. All studies showed that authors had replied to nearly half of the letters [6,13-14]. In a bibliometric study, Mahesh et al, [16] showed that 4.1% (8/196) of the correspondence letters published in Dutch Journal of Medicine discovered mistakes in original articles and in six of these cases, the journal published a ‘correction’ in response to criticisms made in letters. Mayberry [17] reviewed letters published in the Postgraduate Medical Journal during an 18 month period and reported that 30% of authors had published five or more letters, and 13% more than 10 letters. Mayberry [17], while acknowledging the significance of letters for scientific research, emphasizes that they should not be used as a method for making the list of publications larger. Review of literature shows that all research studies on the issue of post-publication peer review have been on American, European and Australian journals. Despite large number of the medical journals published in other parts of the world, we found no study on those journals. In Iran, history of medical journalism dates back to the beginning of twentieth century, [18] however, the issue of post-publication peer review in these journals has not been studied yet. The present research aimed to study this subject in Iranian biomedical journals and investigated the number of letters written in relation to published articles, percentage of articles that attracted correspondence, percentage of letters replied by authors and percentage of correspondence letters discovered mistakes in original articles.

Materials and Methods

This was a bibliometric study on Iranian biomedical journals. We performed a preliminary review on several Iranian journals and found that the number of published letters was very limited. As a result, researchers conducted this study on all Iranian biomedical journals published in Persian or English languages. The databases of Iran Medex (Indexing Articles Published in Iran Biomedical Journals), Medlib (Global Medical Articles Library), and SID (Scientific Information Database) were searched to find letters published in Iranian biomedical journals. These databases allow users to search articles in both Persian and English languages. Medlib and Iran medex index only articles published in biomedical journals, whilst SID files scientific articles in all subjects. We were not able to impose any time limit for inclusion of letters in the study, because databases did not provide such facility to their users. As a result, all the letters found in these databases, regardless of the year of publication, were included in the study. This study was performed at the end of 2012. The terms “letter to the editor”, “letter to editor”, “letter” and their equivalent phrases in Persian were used as keywords for searching databases. We included only those letters in the study that were written in correspondence to previously published articles. Letters in other formats such as commentaries, concise articles, and preliminary results of studies or brief reports were excluded from the study.

Results

At the time of conducting this study (January 2013), Iran Medex indexed 84957 articles, Medlib 161765 articles and SID 170231 articles in Persian and 267363 English articles. The difference in number of articles was attributed to the domain of these databases as described above. A total number of 184 letters were found in Iranian journals that were written in correspondence to previously published articles. 49 ofletters were responses of authors to the criticism made by readers; as a result, the number of letters that commented on or criticized the original articles reduced to 135, among them, 113 were in English and 22 in Persian.Since this study was conducted on biomedical journals, we calculated the ratio of letters tooriginal articles based on the number of indexed articles in Medlib. We were not able to compute the same ratio for SID due to its general domain. On the other hand, Iran Medex indexed a smaller number of articles compared with Medlib, and therefore Medlib was selected to determine the ratio of the letters to original articles. This ratio was calculated to be 1:1198 (135 out of 161765). This figure shows that for approximately every 1198 articles published in Iranian biomedical journals, there is only one letter written in correspondence to previously published articles. In other words, letters consist approximately 0.08% of the articles published in Iranian biomedical journals. 71(53%) letters found mistakes or criticized specific parts of the articles, including design and methods of studies (40 letters), statistical analyses (8 letters), results (13 letters) and conclusions or the generalization of findings (36 letters). 49 (36%) authors responded to the criticism raised in letters. One author [19] who is non-Iranian residing outside Iran has published 19 letters in Iranian biomedical journals. There were 4 authors with 3 letters and 6 authors with 2 letters. The rest of authors (92 people or 89% of authors) published only one letter. The journal with the largest number of letters was Hepatitis Monthly which published 21% of the letters in Iranian biomedical journals (Table 1). The same journal published two original articles which has attracted the largest number of letters. These articles were written by Leonardi and La Rosa [20] (with 10 letters), and Yilmaz et al [21] (with 7 letters). Letters found in this study were published during 2001 and 2012. The databases did not reveal letters before 2001. The distribution of letters (Figure 1) shows that the largest number of letters were published in 2010 (29 letters) and 2011 (58 letters).

Discussion

The present study found that letters consist 0.08% of the articles published in Iranian biomedical journals. This result is too far removed from the findings by Von Elm et al, [13] who reported a median number of one-letter-per-article and Caswell [14] who obtained a percentage of 20% for the articles with correspondence. Another study [6] showed “substantive criticism” against one third of research papers. The low number of letters to editor in Iranian biomedical journals compared to other (western) journals could be attributed to the following possible issues:

1- Post-publication peer review seems to be a relatively neglected and abandoned issue in Iranian medical journalism. Apparently, many Iranian readers, editors and publishers are not aware of the importance and critical role of the letters to editor in the dissemination of scientific evidence. This broad unawareness is probably a significant obstacle for writing letters to editor.

2- Scientific community may read and review articles in Iranian journals insufficiently. At the present time, there is no information as to what extent Iranians read biomedical journals. However, a recent study on seven US and Australian universities showed that faculty members in health and medicine read on average 34.5 articles per month [22].The same study revealed a significant association between publishing productivity of faculty members and the number of article reading per month. Based on this evidence, it seems logical to propose a direct relationship between the amount of article reading by journal readers and the number of written letters in scientific journals. If we accept this theory, we could assume that one reason for the low number of letters to editor in Iranian biomedical journals is the low rate of article reading among Iranian readers. Our reasoning here is mainly based on article reading by academics; other journal readers such as postgraduate students and clinicians should also be included in formulating such a theory. Authors of this article suggest further studies to be carried out on all groups of journal readers in order to uncover potential patterns of article reading among Iranian readers.

3- Iranian readers may not read articles thoroughly and critically. Some readers may not read full text of an article, and rely mostly on abstracts or final conclusions of the papers to obtain the information they need [23-24]. Others may read full text but do not have a critical vision [25]. They readily accept an article without appraising the content or assessing its quality. As a result, for this group of readers there is no motive to write letters to the editor. Perhaps, one major reason for this unsophisticated view about journal articles is the lack of knowledge and skill for critical appraisal of articles. The topic of critical appraisal is routinely taught to undergraduate and postgraduate students and also to health care professionals in many countries [26-28]. However, in Iran, it has not been properly embedded in the curriculum of health and medical sciences yet. Thus, it could be concluded that the problem of low number of letter writing in Iranian biomedical journals could be attributed partly to the lack of critical appraisal skills in some Iranian readers. There is evidence that teaching critical appraisal and writing skills to medical undergraduates could be effective in the production of letters to editor [28].

4- Some readers may not be confident in criticizing others’ works. Johnson [12] believes that these readers might be initially motivated to write a letter to editor, but when the passion for writing a letter is converted to pragmatics of writing, fears set in which hold back them from exposing their views. Because of this problem, Johnson [12] offers a brief guide for writing letters to editor, to help readers feel more comfortable for letter writing. The problem is probably worse in Iran, because, in addition to the lack of critical appraisal teaching in its educational system, criticism and debate trend has not been fully developed in many Iranians, even among high-educated individuals [29].

5- There is little incentive or academic reward for writing letters to editor because in the assessment of academic performance, weight and reputation of a letter is usually less than an original research article [13, 30-31]. In Stang’s [30] opinion, letter to editors should get the same weight as original articles. Concerning Iranian universities, the current regulations for academic promotion [32] seems to be rather unfair in this regard, because while an original article can achieve a score between 2 to 7, a letter to editor may get 1 to 2 scores; moreover, letters to editor can achieve maximally 2 scores, whereas, there is no such a limitation for original articles. Therefore, there seems to be little reward for academics to spend time and effort on this important task [13].

6- The International Committee of Medical Journal Editors (ICMJE) has recommended that “Biomedical journals should provide the readership with a mechanism for submitting comments, questions, or criticisms about published articles…” [33]. This is usually in the form of a correspondence column or section [33]. Although it was not the aim of the present study, but during the process of data collection, authors found that many of Iranian biomedical journals do not have a correspondence section. “Letter to the editor” was not usually among the types of articles acceptable for these journals. As a result, readers of such journals were deprived of their rights to send their comments about published papers to the journals. This problem could negatively affect the rate of letter writing in Iranian biomedical journals.

In the present study, 36% of letters were replied by authors of original articles. This is comparable to the findings of previous studies; all three studies [6, 13-14] showed that authors had replied to nearly half of the letters. Our result indicates that although critics rarely write letters to editor in Iranian journals, whenever they do so, their criticisms are replied by more than one third of the authors of original papers. Apparently, Iranian authors similar to their peers in other places are sensitive enough to reply to criticisms raised in letters.

This study found a percentage of 53% for the letters discovered mistakes in original articles. This result is comparable to the findings of Mahesh study [16] who reported a percentage of 4.1% in this regard. As Gøtzsche [6] states, pre-publication peer review system may not work very well, and thus, it is readers responsibility to assist in post-publication peer review and to correct possible errors in articles [5, 11]. Then, it may be speculated that the large difference between results of this and Mahesh study may be indicative of a relatively less robust peer review system in Iranian biomedical journals.The present study showed that only 11 (11%) authors wrote more than one letter to the editor. In the study by Mayberry, 30% of authors had published five or more letters [17]. Perhaps, this difference could be explained by a lack of expert or professional letter writers among Iranian readers. 76% of letters in Iranian biomedical journals published from 2010 to 2012. This might be due to a boosted motivation for Iranian readers to write more letters to editor, and/or the remarkable growth in scientific production of Iranians in recent years [34]. It would be expectable to find more letters to editor as more articles will be published in Iranian journals.This study has implications for readers, journals and policy makers. Post-publication peer review is regarded as the continuum of peer review process [11]. It is an opportunity for readers to state their views about articles published in a Journal [3]. Iranian readers are encouraged to take advantage of this opportunity and send their comments and criticisms on articles to journals. To this end, they need to read articles more frequently and become regular article readers. Also, they should read articles thoroughly and with a good critical vision; developing critical appraisal skills is an important prerequisite in this regard. Iranian readers, while trying to avoid harsh criticism, should be more confident in writing letters to editors. The study has implications for the Iranian medical educational systems including Ministry of Health and Medical Education, and medical universities. They need to take steps to train systematically academics, university students, clinicians and allied health professions in critical appraisal and writing skills. Medical universities should be aware of the critical role of post-publication peer review in improving the quality of papers and give more weight to letters in assessing research activities of their academics.

Finally, Iranian journals should reconsider their policies on the acceptance of letters to editors. They are advised to take into account ICMJE’s recommendation [33] to include a correspondence section in their journals. Additionally, they need to make letters more visible to their readers and indexing databases. The present study has some limitations. We used three databases of Iran Medex, Medlib, and SID. It was possible that these databases have not indexed some letters published in Iranian biomedical journals and thus, we have not included them in our study.

One explanation for this problem could be the weak visibility of letters in some Iranian journals. In such cases, only original articles may be revealed to databases. We based calculation of the ratio of letters to original articles on Medlib, while, the number of articles indexed in Medlib and Iran Medex was different. This discrepancy needs to be considered regarding study findings. Another limitation of the study was that databases did not show published letters before 2001. As a result, study was limited only to letters published after this year. Some reasoning and interpretations presented in this paper were based on authors’ speculations and not on hard scientific evidence. The author strongly acknowledges the need for further research in order to test hypotheses raised in this article regarding possible reasons for the relatively low number of letters in Iranian biomedical journals.

Conclusion

Iranian readers are much less productive in writing letters to Iranian biomedical journals compared to their peers in other places. Low number of letters could be attributed to several issues including a broad unawareness of the importance of letters, low rate of article reading, lack of confidence and/or skills of critical appraisal and little incentive for writing letter. This study found a relatively high rate of mistakes in original articles discovered by readerssuggesting the necessity for a more robust peer review system in Iranian biomedical journals. The sharp rise in the number of letters after 2009 indicates a boosted motivation of Iranians for writing more letters to editors.

Conflict of Interest

None

1. Squires BP. Letters to the editor: What editors expect from authors. CMAJ: Canadian Medical Association Journal. 1990;142(7):713.
2. Brown C. Unvarnished viewpoints and scientific scrutiny: Letters to the editor provide a forum for readers and help make a journal accountable to the medical community. Canadian Medical Association Journal. 1997;157(6):792-4.
3. Curfman GD, Graham A, Lindenfelser L, Anderson KR, Drazen JM. Innovations in correspondence. New England Journal of Medicine. 2003;348(4):344.
4. Altman DG. Poor-quality medical research. JAMA: the journal of the American Medical Association. 2002;287(21):2765-7.
5. Liesegang TJ. Peer review should continue after publication. American journal of ophthalmology. 2010;149(3):359-60.
6. Gøtzsche PC, Delamothe T, Godlee F, Lundh A. Adequacy of authors’ replies to criticism raised in electronic letters to the editor: cohort study. BMJ: British Medical Journal. 2010;341:c3926.
7. Stroebe W, Postmes T, Spears R. Scientific misconduct and the myth of self-correction in science. Perspectives on Psychological Science. 2012;7(6):670-88.
8. Jenal S, Vituri DW, Ezaías GM, Silva LAd, Caliri MHL. The peer review process: an integrative review of the literature. Acta Paulista de Enfermagem. 2012;25(5):802-8.
9. Mulligan A, Hall L, Raphael E. Peer review in a changing world: An international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology. 2013;64(1):132-61.
10. Lee CJ, Sugimoto CR, Zhang G, Cronin B. Bias in peer review. Journal of the American Society for Information Science and Technology. 2013;64(1):2-17.
11. Horton R. Postpublication criticism and the shaping of clinical knowledge. JAMA: the journal of the American Medical Association. 2002;287(21):2843-7.
12. Johnson C, Green B. How to write a letter to the editor: an author’s guide. Journal of Chiropractic Medicine. 2006;5(4):144-7.
13. Von Elm E, Wandel S, Jüni P. The role of correspondence sections in post-publication peer review: A bibliometric study of general and internal medicine journals. Scientometrics. 2009;81(3):747-55.
14. Caswell A. Letters to the editor 1991. An audit of the MJA’s correspondence columns. The Medical journal of Australia. 1992;157(1):63.
15. Boyton RJ, Arnold PC. An audit of the BMJ’s correspondence columns. British Medical Journal. 1990;301(6766):1419-20.
16. Mahesh S, Kabos M, Walvoort H, Overbeke A. Significance of letters published in the Dutch Journal of Medicine, 1997/98. Nederlands tijdschrift voor geneeskunde. 2001;145(11):531-5. [English abstract]
17. Mayberry J. I read with interest…Postgraduate medical journal. 2004;80(948):559.
18. Azizi MH, Raees-Jalali GA, Noroozi H. A Brief History of the Publication of Biomedical Journals in Iran between 1901 and 1979. Archives of Iranian Medicine. 2009;12(2):204.
19. BiomedExperts. Elsevier Inc; [2/20/2013]; Available from: http://www.biomedexperts.com/Profile.bme/1547628/Viroj_Wiwanitkit.
20. Leonardi S, La Rosa M. Are hepatitis B virus and celiac disease linked? Hepatitis Monthly. 2010;10(3):173.
21. Yilmaz Y, Yonal O, Kurt R, Bayrak M, Aktas B, Ozdogan O. Noninvasive assessment of liver fibrosis with the aspartate transaminase to platelet ratio index (APRI): Usefulness in patients with chronic liver disease: APRI in chronic liver disease. Hepatitis Monthly. 2011;11(2):103.
22. Tenopir C, King DW, Spencer J, Wu L. Variations in article seeking and reading patterns of academics: What makes a difference? Library & Information Science Research. 2009;31(3):139-48.
23. Saint S, Christakis DA, Saha S, Elmore JG, Welsh DE, Baker P, et al. Journal reading habits of internists. Journal of general internal medicine. 2000;15(12):881-4.
24. Florian RV. Aggregating post-publication peer reviews and ratings. Frontiers in Computational Neuroscience. 2012;6:31.
25. Haynes RB, Mulrow CD, Huth EJ, Altman DG, Gardner MJ. More informative abstracts revisited. Ann Intern Med. 1990;113(1):69-76.
26. Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ: British Medical Journal. 2004;329(7473):1017.
27. Norman GR, Shannon SI. Effectiveness of instruction in critical appraisal (evidence-based medicine) skills: a critical appraisal. Canadian Medical Association Journal. 1998;158(2):177-81.
28. Edwards R, White M, Gray J, Fischbacher C. Use of a journal club and letter-writing exercise to teach critical appraisal to medical undergraduates. Medical education. 2001;35(7):691-4.
29. Malekian M. Hadise Arezoomandi. 1st ed. Tehran: Negah Moaser; 2011. [in Persian]
30. Stang A, Poole C, Schmidt-Pokrzywniak A. Pre-peer review, peer review, and post-peer review: three areas with potential for improvement. Journal of clinical epidemiology. 2008;61(4):309.
31. Bhopal RS, Tonks A. The role of letters in reviewing research. BMJ. 1994;308(6944):1582-3.
32. The new regulations for academics promotion. Iran’s ministry of health and medical education [cited 2013 2/20/2013]; Available from: http://aac.behdasht.gov.ir/index.aspx?siteid=179&pageid=10063.[in Persian]
33. Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Publishing and Editorial Issues Related to Publication in Biomedical Journals: Correspondence. The International Committee of Medical Journal Editors 2013 [cited 2013 2/20/2013]; Available from: http://www.icmje.org/publishing_5correspond.html.
34. Aminpour F. The influence of Iranian scientific journals in disseminating medical information. Journal of Research in Medical Sciences. 2012;17(2): 171-175.

Journal name	The number of published letters
Hepatitis Monthly	28
Journal of Research in Medical Sciences	10
Iranian Journal of Medical Sciences	8
Archives of Iranian Medicine	8
Anesthesiology and Pain Medicine	7
Iranian Journal of Medical Hypotheses and Ideas	6
Iranian Journal of Kidney Diseases	5
Iranian Red Crescent Medical Journal	4
Iranian Journal of Radiology	4
Urology Journal	3