Economía y salud
BOLETÍN INFORMATIVO - Año 2013. Diciembre nº 78
En este número Temas

Con acuse de recibo
Buzón de sugerencias

Peer-reviewing pay-to-publish open-access peer review

Cashing in on the informal debate that arose among some colleagues at the Spanish Health Economics Association regarding the recently published article on open access journals by John Bohannon Who´s afraid of peer-review. Science 2013; 342:60-65, we decided to kill two birds with one stone.

First, we tried to elicit opinions among editors of some prestigious journals in connection with the results reported by Bohannon, his approach, flaws, and, most important, the foreseen solutions to the problems faced by the scientific community when it comes to pay-to-publish in open access journals. Second, to continue publishing papers in English in the Boletín Economía y Salud, in an attempt to reach international English-speaking readers. Time has come to expand both the reach of our Boletín and to broaden the thematic range encompassed by the journal, especially with matters such as this one that pervade research, regardless of the realm of the scientific activity.  

We invited fourteen editors of both national and foreign medical, economics and mixed journals in accord with their thematic range to comment on Bohannon’s paper. We received eight contributions that are compiled below.

Ideally, this compilation should be only a first step. A further goal we pursue is to spur the debate and the participation of readers on this and other coming issues by using the Boletín as a forum. We encourage readers to write their comments and to suggest other matters for debate at the end of this contribution.

Carlos Campillo-Artero



We need both peer review and editorial boards that are independent of publishers. The system of peer review is designed to help guarantee the academic quality of published research. But this system has been weakened by recent changes in the way that journals raise revenue.

In the past, it was in the financial interest of journal publishers to ensure that articles met high standards of academic quality. Publishers used to make their money from library (and, to a lesser extent, individual) subscriptions. In deciding how to allocate their subscription budgets, libraries would prioritise higher quality journals, all else equal. Journals with high thresholds for acceptance, including stringent peer review, were more likely to be stocked and, hence, to make a profit for the publisher. The positive association between quality and revenue meant it was in the publisher’s interest to ensure that editorial processes involved robust peer review.

Subscriptions remain an important, but no longer exclusive, source of publisher income. Now authors can pay the journal directly to publish their article. Ostensibly, the payment covers the costs of expanding potential readership from library users to anyone with internet access. But it also means that: (i) a journal’s revenue is positively associated with the number of articles accepted; and (ii) the relationship between quality and revenue has weakened: journals can increase revenue by lowering thresholds for acceptance, perhaps by having less stringent peer review.

By allowing authors to pay journals to publish their work, quality can be compromised, as the Science experiment demonstrates. The editorial board should ensure that this does not happen but, for many of the journals that were part of the Science sting, the “editorial board” was not independent but an integral part of the publisher. Quality can only be safeguarded by ensuring that those making the decision about whether or not to accept an article have no financial stake in the outcome of the review process. Without independent editorial boards, peer review alone cannot guarantee the quality of published academic research.

Andrew Street, Editor, Journal of Health Economics


Bohannon’s article represents a strong questioning to the peer review process in open access journals as a method of quality control of scientific research and, in addition, it casts doubts on the validity of the results of investigations.

The study focuses on the peer review process but, regardless of what is declared by the journals in their web sites, many did not effectively send the article to review. Even when a review is performed, the reviewers are a heterogeneous group of professionals with regard to their competences, their analysis skills and the way of presenting their comments. The next publishing step, the evaluation of the comments by a competent editor or editorial committee, also showed deficiencies. The study focused on open access journals, but the deficiencies found were related to the review process, which does not bear a relation to the form of dissemination. It is worth asking what would had happened had the article been submitted to paid subscription journals. The growing number of journals that work with limited transparency, the convergence of many of them under the same publishing house, and the profusion of bank accounts in certain countries suggest that many of these journals have been created for the single motive of profit and that are closer to speculation than to honest business, an unfair practice present in many fields. It is noteworthy that this issue capitalizes the need of the investigators to publish, often pressured by academic systems that to a large extent base research grants and professional progress on published work.

Scientific communication should be of the highest quality, but even when the technical reviews were satisfactory, a dose of good faith (of investigators, journals and companies) will always be needed, and the scientific quality of the evidence will be submitted to the most exhaustive test: its validity over time in the light of new research.

Damián Vázquez, Editor in Chief, Pan American Journal of Public Health/Revista Panamericana de Salud Pública


The landscape described in Science is worrying but not unexpected. One might wonder who is afraid of the effect of the invisible hand of the market on science. The peer review process is at the core of the functioning of the science process and is mainly fuelled by the selflessness of scientists around the world, who make an incalculable contribution. Yet, recognition and credit have not been always the rule. In particular, we need to identify and recognize high quality reviewers and reward them. There should be a better balance between the huge profits of well-known scientific publishers and the resources devoted to the review system.

On the other hand, we need to consider the causes of the present flood of scientific journals and the threats to the quality and integrity of science. Perhaps we have turned a blind eye to several phenomena in science that are behind the disturbing facts described by Bohannon. To mention a few, we can cite the academic system of merit assessment, the progressive substitution of quality by speed and quantity, the lack of societal impact assessment of science results, the use of science as a marketing tool or the commercialization of scientific work. We have witnessed how the scientific agenda is distorted by the intervention of vested interests and how there is a growing proliferation of tactics to undermine sound science. All threats to science are intertwined and I believe that the marked monetization of science and the greed of some scientific ventures are endangering many of the altruist features that make up the basis of scientific enterprise.

The scientific community should raise its awareness of the existence of these threats and design integral and institutional responses. It is unlikely that a multilateral institution would undertake the mission of protecting world science and scientists. However, relevant academic and scientific centres should feel morally bound to make the necessary arrangements to create a network of institutions committed to monitoring, analysing and putting forward proposals in order to tackle the alarming facts uncovered by the Science article.

Ildefonso Hernández Aguado, Former Deputy Editor, Journal of Epidemiology and Community Health


Academic communities are too clubby, and often lack the resources to undertake a careful peer review process; the latter often is done by unpaid former authors of the journal with little incentives besides collegiality.

My main comment is that referees should be chosen more carefully by 'motivated editors' and listed in the paper if published, so that in the event of fraud, they are made responsible too.

I would set up a system of paid professional journal editors that are actively seeking contributions between published working papers with a much more active role. We should reduce journals that matter into a few that only publish very substantial work, and promote the development of working papers and blogs for a quick spread of ideas.

Joan Costa-i-Font, Editor, Applied Economic Perspectives and Policy


Delay in publication is a major concern for many researchers. In an attempt to shorten time to publication, many journals now offer “publication ahead of print”, where selected articles are made available on the journal website. This effectively side-steps the queue of accepted articles awaiting publication in the traditional paper format. However, the often lengthy duration of the peer review process remains unaffected.

Nature abhors a vacuum. “Open access” journals have emerged as a mechanism for short-circuiting the arduous process of peer review. Virtually all of these are only available as electronic publications and distributed via the internet. With no barriers to entry, these journals have proliferated and now number in the many thousands.

As reported in Science, the experiment offers a rather dismal picture of the world of open access journals. In only a few cases did the bogus paper undergo peer review by experts in the field and subsequent rejection. In most cases, the main criterion for acceptance was the willingness of the submitting author to pay a fee to cover “costs of publication”. Perhaps most disturbing was the finding that many of the journals that accepted the manuscript were owned by leading academic publishers such as Elsevier, Walters Kluwer and Sage.

What lessons can we take from this experiment in “open access” publication? First, it demonstrates that peer review continues to play a role in establishing at least the face validity of research that is communicated to a global audience. Second, results indicate that there are “legitimate” on-line journals that appear to conduct a competent process of peer review, although they are relatively few in number. The question is how to distinguish “good” from “bad” journals; sponsorship by a well-known academic publisher is not in itself a sufficient guarantee of quality. Third, the experiment sounds a note of caution for those who are invited to serve on the editorial boards or as peer reviewers for new open access journals. Careful investigation is in order before accepting such an invitation lest they may unwittingly lend their name to a sham enterprise. Finally, for those considering submitting their work to an open access journal, similar due diligence is recommended. Publishing good work in a bad journal does little to further the author’s reputation while it helps the journal establish its legitimacy.

Greg de Lissovoy, Editor, Medical Care


"It's the peer review, stupid"

The title of Bohannon’s article published in Science about publishing malpractices is “Who's Afraid of Peer Review?”, but in the standfirst (UK)/subhead (US) I read: “A spoof paper concocted by Science reveals little or no scrutiny at many open access journals”. Most of the criticism fuelled by the paper points to the open access journals industry but certainly, this means missing the shot.

The peer review process, as we know it, is deeply flawed and requires strong amendments. But it is easier to blame the (new) pirates of publishing before assessing the reality behind the established industry, holder of a “Corsican patent”. Pay attention: “Acceptance was the norm, not the exception. The paper was accepted by journals hosted by industry titans Elsevier, Wolters Kluwer, and Sage. The paper was accepted by journals published by prestigious academic institutions such as Kobe University in Japan. It was accepted by scholarly society journals”.

Current practices of voluntary peer review give substantial power to reviewers with little accountability other than self-assessment. Editors may choose to ignore poor quality reviews and/or decline to solicit future reviews from those who produce obviously poor or biased reviews. However, editors may not have the expertise to judge the quality of a a given review (Wendler and Miller, 2013).

As usual, “more research must be carried out to establish the efficacy of the different…” styles of peer review. It would be presumptuous to draw conclusions until further research is done. But if more quality is needed, the immediate tools are transparency and accountability. A bad review making way to an infamous paper or rejecting from biased basis some estimable one, must have consequences. 

To identify these practices, presumably not so unusual, a transparent review process is needed, and some kind of formal sanctions should be established as well.

Ricard Meneu, Editor, Gestión Clínica y Sanitaria


Manufacturing doubts about Open Access

"Doubt is our product", a cigarette executive once observed, "since it is the best means…of establishing a controversy". David Michaels’s excellent “Doubt Is Their Product” provides numerous examples about uses of pseudoscientific arguments to mislead the public manufacturing and magnifying scientific uncertainty. In an age of unprecedented disinformation and misinformation by those whose interests are in risk, an attack to open-access model (OAM) is not a surprise.

In "Who's Afraid of Peer Review?", John Bohannon and Science were mostly interested in highlighting the dark side of OAM. Curiously, however, a paper about peer review was not peer reviewed. It permitted that only journals with article processing charges were considered (which represent only 25% of titles of this kind), no random selection was used and no control was implemented (for instance, submitting the same paper to subscription-based publishers). It appears that the creators of this example of sensationalist journalism were afraid to submit this paper to journals that do not accept that a publicly-funded research reports should be accessible to everybody. 

The implication of Bohannon's article was that, if publishers have a pecuniary incentive to accept a paper, since they only get paid in such a case, then they will inescapably admit garbage in their journals. However, no conclusions can be drawn about the degree in which the bogus effect is specific to OAM journals or about the overall prevalence of the phenomenon in this realm. This flawed study shows only three well known things: that there are predatory publishers, that some journals indexed as a part of OAM are in this category, and that peer review is easier said than done. For anything else, it is scientifically unsound.

What it has been done here is to fabricate another version of the tricks many manufacturers of the doubt carry out daily, in the same way denounced by Michaels.

Luis Carlos Silva Ayçaguer, Editor, Cuban Journal of Information in Medical Sciences


In his recently published experiment, Bohannon targets peer-reviewed, “pay-to-publish” open access journals on the implicit premise that they have a stake in minimizing peer review. Most readers would agree that a sample composed entirely of journals of this type allows no valid conclusions to be drawn regarding the contribution of either “open access” or the “pay-to-publish” model to the shameful failure in quality control exposed by Bohannon. A sample consisting of a random mix of open access and closed-access journals, some predicated on the “pay-to-publish” model and others on the traditional publishing model, would have allowed for clearer insights into the root causes of gross negligence in peer review. It may puzzle some readers – as it puzzled me – to see open access getting a bad rap while the blatantly suspect “pay-to-publish” model, in which publication rates drive profits, is merely treated as a colluding partner. This is akin to trying to shoot an elephant by aiming at his shadow. We all know that “pay-for-profit” is not synonymous with open access. Bohannon’s failure to formulate a straightforward “mechanism of action” linking open access to shoddy peer review leaves room for nothing but speculation in the interpretation of his findings.

What Bohannon has demonstrated is that ineffective peer review is all around us and that the traditional gatekeeping role of editors is crumbling before our eyes. We don’t need a study to understand why. The “pay-for-profit” model, fueled, in turn, by the “publish or perish” paradigm, is one obvious reason. Another is the lure of a high impact factor, common to all scientific journals. Editors turn a convenient blind eye to the flaws of “buzz-generating” papers. Put more generally, Bohannon’s culprit is an alarmingly pervasive one: a publishing culture in which profit-making and sensationalism take precedence over science.

María Luisa Clark, Editor, Papers and Reviews, Bulletin of the World Health Organization

Los comentarios sobre la noticia:

It all adds up (Carlos Campillo / 23/12/2013 13:23:01)
Thanks for your comment, Vicente. At this stage of the game, actions such as the watchful appraisal of peer review, the public condemn of the irregularities found, holding referees accountable, and putting forward viable solutions are in order before throwing the towel. A modicum of the solution is up to us. We hold some sway over the journals you mentioned (and some others). It all adds up.

Il faudra payer (Vicente Ortún / 17/12/2013 11:41:44)
Congratulations Carlos Campillo for the excellent idea and the sound contributions from eight editors. Without commited readers of for example Gestión Clínica y Sanitaria or even this butlletin, willing to pay for quality science, comment and information, very few independent providers would remain. Similar problem observed with newspapers.

Deja tu opinión:

E-mail: (No visible)
¿Cuál es el número de en medio?
3 7 (9) 1 6

C/ Bonaire, 7 08301


Tel. 93 755 23 82 | Fax. 93 755 23 83

Editores del boletín: Carlos Campillo ( y Cristina Hernández Quevedo (

Editora de redacción: Cristina Hernández Quevedo (

Comité de redacción:
José Mª Abellán Perpiñán, Pilar García Gómez, Manuel García Goñi, Ariadna García Prado, Miguel Ángel Negrín, Vicente Ortún.

Han colaborado en este número:
José María Abellán Perpiñán, Alejandro Arrieta, Cristina Blanco-Pérez, David Cantarero Prieto, María Luisa Clark, Joan Costa-i-Font, Greg de Lissovoy, Manuel García Goñi, Ariadna García Prado, Ildefonso Hernández Aguado, Félix Lobo Aleu, Ricard Meneu, Marta Pascual Sáez, José Alberto Salinas Pérez, Fernando Ignacio Sánchez Martínez, Luis Carlos Silva Ayçaguer, Alexandrina Stoyanova, Andrew Street, Damián Vázquez.