In the early years of our profession, articles submitted to a journal were often not reviewed by the author’s peers but simply published. It would be hard to imagine an editor sending this acknowledgement to, say, Thomas Edison: “Tom, thanks for your manuscript. It will be reviewed by several of your colleagues and we will inform you of their responses as they are received.”
But times change. As I noted in an earlier column, when John Pierce of Bell Labs was named editor of the Proceedings of the Institute of Radio Engineers (predecessor to the Proceedings of the IEEE), he found he could not understand most of its articles. Nor could the members of his editorial board. He thus began to select specialist reviewers for each Proceedings article.
This practice was embraced by other IRE publications and continued when IRE became part of the newly organized IEEE. And it continues today, with the editor of each publication responsible for its peer-review process.
Qualified peer reviewers are harder to find as our profession continues its division into deeply complex specialties. And what is their reward? Even experienced reviewers having good intentions may find themselves mired in a manuscript beyond their knowledge and comprehension. The sheer (and increasing) volume of IEEE publications is daunting. IEEE reports that more than 1500 conference proceedings are published each year, and a total of 2.8 million conference papers are now available from IEEE.
Complicating the problem is the practice that we, as authors, embed our writing in arcane mathematics and phraseology and encumber it with acronyms decipherable only in context and by the knowledgeable. This helps us to distinguish ourselves not only from other professions, but also from other technical specialties, and may be responsible for the oft-repeated joke that only its author and one or two of its reviewers can really understand a journal paper.
So it perhaps should come as no great surprise that more than 100 IEEE conference papers were published from 2008 to 2013 that were reported to have been reviewed but that were subsequently proven to be hoax papers. They had been generated using SCIgen, a program created by a group of MIT students. It muddles together scientific-sounding words and phrases to auto-generate manuscripts that readily deceive the uninitiated and, evidently, the initiated too.
Meet Ike Antkare and SCIgen
In 2010, Cyril Labbe, a computer scientist at Joseph Fourier University, used SCIgen to generate 102 fake papers by a fictitious author he named Ike Antkare. These were added to the Google Scholar database, making Antkare the world’s most highly cited fake scientist, as well as number 21 among real and fake scientists combined. (Had someone at Google voiced his full name aloud, the embarrassment might have been avoided.)
Labbe’s successful experiment encouraged him to reverse-engineer SCIgen to create a program that can detect papers produced using SCIgen. He used it to detect the hoax papers published by IEEE. The list of those papers and their respective conferences can be found online.
Twenty-two of the papers were accepted by the 2010 Second International Conference on Computer Engineering and Applications. Computer engineering and applications are the central thrust of the majority of the “outed” papers. The paper entitled “Study on Redundancy Refinement in Autonomous Algorithms and Cooperative Methodologies Based on Evolutionary Programming” is one example. Another: “Impact of Grey Algorithms and E-Voting Technology on Exploration of Gigabit Switches and Applications for Redundancy.”
Welcome to Open-Access
The arrival of open-access publishing has further complicated the peer-review process. For a fee, publishers offer authors quick review and publication online. Users have free access. While there are legitimate open-access publishers, abusive practices among others have surfaced, and the term “predatory journal” was coined to describe them. The New Yorker in 2017 reported that a paper submitted to the International Journal of Advanced Computer Technology in 2014 consisted of a single seven-sentence phrase repeated over and over. It was promptly accepted and the “author” was billed $150.
Those who have studied the hoax paper phenomenon do not necessarily agree on its primary driver. A limited number of hoax papers were admittedly created and published to see if it were possible, and some participants saw it as a “fun challenge.” A more basic driver is believed to be the requirement by universities and research organizations that their engineers/scientists publish their work in highly respected journals, with the promise that they will be compensated accordingly.
The dishonest paper practices have alerted the public that certain members of the science/engineering profession are not as honorable and trustworthy as the profession itself expects and demands. And the inability of our profession to prevent the publication of phony papers lends credence to the arguments by the anti-science community that we can’t substantiate some of our claims—as, for example, in the case of global warming.
To help detect SCIgen papers before they are uploaded to Xplore, IEEE now uses an automated program. Its Technical Program Integrity Committee (TPIC) samples conferences for scope and quality issues. Some conferences are chosen randomly, and others based on past history, such as their known publication of SCIgen papers. When a SCIgen paper is discovered, it and all other papers from that conference are banned from posting on IEEE Xplore. If a problematic paper (SCIgen or otherwise) slips through and is posted on Xplore, it is immediately retracted when discovered. IEEE has not had to retract a SCIgen paper over the last four years (2014-2017).
Resources
- Evans, H. Do I Make Myself Clear? Why Writing Well Matters, Little, Brown and Company, 2017.
- Lott, M., “Over 100 published science journal articles just gibberish,” March 1, 2014, foxnews.com, retrieved July 8, 2017.
- Butler, D., “Investigating journals: The dark side of publishing,” Nature News & Comment, 27, 2013, nature.com/news/investigating-journals-the-dark-side-of-publishing, retrieved July 8, 2017.
- Burdick, A., “’Paging Dr. Fraud’: The fake publishers that are ruining science,” The New Yorker, 22, 2017.
- Labbe, C., “Ike Antkare one of the great stars in the scientific firmament,” International Society for Scientometrics and Informetrics Newsletter, 2010, 6 (2).
- Lifson, T., “Something is rotten in the state of science,” The American, Feb. 25, 2014.
- Kulkarni, S., “Is peer review really effective? The case of 120 withdrawn papers,” editage.com, Mar. 26, 2014, retrieved July 19, 2017.
- Van Noorden, R., “Publishers withdraw more than 120 gibberish papers,” Nature, 24/25. 2014.
- Labbe, C. and Labbe, D., “Duplicate and fake publications in the scientific literature: how many SCIgen papers in computer science?,” Scientometrics 94, 379-396, 2013.
- IEEE Conference Proceedings ieee.org/confproc (retrieved 7/19/2017).
- Peer Review at IEEE ieee.org/peerreview (retrieved 7/19/2017).
- Christiansen, D., “Engineers Can’t Write? Sez Who?,” The Best of Backscatter, 1, p. 21, 2008.
- Christiansen, D., “Inside Peer Review,” The Best of Backscatter, 1, p. 27, 2008.
Donald Christiansen is the former editor and publisher of IEEE Spectrum and an independent publishing consultant. He is a Fellow of the IEEE. He can be reached at donchristiansen@ieee.org.
