Scientists who get too many references to their very own work inserted in others’ papers — whether or not by prior association or by asking for additional references throughout peer evaluate — may depart telltale fingerprints within the quotation file, say two researchers who’ve developed a technique to detect what they name quotation hacking.
“If somebody is attempting to control their citations, they’ve to depart this mark,” says bioinformatician Jonathan Wren, on the Oklahoma Medical Analysis Basis (OMRF) in Oklahoma Metropolis. On 13 August, he and Constantin Georgescu, additionally on the OMRF, posted an analysis of 20,000 authors’ quotation patterns to the bioRxiv preprint server1, during which they discovered round 80 scientists whose citations, they are saying, point out “power, repeated” reference-list manipulation. The patterns additionally recommend that round 16% of authors of their pattern “could have engaged in reference listing manipulation to some extent”, they add.
The research has not but been peer-reviewed and has many caveats, however its methodology seems technically sound, say two bibliometricians who weren’t concerned within the work.
Manipulating citations
Researchers typically complain that reviewers ask them to add unnecessary references to papers, a observe termed coercive quotation. Surveys recommend that round one-fifth or extra of scientists have skilled this. The writer Elsevier said last year that after inspecting peer-review data at its journals, it’s investigating some scientists whom it suspects of intentionally manipulating the peer-review course of to spice up their citations. Authors additionally typically arrange to cite each other.
Wren started to review quotation patterns after he found an outlandish case during which a extremely cited US biophysicist repeatedly manipulated the peer-review course of to achieve additional citations. The researcher, whom Nature reported this year is Kuo-Chen Chou, was subsequently barred as a reviewer from the journal Bioinformatics (the place Wren is an assistant editor), from the editorial board of the Journal of Theoretical Biology, and, Nature has now realized, as a reviewer for the journal Database. Chou advised Nature that he had not engaged in “reviewer coercion”.
Wren says that after he uncovered Chou’s behaviour, he started getting e-mails from researchers asking him to test the data of different students whom they thought is perhaps concerned in quotation hacking. However as a result of most peer-review processes are confidential, Wren hoped to identify such instances by inspecting quotation data. Heavy self-citation is easy to measure, however deciding what counts as an uncommon sample involving different authors is far tougher.
Wren and Georgescu thought-about many potential “pink flag” indicators, comparable to when researchers regularly obtain blocks of consecutive citations in others’ papers, or get disproportionately many citations from one journal. They discovered {that a} key measure that correlates with lots of the pink flags is the general skewness, or inequality, within the distribution of citations that scientists obtain from others’ work: some researchers are cited an unusually massive variety of instances by just some papers.
The researchers didn’t have entry to a database of all scientists’ work, so that they analysed public data within the database PubMed. In addition they restricted themselves to authors with center initials on papers, to make misidentification much less seemingly. This limits the research, however provides an thought of the quotation patterns for round 20,000 scientists. Round 80 — together with Chou — have extraordinarily skewed patterns of citations accrued from others, along with different red-flag indicators. Researchers who’ve unusual patterns of citations have a tendency additionally to quote themselves closely, the research discovered.
Requested for remark, Chou advised Nature that the research was “meaningless”, as a result of the “variety of citations isn’t essential”.
Wren doesn’t straight title researchers within the paper, due to authorized issues, however he analyses a couple of particular person quotation data intimately and says the complete figures can be found on request. One such scientist, in accordance with supplementary knowledge Wren confirmed Nature, is Dimitrios Roukos, an oncology researcher on the College of Ioannina in Greece. From 2009 to 2014, Roukos benefited from virtually 2,000 citations from 71 papers in a single journal, Surgical Endoscopy. Every research referenced his work round 20–30 instances, and was written by colleagues or mentors. Roukos didn’t reply to requests for remark for this story.
Surprising skew
The evaluation solely factors to uncommon citing patterns, and might’t assess whether or not a researcher truly did organize for additional references to their work; there could also be harmless explanations for unusual distributions, Wren notes. They is perhaps skewed by mega-reviews of a area that closely cite key researchers, as an illustration, though he says he examined this by eradicating such opinions from the evaluation. He sees skewed citing patterns as markers for additional investigation.
Plotting the worldwide distribution of skewness in data of scientists’ citations by others ought to supply a symmetrical curve, says Wren, however doesn’t. On that foundation, he means that round 16% of authors total have engaged in some type of reference-list manipulation at one time or different, even when it’s not attainable to conclude that their particular person data are unnatural. However Ludo Waltman, a bibliometrician on the Leiden College within the Netherlands, says that he doesn’t really feel snug with the best way the evaluation attracts a binary distinction between ‘manipulated’ and ‘non-manipulated’ references, when there are a lot of complicated causes for citing others.
Wren says he’d like editors and reviewers to develop a database that makes clear which references have been added throughout peer evaluate. Each Waltman and Vincent Lariviere, a bibliometrician on the College of Montreal in Canada, say that making peer-review studies extra clear may assist to deal with the problem.
An underlying downside that incentivizes quotation hacking is that scientists are too-often rewarded primarily based on easy quotation counts, says Waltman. This, he provides, is what finally wants to vary.
