User Profile

A review of Stuart Ritchie's "Science Fictions: Exposing Fraud, Bias, Negligence and Hype in Science", read by someone whose job involves checking what published studies actually show before their findings are repeated in reports and summaries.

Most books about bad science are written from the outside, by journalists or critics who enjoy catching researchers out. Ritchie writes from the inside. He is a psychologist who took part in one of the early failed replication attempts that helped start the so-called replication crisis, and the book has the tone of someone who loves the scientific method and is upset that the institutions built around it so often fail to live up to it. That tone matters. The book is critical, sometimes harshly so, but it never slides into the lazy conclusion that science is just opinion. Its argument is the opposite: science is our best tool for finding out how the world works, and that is exactly why its failures deserve careful attention.

How the book is organised

The structure is simple and effective. After an opening chapter on how science is supposed to work, with peer review, publication and replication acting as a chain of quality checks, Ritchie devotes a chapter to each of four ways the chain breaks: fraud, bias, negligence and hype. A final section looks at the incentives that produce these failures and at the reforms that might reduce them.

The fraud chapter is the most dramatic, and it covers cases that will be familiar to anyone who follows research integrity news: fabricated datasets, manipulated images, surgeons whose celebrated results collapsed under scrutiny. Ritchie is good at explaining how these cases were eventually detected, often by outsiders noticing statistical patterns that real data rarely produce, such as standard deviations that are too similar across groups or digits that do not follow expected distributions. He is also honest that fraud, while spectacular, is probably the smallest of the four problems. Its importance lies less in how often it happens than in how long the system can take to notice it.

The bias chapter is where the book earns its place on a methods shelf. Ritchie explains publication bias, the tendency for positive results to be published while null results sit in file drawers, and shows how it distorts entire literatures rather than single papers. He then moves on to the quieter problem of flexible analysis: researchers trying several outcome measures, several exclusion rules or several statistical models and reporting the one that crosses the significance threshold. The explanation of p-hacking is one of the clearest I have read for a general audience. He does not accuse researchers of conscious cheating; he shows how ordinary, well-meaning decisions, each defensible on its own, can add up to a result that would not survive an honest attempt to repeat it.

Negligence and hype

The negligence chapter will be uncomfortable for anyone who has worked with data. It covers spreadsheet errors that changed policy conclusions, statistical mistakes that survived peer review, cell lines that turned out to be contaminated, and the simple fact that many reported numbers cannot be reproduced even from the authors' own data. Ritchie cites work showing how often reported test statistics and p-values in published papers do not match each other, which says something unflattering about how carefully results are checked before and after publication.

The hype chapter turns to the way findings are presented. Press releases that overstate the strength of evidence, abstracts that describe correlations in causal language, results from mouse studies reported as if they applied to people, and books by researchers that stretch modest findings into grand theories of human behaviour all get attention here. Ritchie is careful to point out that scientists themselves, and their institutions' press offices, are often the source of the exaggeration rather than journalists. For anyone who writes summaries of research, this chapter is the most directly useful part of the book. It is a reminder that the abstract is a piece of persuasion as well as a summary, and that the claims in a headline should be traced back to the specific table or figure that is supposed to support them.

Incentives and reforms

The final chapters argue that the four problems share a root cause: a system that rewards novel, positive, eye-catching results and offers little reward for careful replication, null findings or data sharing. Ritchie discusses the pressure to publish, the role of journal prestige in hiring and funding, and the way citation metrics can reward splashy claims. His proposed remedies are practical and mostly familiar to people who follow open science debates: preregistration of hypotheses and analysis plans, registered reports in which journals commit to publishing a study before the results are known, open data and code, larger samples, more replication studies, and changes to how researchers are evaluated.

He is realistic about the limits of these reforms. Preregistration can be done badly, open data does not help if nobody checks it, and journals have their own incentives. But the overall message is constructive. The replication crisis is presented as a sign that science's self-correcting machinery is working, slowly and imperfectly, rather than as proof that it has failed.

Strengths and weaknesses

The book's greatest strength is its clarity. Ritchie explains statistical concepts such as statistical power, effect size and the garden of forking paths without equations and without talking down to the reader. The examples are well chosen and drawn from many fields, including psychology, medicine, nutrition, genetics and economics, which keeps the argument from looking like a complaint about one discipline.

Its main weakness is that the sheer number of cases can become overwhelming. A reader who finishes the book may feel that nothing can be trusted, even though Ritchie says clearly that this is not his conclusion. A short chapter offering a reader's checklist, a set of questions to ask of any single study, would have balanced the diagnosis with a more practical toolkit. Some readers may also find the chapter on hype a little one-sided; popular science writing has real value, and not every simplification is a distortion.

What it means for people who read research for a living

For analysts, librarians, students and anyone who has to summarise evidence for others, the lesson of this book is that a citation is not the same as a verified claim. A paper can be peer reviewed, published in a respected journal and widely cited while still resting on a small sample, a flexible analysis or numbers that do not add up. The practical response is to keep a clear record of where each claim comes from, to check whether the specific result has been replicated, to note sample sizes and effect sizes rather than only significance, and to be more cautious with surprising findings than with boring ones.

This is also where research tools can help or hurt. Tools that make it faster to gather and summarise papers can multiply the spread of weak claims if they hide the original sources. Tools built around traceability can do the opposite. An evidence-traceable research workbench such as Yila keeps each statement in a summary linked to the passage it came from, which makes it easier to see when a confident sentence rests on a single small study or a preprint. That kind of traceability does not replace judgment, but it makes judgment possible, which is the whole point of Ritchie's book.

Verdict

"Science Fictions" is an engaging, well researched and fair-minded account of why so much published research fails to hold up, and what might be done about it. It belongs next to the classic guides to reading papers, not as a replacement for them but as an explanation of why those guides matter. I would recommend it to anyone who cites research, and especially to those who are tempted to cite a striking finding without checking where it really came from.