A tweet by William Lee (@wildscrutineer) hit the mark: “No punches being pulled at #METRICSconf15: Need to fix science before the ppl who pay for it reali[z]e what bad deal[s] they get.”
Hosted by the Meta-Research Innovation Center at Stanford University (METRICS), the conference gathered a vanguard of reformers around the problems of reproducibility and transparency, and why a lack of each is undermining trust in biomedical research. The goal of the conference was to move beyond describing the problems—although there was much of that—and find at least the outlines of a solution.
Much as the muckraking journalist Upton Sinclair observed that “It is difficult to get a man to understand something, when his salary depends upon his not understanding it,” the conference wrestled with the dissonance in scientists valuing transparency and reproducibility as core virtues of good science even as the academic and commercial world encourages them to assiduously avoid the former and ignore the latter. Openness and collegiality can be fatal to an academic career. Many academic researchers feel they must protect their highly valuable research as not to be “scooped” by competitors, but simultaneously, they realize the value of data sharing and how it can advance science for the common good. It’s a conundrum for any academic aspiring to tenure.
Brian Nosek, Executive Director of the Center for Open Science at the University of Virginia, argued that scientists needed to draw out these core values—quite literally in the form of badges signaling a commitment to data transparency. Such public symbols, while seeming “stupid,” as Nosek put it, actually work: In the journal Psychological Science, badges drove data sharing from 3% to 38%. Why? They signal community standards and enable social norming.
Stuart Buck, Vice President for Research Integrity at the Laura and John Arnold Foundation (which is a funder of both METRICS and Sense About Science USA) argued that regulators needed to set much higher community standards for data transparency: the Food and Drug Administration (FDA), for example, is sitting on a trove of data that doesn’t appear in academic study publication, and which is often vital to clinicians. He also detailed the absurdist lengths that Gilead Pharmaceuticals went to in order to avoid publishing data from its clinical trials: they claimed proprietorial confidentiality on the statistical methods they used. Ask any statistician and they’ll tell you that sharing your statistical methodology is essential to validating sound scientific findings and necessary for proper peer review; but that’s only half the point: how exactly can you justify proprietary claims on statistical methods? These processes are the building blocks of scientific research in the same way that addition and multiplication are the building blocks of math.
Tracey Brown, founder and director of the UK’s Sense About Science argued for the importance of including the public in this process of reform—as the AllTrials campaign has done by working with patients and patient groups to promote clinical trial transparency. Unless the public was engaged in “science about science” (to use Nosek’s phrase), the result would be cynicism: People would not discriminate in its mistrust of science, with disastrous consequences for public policy and individual behavior—along with funding for future research.
One commenter, who strongly advocates for reform of the current www.clinicaltrials.gov reporting system, went so far as to suggest that patient power and strike action—catalyzed by a “Patients Bill of Rights”—would be the most effective way to bring about change. Such a bill would advocate for non-participation in clinical trials if certain criteria were not met; other parties claimed that the standard of patient care was actually better when participating in a clinical trial.
While there are great strides being taken to promote clinical trial data sharing, there are still many hurdles to overcome—many of which involve the researchers themselves, their companies, academic journals, and academic institutions. METRICS showed that academics are ready to explore multifaceted solutions—and also that such change has to be in partnership with the public.