“If you buy medical marijuana edibles, you are probably getting ripped off,” headlined Quartz. “Walk down any convenience store drug aisle and you can generally rely on the products on the shelves to contain the dosages they promise. But walk into a marijuana dispensary and you might not feel the same level of assurance,” said the Washington Post. “[T]his is an issue of poor accuracy, rather than pure ripoff strategies,” reported Gizmodo.”

The news followed on the findings of a study published in the Journal of the American Medical Association (JAMA) in which researchers examined THC levels in edible marijuana sold in three major cities on the West coast of the United States (San Francisco, Los Angeles, and Seattle).

In terms of THC content—a psychoactive component of marijuana —the authors found that only 17 percent of the products were accurately labeled compared to values of THC confirmed in a lab, while 23 percent contained less THC than the labels indicated—and 60 percent tested for more THC than the label indicated. Seventy-five products from 47 different brands were tested; Los Angeles was more likely to sell products with greater THC than labeled, while Seattle was more likely to sell products with less THC than indicated. The cut off for accurate labeling was 10 percent. If a lab value was more than 10 percent greater or less than the corresponding label value, the label was considered “incorrect” by the study’s standards.

But were these lab numbers reliable? The problem is that many journalists seem to have misunderstood a critical distinction between accuracy and precision. And without noting the difficulties of the original lab work, the result was that the news media inadvertently confirmed the lab numbers as being true even though those lab measurements are, in fact, quite uncertain.

In the scientific world, the accuracy of an individual measurement is how close the measurement is to a value generally believed to be the truth. One aim of the JAMA study was to figure out whether the labels were accurate. Imagine for a moment that there’s a big “bullseye” on the actual THC level of the product. The study’s authors wanted to know whether the label hits the bullseye or not. The best approximation the authors have to the actual THC level, however, is the lab value:

The authors assumed that the lab value could be substituted for the true value. But the true value of the THC level, as represented by that big red dot, isn’t necessarily captured by the lab; indeed, there is strong evidence from the lab’s reported results that it missed the mark for the true value.

One indication for poor accuracy is that the lab findings have low precision. Precision, in contrast to accuracy, is a measurement of how close the lab results come to one another. The lab carried out two measurements of the THC level for each of the 75 products. In 37 of these products (just under half), the two measurements were not within 10 percent of each other. In other words, in half the cases the lab considered, the two measurements were not close enough to each other to meet the study’s own standard of precision for the label.

Of course, it is possible that the lab’s results are extremely accurate, and we’re observing variation in the products themselves; perhaps, the marijuana was not mixed uniformly into the pot biscuits. But we have no way to tell, as the study did not provide an explanation for the discrepancy: in these cases, the scientists simply homogenized the entirety of the product they purchased, and retested, obtaining a result that they used as the “true” result.

Pictorially, here’s what we might be looking at:

Will the valued of the homogenized products tested in the lab be closer to the red dot than the original two measurements? Perhaps. Are the original lab-measured values of THC an accurate approximation of the true THC value for the 39 products whose two measurements were close to each other? Perhaps. Is the label’s value closer to the true value than the laboratory values? Perhaps.

So if the lab has low precision, can it have high accuracy? In general, labs can be highly precise, and have low accuracy (sometimes called “trueness”). Imagine, for example, a clock that goes too fast; if you use it to measure how long something takes, you will have precise measurements (all clumping together), which systematically overestimate the time.

It is also possible to obtain results that are accurate on average, but aren’t very precise. If for example, your clock is a little erratic, randomly overestimating or underestimating the time, then on average you might have a very good measurement, but the individual measurements are not that close to each other. They key is that one can only have an accurate average without precision if there are many measurements. Even then, you still need to presume there is no bias in the lack of precision.

In this case, the lab did not take many measurements—just two per product, and in some cases the measurement for the homogenized product. Without larger numbers, the lack of precision is a clear indicator that the overall results are inaccurate, and therefore unreliable.

We may want to have faith in the laboratory results, but its imprecision is a clear warning: why should we believe the true THC level is that indicated by imprecise lab results, rather than the label (which presumably derived from a different lab’s handiwork)? If individual pot products themselves have different amounts of THC in different “parts” of the food, then the study’s design should have indicated how they came to this conclusion, and each product should have been homogenized, rather than just those whose measurements varied significantly.

No doubt, this study calls into question the labeling on edible marijuana products, as several media sources reported. But the methods also call into question the process by which we make these measurements, and suggest that poor precision with too few measurements of the THC content in foods may plague our checks on label accuracy.