Percentages: The Devil’s Metric

I once had a supervisor call to discuss our mortality rates for a particular diagnosis, and I honestly thought I was getting congratulated because only two people died that month, compared to twelve the same month a year prior. However, my supervisor had noticed that our mortality was approaching 50% and had actually called to ask why our numbers were so bad.

Suddenly, I was in trouble despite fewer deaths. What happened?

Perhaps this story is one reason why I have such strong feelings about percentages. Throughout my career, one of the best examples of frequently incongruent data is the horrific use of percentages. Too often, it’s a percentage that leads to mass chaos and confusion, and I caution anyone working with data to never, ever use a percentage without describing the total population it is based on.

The total population is also called the sample size, and when we know the sample, we can ask questions: Why were some numbers included or excluded? What are the raw numbers behind the percentage?

Every percentage is only a reflection of the sample size: When only two people died in a population of four who became sick enough to be counted, there is a 50% rate of death.

The year before, we had a pool of thirty seriously sick people and twelve of them died. In real terms, twelve deaths are worse than two, and four sick patients is way, way better than thirty; fewer people were becoming severely ill overall.

We were saving lives.

But when that life-saving work was displayed as a percentage without regard to the change in the total population of patients, it looked like nearly everyone was dying and the dashboard was bright red. Once we discussed the population, my supervisor decided that we were doing a great job. Whew!

The issue of population and sample size is an important one: We are trying to help patients get better and go home, which means we are really measuring against perfection—a high standard to be sure, but not too high when it affects real human beings. If we measure against a low standard—a population we know we’ll look good against—we may “standardize” ill health, disparities, or poor outcomes, reinforcing the problem rather than improving it.

So next time someone tells you that 80% of frogs are pink, ask them which frogs? Where? What species? And before you get depressed about the numbers on the screen, remember to ask how they got there in the first place.