Blog Post

What “Predictive Benefits” Actually Means and the Questions You Should Be Asking

Almost every benefits vendor now says it can predict something: who will need surgery, whose claims will climb, who will leave the plan. But the term prediction no longer tells buyers what they need to know when two vendors can both claim to predict something and yet mean very different things by it.

·

·

8 min read

Almost every benefits vendor now says it can predict something: who will need surgery, whose claims will climb, who will leave the plan. But the term prediction no longer tells buyers what they need to know when two vendors can both claim to predict something and yet mean very different things by it.

Why one accuracy number tells you very little

The care that costs the most is also uncommon. Surgery, advanced imaging, and infusion therapy drive a disproportionate share of benefits spend, and in any given year they happen to a small share of members.

That changes what an accuracy number means. Imagine a plan of 1,000 members in which 10 will have a particular procedure this year. A model that predicts “no” for every member is right for 990 of them. That’s 99 percent accurate, but it’s identified none of the 10 members who needed help. 

Now take a model that does the work. Of those 10 members, it correctly flags 8 and misses 2. It also flags 20 members who don’t go on to have the procedure, so it flags 28 members in total. Counting all 1,000 members, it’s right for 978 of them, or 97.8 percent. By accuracy alone, the model that found none of the 10 beats the model that found 8.

That’s why a headline accuracy figure can’t tell you whether a system finds the members heading toward costly procedures.

Two important questions 

To get a fuller picture, two questions must be asked and answered.

The first: of the members the system flagged, how many actually needed the care? In the example above, the second model flagged 28 members, and 8 of them did. The second: of the members who needed the care, how many did the system flag? That model found 8 of the 10.

Either can be improved by sacrificing the other. A system that flags almost no one can get most of its few flags right while missing nearly everyone who needed help. A system that flags nearly everyone will find almost all of them, with most of its flags wrong. A vendor who reports only one of the two is showing you the flattering half.

Why we rank instead of promising probabilities

A probability sounds precise. “This member has a 70 percent chance of surgery” reads like knowledge. For rare events, that kind of number is hard to support for any one person, and it invites the wrong conclusion: that we know what a specific member will do. We don’t.

A ranking makes a smaller claim and one we can stand behind. Among these members, these are the ones most worth reaching first. It says nothing about what will happen to any of them. It also matches how the work happens. Outreach is limited by time and attention, so the practical question is who to talk to first.

A ranking is also easier to check. You can look at the members the system put at the top and see whether they went on to need that care more often than the rest.

At HealthJoy, what we identify is a signal about who to reach. Each one carries a label saying how much to trust it and none is presented to a member as a diagnosis or a forecast of a procedure.

Questions to ask any vendor claiming predictive capabilities

These work for any vendor: 

  1. What exactly are you predicting? A specific procedure within a stated time frame can be tested. “Members likely to be high cost” can’t and a vague target makes almost any result look good.

  2. How does your result compare with a model that says “no” for everyone? If the vendor can’t answer, the accuracy number isn’t telling you much.

  3. What are your precision and recall, reported together? Precision is how many of the members you flag actually need the care. Recall is how many of the members who need it you flag. Ask also how many members the system flags at a time and how that number was chosen. 

  4. Were you testing on information the model wouldn’t have had at the time? Results measured on data that only exists after the care happens look better than they are. Ask how testing is guarded against it.

  5. How early do you know and how often does the picture refresh? A flag that arrives after the member has chosen a provider can’t change the choice. Some tools re-score once a quarter or once a year.

  6. What happens when you flag a member and who sees it? Identifying someone early and putting them in a report isn’t the same as reaching them while the choice is open. Ask also who can see individual-level information and what stays with the member.

  7. How does the system indicate the trustworthiness of each flag? If every flag looks equally certain, the system isn’t telling you something it should know.

What a good vendor answer sounds like

A good answer to these questions is specific. It names the outcome and the time frame. It gives both measures and says how many members the system flags at a time. It explains how testing avoided information the system wouldn’t have had in practice. And it includes where the system is weaker: the flags that deserve less trust than others and the members it finds less reliably.

A weaker answer leads with one number, describes its target in broad terms, and moves quickly past what happens after a flag. That doesn’t make the system a poor one. It means you don’t yet have enough to judge it, so keep asking questions.

Identifying members earlier matters because a window exists before a member chooses where to get care and help that arrives inside it can still influence the choice. These questions test whether a system reaches the right members in time. A flag only matters if it changes what happens next, and you can only judge that when the vendor shows you the whole picture.

© 2026 HealthJoy. All rights reserved.

© 2026 HealthJoy. All rights reserved.

© 2026 HealthJoy. All rights reserved.