Decoded #7, part 2: What happens when PHI meets AI

Last week we discussed: health information becomes protected once it's part of someone's care. Away from care the same facts are personal data.

Now put AI in the middle of it. A lot of care runs through AI already (e.g., note taking, copilots) and that AI sometimes lives on someone else's platform. So the data leaves your hands and lands somewhere new. A model you don't own on a cloud you rent.

That brings back two questions aimed at the AI vendor.

First, are they on the hook? The serious cloud and AI providers will sign the same kind of agreement any hospital vendor signs. If they won't sign, your data has no business being there.

Second, do they learn from it. This is the newer one. When a patient's information runs through a model, does the vendor use it to train the next version? You want that answer to be no, in writing.

There's one more thing that's easy to miss. The platform and the company building on top of it each own a piece.

The platform secures the floor. The company decides what's allowed in the door. This is important not to lose sight of.

Most problems don't come from either piece failing. They come from the seam between them, where each assumed the other had it.

So when something gets called AI-powered in healthcare, the questions I recommend asking: Where will the data actually go, who signs for it, and will they promise not to keep what they learned?

Thoughts and perspectives welcome.