Pew Survey on AI in Health Care: What U.S. Adults Say About a Human in the Loop

Healthcare Dive's headline on a Pew Research Center survey reads: "Patients want a human in the loop for AI healthcare tools."¹

What happened

Healthcare Dive reports on a Pew Research Center survey of 3,488 U.S. adults about AI in health care.¹ As Healthcare Dive reports it:

• 44% think AI can schedule appointments well with a human involved.
• 46% say AI should not make insurance claims decisions, even with a human in the loop.
• 58% have little to no confidence that payers will use AI in good faith.
• 51% expect AI to hurt patient privacy and data security.

We are working from Healthcare Dive's report, not from Pew's own release. We give the figures as reported and add nothing about how the survey was run.

What it means for health system leaders

"Human in the loop" means a person reviews or approves what the AI does. As reported, the survey suggests that for one kind of decision, insurance claims, that phrase alone does not settle many people's concerns.

For health system leaders, this raises a concrete decision. For each task an AI agent takes on, leaders need to settle three things:

• which steps the agent takes on its own;
• which steps only a person takes;
• who is the named person accountable for the result.

Our perspective: what we think so far, and how sure we are

Our evidence comes from our own work with AI agents, not from hospitals. Treat what follows as a working hypothesis for health care, not a finding about it.

What we have published points to drawing the line between agent and person explicitly, one step at a time:

• Our AI Pattern "Steps Only a Person May Take" advises naming the few steps no agent takes, such as granting access, deleting records, and sending words in a person's name. Agents prepare those steps exactly, and a person reads, decides, and takes the step. The pattern states its status as Running for access and administrator changes and for the exact text of messages, and Early for requests to a person.²
• "A Record of Every Action, with a Way to Undo It" advises keeping a record of every action agents take for the organization: when, who, what, where, who approved it, and how to undo it, never of what people say to them. Its status: Running for recording actions; Planned for one log.³
• "Failures Reach a Person" advises sending every failure to the person who can act on it, having every routine confirm that it worked, and noticing when something that should have happened did not. It states its status as Early for checking results and sending failures to a person; a further part is Planned.⁴
• Our research note on a fully agentic organization describes a live test at this publication. AI agents do the daily work, each answering to one person, the editor in chief, who approves every article. Each kind of routine work is meant to run on its own only after its record earns it, with a random sample reviewed every week. Its status: Early.⁵

In a health system, our working view would look like this. For each task, name the steps that stay with a person. Keep a record of who approved each action. Make sure failures reach someone who can act on them.

These patterns come from our own work, not from care settings. Two kinds of evidence would change our mind. The first would be health systems finding that patients' trust does not change when these lines are named and recorded. The second would be people taking the named steps in name only, without real judgment.

The strongest view against ours

The survey itself supports a serious challenge to our view. As Healthcare Dive reports, 46% say AI should not make insurance claims decisions even with a human in the loop.¹ On that reading, the question for some decisions is not where to put the person, but whether AI should be part of the decision at all.

We think this view is fair. It asks a question our working view does not settle: which decisions should involve AI at all. Each health system has to answer that for itself, task by task.

One step to take

For each AI agent your health system runs or plans to run, can you name the steps only a person takes, and the person accountable for the result?

More coverage is in our Healthcare section.

This is not medical, legal, financial, or investment advice.