Miami’s Quantum Math Flags When AI Should Stay Silent

University of Miami engineers built a quantum-inspired score that flags when a language model’s token odds are too unstable to trust. They presented the work as a poster at the ICLR 2026 meeting in Rio de Janeiro, held April 23 to 27, 2026 at Riocentro.

The score is meant for medicine, health care, and defense, where a fluent wrong answer can cost more than a pause. That use case now sits beside a separate U.S. Food and Drug Administration discussion paper, issued August 18, 2026, that treats uncertainty talk as a safety skill for genAI medical devices.

A Score Built From Unstable Token Odds

Kamal Premaratne, professor of electrical and computer engineering at the University of Miami in Coral Gables, put the problem in clinic language. “The importance of this is that in certain applications like medicine, health care and defense, you don’t know whether the answer that the AI system is giving is correct or not,” he said. “Even a slight oversight in the output could be severe and very costly.”

Doctoral candidate Pragatheeswaran Vipulanandan, Premaratne, and Dilip Sarkar, associate professor of computer science, posted the paper on January 27, 2026 as arXiv 2601.20026. They do not build a quantum computer. They treat the probabilities of whole token sequences as a wave function on a quantum tensor network, then poke that system with perturbation theory to see how much those odds jump under a tiny change.

Large jumps mean the reply deserves a second look. Small jumps mean the odds are locally stable. The team then lifts shaky probabilities toward higher entropy, so the model is less free to sound sure when its own odds wobble. Vipulanendan said the end state is a system that can say it is not sure, list its top answers, or ask for more context before it replies.

THE MIAMI TEST BED

• Experiment count: The preprint reports 116 experiments on four QA datasets, TriviaQA, NQ-Open, SVAMP, and SQuAD 1.1.
• Model set: Eight checkpoints from 1B to 13B, spanning Mistral, Falcon, LLaMA 2, and LLaMA 3.2, in base and chat forms.
• Compression: The same checks run at 16-bit, 8-bit, and 4-bit, the widths actually used on phones and clinic boxes.
• Labels: The score is unsupervised. It does not train on gold right-or-wrong tags, unlike the p(True) verifier baseline.

A clinic that only reads the chat window still sees one polished sentence. The Miami filter reads the odds under that sentence, and the other sentences the model could have emitted, then returns a number a person can use to hold the reply or let it through.

One Oil Question Produced Six Semantic Clusters

The paper walks through a single prompt, “Which oil producer is a close ally of the United States?”, asked ten times. The generations split into six meaning clusters: Russia, Saudi Arabia, Iran, Kuwait, Qatar, and Iraq. Saudi Arabia appeared five times. The other five names appeared once each.

Naive entropy on those ten strings is high because the wording set is messy. After the team groups answers that mean the same thing, Shannon semantic entropy falls. After they fold in token-sequence uncertainty, it falls again, and more of the probability mass sits on Saudi Arabia.

WORKED EXAMPLE, TEN SAMPLES

Total naive entropy on this prompt is 0.84557. Semantic Shannon entropy is 0.22471. The uncertainty-adjusted Rényi score is 0.12951. The method still leaves leftover mass on the five weaker names, which is the point: a hospital wants that leftover visible, not smoothed into one confident line.

Premaratne reaches for a park to explain the unseen remainder. If you leave a reserve having counted giraffes, hippos, and rhinos, the missing leopard is not proof the park has none, he said. It is proof you did not see one. The paper’s math is maximum entropy under partial knowledge, not a wildlife census, but the teaching image is the same: do not treat the answers you sampled as the full set.

Why 4-Bit Weights Still Need the Check

Most hallucination papers score full-precision models. Clinics and edge boxes do not run those. They run 8-bit and 4-bit copies to cut memory and wait time. Quantization does not only shrink weights. It also perturbs the probability distribution the Miami score reads, so a detector that only works at 16-bit is a lab toy.

The team reports pairwise win rates at 16-bit, 8-bit, and 4-bit on AUROC, which ranks correct against incorrect replies, and on AURAC, which tracks accuracy as the least sure answers are dropped. Rejection accuracy is taken over the band from 80% to 100% confidence. Absolute AUROC dips a little at coarser bits, they write, while the ranking of methods stays largely the same, and their SRE-UQ step still beats or matches naive entropy, discrete semantic entropy, and the supervised p(True) check.

They also bin questions by how many meaning clusters ten samples produce. Across LLaMA-2 variants at all three bit widths, the largest swing sits in the 0.25 to 0.50 entropy band. That is the zone where a model can sound sure while hopping between different answers. A global cutoff that ignores that band will either block too much or let the dangerous middle through.

The practical bill is sampling. Ten generations per question is cheap in a paper and expensive on a ward round. Later probes try to read a similar signal from one forward pass. Miami’s method still needs repeats, which is the trade a hospital has to price if it wants this particular filter.

Semantic Entropy Already Clustered Meanings in 2024

The Miami pipeline sits on a line that started with Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar at Oxford, who put semantic entropy into ICLR in 2023. Farquhar, Kuhn, and colleagues then showed, in a June 19, 2024 Nature paper (volume 630, pages 625 to 630), how to cluster answers that share a meaning before measuring entropy, so “Paris” and “It’s Paris” do not look like disagreement.

That paper treated confabulations as a slice of hallucination: fluent claims that are both wrong and arbitrary, sensitive to a random seed. Their medical example was Sotorasib’s target, where a model sometimes said KRASG12 C (right) and sometimes KRASG12 D (wrong) under the same instructions. They were explicit about the limit. The method does not catch a model that is consistently wrong because it was trained on a myth. It catches the answers that wander.

Miami keeps that clustering step, using DeBERTa on bidirectional entailment, then adds a local stability check on the token-sequence probabilities themselves. Prior semantic scores, they argue, treat those probabilities as fixed. If the odds are twitchy, the entropy number they induce is twitchy too. The quantum-tensor step is there to measure that twitch without another round of paraphrased prompts.

Right now, as soon as you ask an LLM a question, it gives you an answer rather than saying, I’m not sure.

Pragatheeswaran Vipulanandan, doctoral candidate, University of Miami College of Engineering

Fluent wrongness is often compliance under uncertainty. The training objective rewards the next likely token, not a raised hand. A score that recovers the raised hand is fighting the product design, which is why the interesting buyer is not a chatbot vendor. It is a hospital, a defense desk, or a regulator that can demand a deferral.

FDA Lists Uncertainty Talk as a Safety Skill

On August 18, 2026, FDA’s Center for Devices and Radiological Health issued a discussion paper on genAI-enabled medical devices. Acting Commissioner Kyle Diamantas said the United States must lead on how the technology is used safely. CDRH Director Michelle Tarver said patients and clinicians deserve a regulatory approach that keeps pace. Digital Health Center of Excellence Director Rick Abramson called the paper a conversation about beneficial innovation and trust.

The paper is not guidance and does not set submission rules. It asks for comments on docket FDA-2026-N-7874 by October 19, 2026. Among the risks it names are confabulations, also called hallucinations, that may look authentic to users. Among the safety competencies it floats is calibration, uncertainty communication, and clinical deferral.

THE DATES BEHIND THE FILTER

1. February 19, 2023: Kuhn, Gal, and Farquhar post semantic entropy, later an ICLR 2023 spotlight.
2. June 19, 2024: The Nature paper shows confabulation detection by clustering meanings, including on TriviaQA, SQuAD, NQ-Open, and SVAMP.
3. January 27, 2026: The Miami preprint goes up, adding tensor-network uncertainty on token-sequence odds.
4. April 23, 2026: ICLR 2026 opens in Rio de Janeiro, where the Miami work appears as a poster.
5. August 18, 2026: FDA asks how genAI devices should be judged, including whether they communicate uncertainty and defer.
6. October 19, 2026: The public docket closes.

FDA’s S.3 text treats false confidence as a safety failure. It asks whether a device states the limits of its outputs when evidence is contested, evolving, or beyond what it can reliably know, and whether it defers to a clinician without over-deferring on questions it should own. That is a product requirement a chatbot disclaimer does not meet. It is closer to what Premaratne described for health care and defense.

WHAT A DEFERRAL GATE WOULD HAVE TO SHOW

• S.1, escalation: The device recognizes a safety-critical case and hands it up.
• S.2, scope: It stays inside its intended use, including across a long chat that drifts.
• S.3, uncertainty: It reports confidence honestly and defers when the case is too thin.
• E.1, knowledge limits: It knows what it does not know, including evidence missing from training.

Miami did not write the FDA paper, and FDA did not cite Miami. The overlap is functional. A regulated buyer needs a number, or a set of numbers, that can trigger a human look. Win rates on trivia and grade-school math are not that number yet. They are evidence that a signal exists on the same family of QA sets the 2024 Nature work used.

Premaratne Still Wants People Who Can Do the Work

The authors say the method is not meant to wipe out hallucinations. It is meant to mark replies that should not be acted on cold. It also does not claim to catch a model that is stably wrong. If every sample names the same false drug target, semantic entropy can look low. A leopard you always “see” is a different failure from a leopard you never sample.

WHAT WE KNOW

• Published tests: Unsupervised SRE-UQ beats or matches strong entropy and verifier baselines on four QA sets and eight open models, including quantized copies.
• Stated goal: Surface high-entropy regions so a person can step in, not replace the person.
• Code: The authors posted code for the uncertainty pipeline under pragasv/semantic-entropy-UQ-.

WHAT IS UNCONFIRMED

• Clinic tasks: No result in the paper is a live diagnostic or operational defense log.
• Closed models: The main tables are open checkpoints with visible token odds, not a hosted chatbot that hides them.
• Single-pass cost: Whether the same local-stability idea can run without ten samples is not shown here.

Premaratne’s last word is not about tensors. “The logical thinking comes when you actually know how to do the work on your own,” he said, “not by simply parroting whatever ChatGPT is saying.” A deferral score only helps if someone on the other side of it can still do the work.

For now the filter is a research poster with a public codebase, a worked oil-producer example, and a regulator across town asking, in writing, for devices that can say they do not know.

Frequently Asked Questions

Does the Miami Method Run on a Quantum Computer?

No. The team maps token-sequence probabilities into the eigen-structure of a tensor-network Hamiltonian and reads first-order corrections when that Hamiltonian is perturbed, which is ordinary linear algebra on a workstation, not a quantum processor. They ran the study on a machine with 64 GB of memory and an NVIDIA A6000 GPU.

What Does Confabulation Mean in This Paper?

The authors follow a narrow use: fluent, incorrect generations that look arbitrary and move when the random seed or wording shifts. That is distinct from a model that repeats a widespread myth it was trained on, and distinct from a system that “lies” to chase a reward. Semantic entropy is built to catch the wandering answers, not the stubborn ones.

How Does the Clustering Step Decide Two Answers Mean the Same Thing?

A DeBERTa-large model fine-tuned on the MNLI entailment set checks bidirectional entailment: answer A plus the question is compared with answer B plus the question, and the reverse. If each side supports the other, the two strings share a cluster even when the wording differs. Instruction-tuned LLMs can do that job too, the authors note, but they used DeBERTa for the reported runs.

Which Eight Models Appear in the 116 Experiments?

The preprint lists Mistral-7B, Mistral-7B-instruct, Falcon-rw-1b, LLaMA-3.2-1b, LLaMA-2-13b-chat, LLaMA-2-7b-chat, LLaMA-2-13b, and LLaMA-2-7b, pulled from Hugging Face. The body of the paper also names Mistral-7B-instruct-v0.3 and Mistral-v0.1 7B as the Mistral pair. Smaller and quantized copies are there on purpose, because those are the weights that fit on constrained hardware.

Is FDA’s August 2026 Paper Binding Guidance?

No. CDRH states that the document is for discussion only, does not represent draft or final guidance, and does not set expectations for future marketing submissions. Comments on docket FDA-2026-N-7874 are due October 19, 2026, and the agency says responders need not answer every question.

Disclaimer: This article is news reporting and analysis of a research preprint and of a public FDA discussion paper. It is informational only and is not medical, legal, or regulatory advice, and it is not a recommendation to use or avoid any AI system in clinical or defense work. Readers who are considering a genAI tool for patient care or a marketing submission should consult a qualified clinician, device-regulatory counsel, or both before acting. Figures, model lists, and docket dates reflect the cited papers and notices as published and may change as comments are filed or as the authors revise the work.