Long-context medical reasoning to reduce denials: post-training NVIDIA Nemotron 3.5 Super VL

Shan Chen, Daniel Borders, Tymor Hamamsy, Martin Seneviratne, and Jack Gallifant

Part one of our research series with NVIDIA explores how specialized open-weight models could help lower healthcare’s administrative costs, combining better appeal drafts with more efficient generation.

Key takeaways

• A passing appeal must cite sources that support every claim, include the essential facts and follow the required format. Under these strict checks, Opus 5 and GPT-6 Sol passed 16.4% and 35.2% of single attempts in their tested setups.

• Our ongoing collaboration with NVIDIA brings models and healthcare knowledge together. We combine Nemotron 3.5 Super VL and NVIDIA NeMo open training stack with R1’s appeal tools and task environments to develop R1-super.

• Our custom harness boosts the single-attempt pass rate by 7 percentage points for base Nemotron and GLM-5.3. Each comparison holds the main-model weights fixed and uses Pi with R1 skills as the baseline.

• Post-training nearly doubles R1-super’s single-attempt pass rate, from 17.2% to 31.3%. Each additional attempt adds generation cost, but four complete attempts raise its chance of at least one passing draft to 71.9%, compared with 56.3% for GPT-6 Sol and GLM-5.3 in their tested setups.

• Lower administrative costs require both better workflows and efficient AI. Evidence-grounded reasoning could help payers and providers resolve claims with less repeated work. Open-weight models offer control over adaptation and serving costs as these systems scale.

The work behind a hospital appeal

U.S. healthcare spending reached $5.3 trillion in 2024, or 18% of GDP. Reducing the administrative work around care is one practical opportunity to ease that burden. AI could play a deflationary role by helping payers and providers resolve claims with less repeated work. With specialized open-weight models, the aim is to lower hospitals’ total administrative expense without simply replacing it with token costs.

Denials and appeals are a practical place to start: they require payers and providers to revisit clinical records, payment rules and the evidence behind a decision.

When an insurer denies payment for care already delivered, a hospital can challenge that decision with an appeal. This work carries a substantial administrative cost. In an analysis published in 2025, Premier estimated that U.S. healthcare providers spent more than $25 billion contesting denied claims in 2023, including nearly $18 billion on denials that were ultimately overturned and paid.

Preparing an appeal requires skilled staff to work back through the patient’s record, establish why the care was needed and explain how it meets the insurer’s requirements for payment. Making that evidence clear and traceable could help both sides resolve disagreements with fewer requests, reviews and resubmissions.

The final letter may be short, but the research behind it is extensive. Three features make the work difficult:

• Evidence is scattered across clinical notes, test results, billing records and payer policies.

• Records can conflict, so the writer must reconcile the facts and account for relevant counterevidence.

• Payer rules can be complex, and each claim needs a precise citation that supports the argument.

Why capable models still make critical errors

For AI, this is both a long-context and a long-horizon task. The model must read a large amount of information and carry out many linked steps, from finding evidence to checking citations and revising claims. Here, “long horizon” refers to those steps within one appeal-writing task.

Frontier models can research and write well, yet still make claims that their sources do not support. A citation may point to a real document while overstating what it says. Our current internal benchmark treats unsupported claims as critical failures, alongside missing essential evidence and failures to follow the required format.

Building an appeal model with NVIDIA

R1’s experience with appeals gives us a foundation for defining this work. At R37, R1’s AI lab, we turn that expertise into environments that recreate appeal tasks, tools that help models work with the evidence, and checks that reveal where they fail.

Through our ongoing early-access collaboration with NVIDIA, we combine that foundation with Nemotron 3.5 Super VL 120B. We built a harness with R1’s tools, instructions and shared knowledge, then used supervised fine-tuning in NeMo AutoModel open library, part of NVIDIA NeMo stack, to teach the model from selected examples. Our results show improvements from both the harness and post-training.

The result is R1-super, our family of appeal models. Access to the weights lets us adapt the model to this work and test each version in a consistent environment. It also gives us control over how the model is served, creating room to improve efficiency alongside quality.

What counts as a passing appeal

Our initial study used an evaluation sample of 16 complex synthetic cases, separate from the training cases, spanning coding and medical-necessity disputes. Each case includes a denial, 25–52 supporting documents and instructions for the appeal packet—589 supporting documents across the sample. Denials experts qualitatively reviewed the cases and judged them representative of difficult denial cases. This initial study examines gains from the harness and post-training; an expanded exploratory evaluation of 50 synthetic cases includes this initial sample and additional cases. The appendix gives the case mix and evaluation details.

The model produces a structured appeal with its evidence, plus a readable letter. A fixed GLM-5.3 evaluator, served through Baseten, then checks the result against the sources and rules. This final evaluation is separate from the GLM-5.3-Flash helper and verifier used inside R1’s harness.

A citation alone is not enough: its source must support the claim. One unsupported claim fails the evidence check. Missing essential facts or breaking the required format can also fail a draft.

Reading the scores

One attempt is a complete run: the model reads the case, uses tools, drafts and checks its claims, makes any permitted repairs, and submits an appeal. Pass@1, the single-attempt pass rate, is the share of these attempts that pass the final checks. Pass@4, the four-attempt score, estimates the chance that at least one of four separate runs on a case produces a passing draft.

Models can take different paths and write different letters from the same evidence. Repeated attempts help us measure that variation. A four-attempt score does not guarantee success or show that a deployed system can identify the passing letter. These scores measure critical failures; they do not predict insurer payment or rank the richness of passing appeals.

A harness built for appeals

Our custom harness gives the model R1’s appeal tools, reusable skills and shared knowledge. The Pi baseline is configured with R1’s appeal skills, casepacks and task tools. Pi, a public coding harness, provides tools for reading files, editing them and running code. R1’s harness supports the appeal workflow with tools for reading and citing evidence, skills that guide the work, and a wiki of shared appeal knowledge.

During a run, tools help the model find source passages and attach citations. The helper handles bounded evidence tasks, while sentence-level verification identifies unsupported claims for repair. Figure 1 shows this workflow; Figure 2 shows the broader capabilities around the model. The final evaluator applies the same scoring rules after the appeal is submitted.

With the same main-model weights, our custom harness improves pass@1 by about 7 percentage points for both base Nemotron and GLM-5.3 relative to Pi with R1 skills. For base Nemotron 3.5 Super VL, that more than doubles the pass rate. The first trained R1-super version gains about 5.5 percentage points. These paired comparisons use the same task set and scoring rules; the appendix reports their attempt counts. The base Nemotron comparison precedes appeal-specific training and is separate from the latest R1-super results below.

The harness gives us a stronger starting point for training the model to work with appeal evidence. These experiments measure the combined system, rather than the separate effect of each tool or instruction.

Post-training improves performance further

Post-training teaches an existing model to do a particular job better. We used selected successful attempts and corrections from a stronger model for supervised fine-tuning of R1-super, then tested each version on separate development cases.

Across three trained versions, called iterations 0, 1 and 2, the single-attempt pass rate rose from 17.2% to 18.8% to 31.3%. Iteration 0 was already post-trained, so this is nearly twice its starting rate within the same evaluation harness. The four-attempt score rose from 37.5% to 50.0% to 71.9%.

At four attempts, R1-super scores 15.6 percentage points above GPT-6 Sol and GLM-5.3 in their tested setups. GPT-6 Sol remains ahead on one attempt, at 35.2% versus 31.3%. The comparison shows what each complete system achieves with the same attempt budget.

Fewer failed drafts lower the cost

Better drafts improve the economics of generation. Across the three R1-super versions, the estimated generation cost per passing draft falls from $1.80 to $1.60 to $0.96, a reduction of about 47% under the reported serving assumptions. Failed attempts are included in that cost.

For hospitals, those economics matter: the goal is to reduce the total cost of handling a claim, including the AI used to support it. An open-weight approach gives us flexibility to specialize the model and optimize its serving infrastructure around this workload.

What additional attempts buy

Generating more drafts can improve the chance of producing one that passes, but it also costs more. Each curve shows one, two and four complete attempts on the same case. Higher means a greater chance of at least one passing draft; farther left means lower generation cost. This is a sampling comparison, rather than a tested workflow for staff to request and choose successive letters.

Under these serving assumptions, four R1-super attempts cost an estimated $1.20 and reach 71.9% on our benchmark. Four GPT-6 Sol attempts cost $7.08 and reach 56.3% in its tested setup.

A broader comparison

Based on generation-cost and performance considerations in the first-stage evaluation, we focused the expanded comparison on our post-trained open-weight R1-super model, the open-weight GLM-5.3 model and the proprietary GPT-6 Sol model. Figure 7 presents the expanded exploratory evaluation of 50 synthetic appeal cases, including the initial sample, with four attempts per case.

For scale, R1-super’s main model is Nemotron 3.5 Super 120B. GLM-5.3 has 753 billion total parameters, with about 40 billion active per token—roughly 6.8 times our main model’s total parameter count. GPT-6 Sol was the strongest proprietary baseline in our initial study.

In this exploratory comparison, R1-super reaches 43.8% on a single attempt and 72.0% across four attempts, alongside 42.8% and 70.0% for GPT-6 Sol. GLM-5.3 reaches 29.8% and 71.0%, respectively. These exploratory results motivate follow-up evaluations on real-world cases and finer-grained assessments of reasoning, evidence use and alignment with human reviewers.

Toward lower healthcare costs

The opportunity for AI to be deflationary in healthcare lies in reducing repeated work across the payment process. Stronger automated adjudication and appeal systems could give payers and providers a shared, traceable basis for decisions, identify evidence gaps earlier and help resolve disagreements with fewer reviews and resubmissions.

The cost of the AI itself matters too. In our initial study with NVIDIA, post-training improved appeal-writing performance while lowering the estimated generation cost per passing draft under the reported serving assumptions. Open weights let us adapt the model and optimize how it is served, creating room to make reliable assistance more economical at hospital scale.

The next step is to test how these gains translate into real workflows, including staff time, review burden and payer outcomes. In the rest of our series with NVIDIA, we’ll share lessons from reinforcement learning in complex, multi-turn clinical environments, along with finer-grained evaluations of reasoning, evidence use, workflow efficiency and downstream economic impact. The goal is better collaboration between payers and providers, with less administrative expense surrounding the care itself.

A — Evaluation setup and scoring

Initial evaluation sample: n=16 synthetic cases (10 DRG, 6 medical necessity), separate from training cases. Most configurations use four attempts per case; archived Nemotron 3.5 Super VL base and iteration 0 Pi comparisons use eight. Failed and missing outputs count as failures.

The initial 16 casepacks contained 589 supporting documents, excluding denial notices (25–52 per case). The expanded exploratory evaluation comprised 50 synthetic cases: 20 medical-necessity cases and 30 DRG/coding-dispute cases, including the initial 16 cases.

The fixed Baseten GLM-5.3 evaluator uses two coverage draws, except the archived eight-attempt Pi comparisons, which use one. Final judging is separate from in-run verification. In the 50-case comparison, the two coverage draws yielded matching final pass/fail decisions for 85.4% of readable packets (82.2% for R1-super), with mechanical and entailment checks held fixed.

An attempt includes permitted repairs. Pass@1 measures single-attempt success; pass@k averages the fraction of k-attempt combinations containing at least one pass across cases and evaluator draws.

B — The rubric and the three passing conditions

A pass requires the correct structure, supported claims and essential facts, including relevant counterevidence. One unsupported claim fails the evidence check.

C — Reported results and comparison settings

Reported configurations from the initial development study. Iteration 0 is already post-trained; R1-super uses R1’s harness. Pi denotes the Pi harness with R1 skills throughout.

The paired harness comparison

Each pair holds main-model weights fixed and compares Pi with R1 skills against R1’s harness. R1 uses GLM-5.3-Flash for evidence tasks and sentence verification.

Settings and limits

Pi baselines have a 3,600-second limit. GPT-6 Sol uses high effort; Opus 5 and Sonnet 5 use earlier medium-effort runs.

The three R1-super iterations share one harness; untrained Nemotron uses an earlier revision with smaller repair and turn budgets.

These synthetic development results compare tested systems; they do not establish clinical effectiveness or payer outcomes.

D — Results for individual cases

Each column is one case from the initial development study. Darker cells indicate a larger share of passing drafts.

E — Generation cost assumptions

Initial-study generation estimates in USD include failed attempts, but exclude judging, selection, training, human review and deployment overhead.

How the costs are calculated

Cost per passing draft = cost per attempt ÷ pass@1. The k-attempt curves use k × cost per attempt.

R1-super’s $0.30 estimate combines $0.047 helper usage with projected $0.25 B200 serving at eight lanes. It assumes cached evidence preparation and excludes first-time extraction. Measured evaluation GPU cost was about $0.60 per attempt before helper usage.