Connect with us

NEWS

Gemini 4 Carbon Halves Hallucinations and Still Loses the Default

Gemini 4 Carbon cuts false code claims to 18% and still loses the coding default, as Google gates its security edition and doubles prices on December 1.

Published

on

Gemini 4 Carbon cuts fabricated claims in code review from 36% to 18%, developer Theo Browne found, and he still will not make it his coding default. Browne, who builds the T3 Code editor, published that verdict on November 11 in a 29-minute video after Google opened the model to paying API users the day before.

The launch rate is $2 per million input tokens and $10 per million output tokens through November 30, 2026, then $4 and $20 from December 1. Google already tells teams to keep Gemini 4 Argon for most software work. Carbon’s one clean win in his tests is a security fix that matches the cyber edition Google still holds behind an application.

Eighteen Percent Is Still a Lot of Invented Code

Browne’s complaint is not a leaderboard miss. It is a model that invents database columns, line numbers, and architecture that is not in the repo, then says it with confidence. Across his evals, the blended false-claim rate fell from 36% on Gemini 4 Argon to 18% on Carbon, a cut that matched a figure DeepMind engineers had already given him in private before launch.

On his T3 codebase pass, Argon failed 47 of 92 claims and Carbon failed 23 of 86. He still would not ship from those answers. “He knows the vibe of what this repository does,” Browne said of Carbon. “He’s just not going to tell you with perfect accuracy what’s in it.”

BROWNE’S FALSE-CLAIM RATES

Model False-claim rate What he counted
Gemini 4 Argon 36% Blended share of invented claims
Gemini 4 Carbon 18% Same blended method, DeepMind match
Muse-line Wonder 14.5% His next-closest measured stack
Most other models 0-1% A handful sit at 3-4%

A Rust architecture review shows why the blended 18% and a single file can disagree. Argon made 36 claims and got 24 wrong. Carbon made 79 claims and got 23 wrong, more opinions, almost the same pile of fiction. Most rivals, on the same style of pass, invent zero or one fact. Official hallucination tests skip this failure mode, which is the one that makes a coding model unusable on a real tree.

Part of Argon’s mess was the old ACP bindings, the layer between T3 Code and Google’s model, which were not passing data such as line numbers the way the official Antigravity CLI does. DeepMind fixed that. Carbon still invents column names after the fix.

Google Already Split Carbon off the Daily Driver

Koray Kavukcuoglu, SVP of Google DeepMind and Google’s chief AI architect, called Carbon the company’s strongest model, with frontier cyber performance and long-horizon agent work. The November 10 product post is more careful about who should type it all day. Google wrote that Argon remains the ideal choice for the majority of software engineering and knowledge work, while Carbon is for jobs that burn more tokens on purpose: multi-step coding and debugging, open-ended scientific work, long verification loops.

Argon itself was announced on September 30 and reached the paid API on October 13. Carbon was shown on October 28 to a slice of cyber defenders, then opened on November 10 to the paid Gemini API, Google AI Studio, Gemini for business, and Google AI Ultra seats in the Gemini app, AI Mode, and Google Antigravity. There is no free API tier. In the Gemini app picker, Argon is the “Code & analysis” line and Carbon is “Most complex work.”

WHERE GOOGLE PUTS EACH MODEL

  • Argon: Everyday advanced coding, legal and finance knowledge work, faster production loops, tighter token use.
  • Carbon: Harder multi-step debugging, long agent runs, scientific discovery, and the Fairwind cyber build.
  • Carbon for cyber: Finding, exploiting, and patching bugs, still limited to approved Fairwind partners after the public API opening.

That split is the product, not a reviewer quirk. Google also says Carbon ranks first on CursorBench 4.0, Perplexity’s WANDR, and Zapier’s AutomationBench, at more than 2 times lower cost per task than the next external model on those tests. It ranks second on a Multimodal Knowledge Agent bench Google co-built with Artificial Analysis, at more than 6 times lower average cost than the top model. Those crowns sit next to a coding default Google is already steering off Carbon.

Tokens per Second Are Not the Wait

Browne rejects the fast-model line outright. Tokens per second, he said, is not what a user sits through. “Everybody’s saying, ‘Well, it’s a fast model.’ Shut up. You have no idea what you’re talking about. Like, I’m sorry if you think TPS means speed.” He called sessions with Carbon genuinely obnoxious and miserable to use.

In his timing, Carbon averaged 22 minutes a task on low reasoning effort and 27 minutes on high. Anthropic’s Opus ranged from 3 minutes 1 second on low to 25 minutes at max. Carbon was the slowest model in that set, and turning reasoning up barely moved the clock. A security pass he ran still took about 25 minutes.

WAIT TIMES ON THE INDEX

  • 67.83 seconds: Time to first answer token for Carbon at high effort, per Artificial Analysis.
  • 80 tokens/s: Output speed after that first token.
  • 25.5 minutes: End-to-end time per Intelligence Index task at high effort.
  • $7.76: $7.76 per Intelligence Index task at list rates of $4 in and $20 out.

On the same board, Carbon at medium effort is 23.0 minutes and $6.59 a task, which is why Google staff already tell colleagues to leave it on medium. The index is not Browne’s repo loop, and the two clocks should not be mashed into one “slow” number. Both still describe a model you wait on. After the November 10 opening, Antigravity users have watched simple follow-ups sit unanswered for minutes, which is the feeling TPS never captures.

Artificial Analysis lists Carbon (high) at 60 on Intelligence Index v5 and Argon (high) at 50. Carbon (high) sits at 4th of 74 models on that release page. Speed rank is 62 of 73. The score moved. The wait did not.

What Gemini 4 Carbon Costs After December 1?

The launch card looks cheap until the calendar and the cache sit on it. AI Studio’s paid-tier sheet is $2.00 per million input tokens and $10.00 per million output tokens through November 30, including thinking tokens. From December 1 those rates are $4.00 and $20.00. Context-cache reads are $0.10, then $0.20, which is the stated 95% off input. Carbon is not on the free tier.

GEMINI 4 CARBON API CARD

Meter Through Nov. 30, 2026 From Dec. 1, 2026
Input, per 1M tokens $2.00 $4.00
Output, per 1M tokens, thinking included $10.00 $20.00
Cache read, per 1M tokens $0.10 $0.20

Browne’s read is that the public discount is under a month, tighter than the two-to-three-month windows Google used to give, and that the post-promo card is Opus-level pricing for a worse time at the keyboard. “Normally intelligence is supposed to get cheaper over time but Google truly innovates on shitty things,” he said. He also treats the $200 Gemini subscription as a worse subsidy than Anthropic’s $200 plan, which he says feels near-unlimited for real work.

The leak is cache behavior. OpenAI exposes cache writes. Anthropic exposes writes plus a time-to-live. Google’s own pricing page says explicit caching is not supported, implicit only. Browne measured a 20% miss chance on a 5-second window, 34% on 30 seconds, and worse at 120 seconds, up to a 50% jump in requests in the bad case. “The token cost does not matter. The cost per task is what matters, especially when you consider how many tokens are being used in a given task.” Carbon, he said, burns tokens like nothing he has seen. Google’s own split already admits Carbon uses more tokens than Argon to rummage and re-check.

THE CARBON ROLLOUT

  1. September 30, 2026: Google announces Gemini 4 Argon and starts Fairwind access for cyber defenders.
  2. October 13, 2026: Argon reaches the paid Gemini API at the same $2 / $10 launch rates.
  3. October 28, 2026: Carbon is announced, first to a Fairwind subset, with the same launch rates and a footnote that $4 / $20 follows.
  4. November 10, 2026: Carbon opens to paying developers, enterprises, and Google AI Ultra seats.
  5. November 30, 2026: The $2 / $10 / $0.10 card ends.
  6. December 1, 2026: Input, output, and cache-read prices double.

Index cost at list rates still lands under some rivals on Artificial Analysis’s own tasks. Browne’s point is the task you actually run, with misses, retries, and a 25-minute loop, not the index average.

The Security Fix Nobody Else Would Write

The review turns on a vulnerability report against T3 Code from an independent developer, Brandon, who posts as Pseudo Brandino. He had Mythos access through a security program and found two niche issues, one almost impossible without a targeted attack, one easier if you trick an agent. Browne’s other models handled the write-up poorly. Fable 52 was asked how important the fix was. Opus 4.8, which he said was not even good at launch, produced nothing. Mythos named the bug and then proposed a six-digit code generated on a second machine, a ritual Browne has not physically done in months.

Carbon, tried almost as an afterthought, described the issue and proposed the split he wanted: known-machine flows on one path, raw-URL flows on another, tighter URL retention in the peer-link section, and no heavy cryptographic redesign. It also flagged two extra issues the report had missed. “You do not need heavy cryptographic redesign to kill this vulnerability cleanly. And doing the separation properly actually fixes two extra issues that the report didn’t call out.”

The fact that I had the best experience addressing these reports with this model is kind of insane. Like genuinely, I did not expect this.

Theo Browne, T3 Code, Gemini 4 Still Isn’t What I Need (Yet)

A council pass on Carbon’s first security draft scored it 9/10 from Opus 5.5, 9/10 from Muse, 7.5/10 from 61 Soul, and 7/10 from Terara. A separate Windows remote-update hunt, first flagged by a Microsoft engineer and hard for Browne to reproduce, drew 8-9/10 from reviewing models. He flags his own early-access seat: he may not be throttled like other users, which could pad the incident-response result.

Google’s cyber numbers point at the same corner of the catalog. On October 28 it said Carbon cuts unsupported claims by approximately 2x over Argon, which is the same direction as 36% to 18%. It posted 88.2% on SEC-bench Pro for finding bugs, 54.8% on ExploitGym for exploiting them, and 71.0% on CWE-bench for patching, all without guardrails. Internal secure-code success is more than 2 times Argon. Chrome’s team, Google said, saw more than 10 times better high-quality patches than another lab’s large model. High-severity misalignment alerts in live coding dropped by almost 80%. Indirect prompt-injection success on Gray Swan’s test is 0.0%, against 0.7% for Argon.

In Project Fortify, Google’s scan-triage-remediate pipeline, Carbon hit 100% recall on scanning and 95% precision on triage on an internal bench, with over 25% relative recall gain versus Argon on open-source repos. Cloud Vulnerability Research used it to prove two bugs in a popular open-source project were exploitable at a severity Google compared to PHP2Shell, then helped maintainers re-class the issue as critical. Wiz, which Google calls its leading cloud security platform, used Carbon in Scan for Good and found 0-day bugs in two major SaaS providers in under two hours, work the team said usually takes weeks.

The catch for buyers is access. General Carbon is on the paid API. The unguarded cyber build still runs through the Fairwind Program for trusted cyber defenders. Browne’s surprise is Google’s pitch with the safety rail taken off. He also said Carbon is nearly exclusively useful for cybersecurity dev work if you already have other models for the rest of the day.

Rings Floated off the Fish

Google’s November 10 post spends a long stretch on full-stack apps, UI taste, and 3D, including Antigravity /swarm demos of an Eiffel Tower exhibition, a Platform 37 reconstruction, a pelican riding a GBike around the Mountain View campus, and a reconstructed fortepiano. Browne’s samples did not look like that reel.

With Anthropic’s design skill enabled, Carbon’s front ends were worse than its native output. Natively it repeats a template: pill-shaped section, two-line centered text, a logo, a “begin” button, four links, a cropped screenshot, with the color swapped. He counted roughly two of the batches as actually different. On a fish-model 3D demo, the first pass was embarrassing next to Fable 52. Told to make it beautiful, Carbon made decent water and broke everything else. After about 25 minutes the fish was less bad, still not good, with a questionable propeller and rings floating off the body instead of around it.

Google lined up quotes from Hebbia, Balyasny, Glean, OpenCode, Cartwheel, HubX, and others for planning, finance decks, cloth simulation, and a 94% resolution rate on 16 bugs in a 14,000-test codebase. Browne says he had to revert entirely after the OpenCode team praised Argon. His own 3D pass is why he will not take the design claims on trust. WebDev Arena, in Google’s October 28 table, has Carbon at 1768, up 89 elo, in 4th. That is a public board. It is not the fish.

The Lab That Answers in a Day

Browne spends a large share of the video on DeepMind’s support, which he ranks first among labs for closing bugs he files, ahead of OpenAI and the SpaceX AI group that now owns Cursor. The commercial path matters as much as the manners. Google will not let a Gemini subscription drive inference inside arbitrary harnesses. It will let another tool drive Antigravity through ACP bindings. Those bindings used to be a separate binary T3 offered as an install. DeepMind has since folded them into the Antigravity CLI and added a T3 Code identifier so the tool is treated like Zed.

WHAT DEEPMIND CHANGED AFTER HE PUSHED

  • Subscription policy: Unclear rules on using a Gemini plan inside third-party tools were rewritten the next day.
  • ACP data path: Missing line numbers and related payload bugs were audited and patched after he reported absurd hallucination rates.
  • Editor parity: A T3 Code identifier now sits beside Zed; only SpaceX, for the original Grok build, and DeepMind have landed code directly in T3.

He still objects that the Antigravity CLI is no longer open source in the way the separate Gemini CLI was, and that the interface prints his email address everywhere. A separate hands-on review after the November 10 opening called Carbon slower, worse at tool calls, and in some tasks behind Argon, a colder read than Browne’s “less bad than before.” Replies under Google’s own availability post went at the Ultra-only consumer gate, with Anthropic’s Fable sold as open to everyone.

Browne is considering promoting Carbon out of “jester” status in his model council so it can collect review feedback over time. He would not pick it as the coding default. “Would I ever pick this model as my default for coding? No. Absolutely not. It’s just Opus is unbeatable still.” He also called it crazy that a model from the company with one of the biggest codebases in the world struggles on big codebases, and that the others do not. He thinks Google should have held Argon and shipped Carbon first. “I’m not going to say Google cooked, but it’s significantly less bad than before.”

From December 1, Gemini 4 Carbon bills at $4 per million input tokens and $20 per million output tokens, with implicit cache reads at 20 cents. Browne said he would not make it his coding default. Google already told everyone else to keep Argon for that job.

Harry is the editor and lead writer of CUMBERNAULD MEDIA, which he runs as an independent publication after a decade in journalism spent moving from reporting to editing. His habit is to open the document before the summary of it. A company result is read from the filing rather than the press release, a court or regulatory decision from the judgment itself, a scientific finding from the paper and its methods section rather than the headline claim, and a sporting sanction from the governing body's own ruling. That approach shapes coverage across news, business and technology as much as science, sports and entertainment, and it carries into the lifestyle, travel, auto and gaming pages, where product specifications are checked against the manufacturer's sheet and, where possible, against Harry's own testing. Every number is checked before publication, and where a source's figures are disputed the story says so. Corrections follow a public policy and are marked on the page. Readers anywhere in the world who write in get a reply from him, and the address is support@cumbernauld-media.com.

Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Trending