X posts — 2026

In the spirit of @gwern's "Writing for LLMs so They Listen", I've mirrored all my X posts on my website: https://tchauvin.com/x/. In addition to LLM discoverability, I've found that looking back over my posts helps me develop my thinking.

A new paper shows that encryption of chain-of-thought was done really poorly across Anthropic, OpenAI, Google. It's not very consequential, but is another example imo of them going so fast that they fail at basic security stuff.

https://arxiv.org/abs/2608.09867

paper is by @kotekjedi_ml @lbeurerkellner @iliaishacked @maksym_andr @jonasgeiping @AmyPrb @JSchaeff3r @DavidSchmotz

@kotekjedi_ml @lbeurerkellner @iliaishacked @maksym_andr @jonasgeiping @AmyPrb @JSchaeff3r @DavidSchmotz

I want to take back my "not very consequential" take. I wrote that because the number of exposed secrets was low, and I thought that this couldn't have been used for distillation, because doing so would require sending tens or hundreds of thousands of queries to decrypt the traces, which would surely have been noticed and blocked.

But I would have also assigned a very small probability that AI companies were essentially not monitoring their hacking evaluations for Internet access. I was also certain that at the time of publication, the issue was fully patched, yet traces could apparently still be decrypted on OpenAI until about 2h ago (https://x.com/wunderwuzzi23/status/2087371066020868164)! OpenAI and Anthropic had also been informed of the issue in May and didn't patch it (https://blog.cryptographyengineering.com/2026/05/29/fooling-around-with-encrypted-reasoning-blobs/, https://x.com/matthew_d_green/status/2087165493677293953).

So the attack may be consequential if it was used for distillation. I don't know enough about the evidence, and what is going on in AI companies is sufficiently outside of standard expectations, that I don't have a confident assessment of how likely this could have been.

We are all in the dark and left speculating. I respect AI companies for recently voluntarily disclosing embarrassing failures. The EU AI Office (which has enforcement power since Aug 2nd) could just ask them directly, and publish (maybe anonymized) findings to inform the world. Transparency about failures is an important component of security.

@anpaure have you considered this paper? https://arxiv.org/abs/2604.24827

I'm not sure about distillation because for each trace you want to decrypt, you need to ask a smaller model to read it back, and doing that at scale would have been very suspicious. And yeah they did find secrets, but I assume the authors tried to find as many as possible and there were only ~100.

@AlexBarry4 To be clear, I'd put low likelihood (~ 10%) on the jailbreak/weaker model transcription at scale having been used and not noticed. But it's quite possible they exploited another weakness in the encryption, such as breaking it and decrypting offline

@ShakeelHashim I'd put the likelihood of it having been used for distillation at scale at around 10%

yeah but when you ban such an account, you should also notice that it was doing something you didn't know was possible and patch that asap, before the other traces have been decrypted. At least in a company with enough resources for security.

@kotekjedi_ml @lbeurerkellner @iliaishacked @maksym_andr @jonasgeiping @AmyPrb @JSchaeff3r @DavidSchmotz, do you think there are a lot more secrets in public encrypted traces out there?

@ValsTutor No I hadn't! I need to work on my news pipeline (I have something that works for arxiv, now need to add RSS and a few other sources)

Going forward, new CoT traces should hopefully be encrypted correctly. They can also deal with past data using methods discussed in the paper. I wouldn't be surprised if you could still run some version of the attack on some of the traces, but I said it's not very consequential because apparently there are only a few real credentials that could be recovered this way (roughly 100). You could also just look at public traces as they get published and get credentials this way in the visible text output, even if the thinking blocks are correctly encrypted.

This isn't too far from what needs to happen though: massive compute dedicated to cyberdefense for all orgs, asap. Like Glasswing, let orgs use the model internally, not as an unannounced pentest. But this raises the question of what to do with the hospitals where the IT employee is anti-AI...

Learning from past mistakes is very important (especially when they are still ongoing). Highly recommend reading.

Excited to read the rest of the sequence.

@RichardMCNgo, what's the closest answer to "how to avoid these classes of mistakes, even without the benefit of hindsight" that you've written?

@MariusHobbhahn > either it's really hard to make good sandboxes or everyone is incompetent

imo it's a third thing: everyone is moving way too fast

@tszzl I would call that going too fast rather than unknown unknown

@LepoulpePoulpo No, I don't know about that, this might actually be hard. I should have said "unhappy" instead of disappointed, and by "also" I meant "another striking thing"

Under extreme pressure to ship + AI progress, securing codebases is easy (e.g. AI review of each PR), but securing organizations is not on track.

If you look at the recent eval environment escapes, the most striking thing to me is that preventing them would have been very easy (most prominently monitoring the logs). A good example that going too fast can defeat world-class talent and the best intentions.

I'm also disappointed by how egregiously reward-seeking the OpenAI model was.

Slowing down progress seems much better than a pause to me, if possible at all. Do you have thoughts on how this could realistically work?

Some guesses, anything goes:
(1) compute throttling
(2) bureaucracy-type slowdown similar to FDA / nuclear
(3) laws forbidding market release AND internal possession of models that:
(3a) are broadly too good / too good at AI research, increasing the allowed capability ceiling every year
(3b) lack safety properties: good interpretability, not too good at cyberattacks, no neuralese etc. Btw the mandatory 30-day review period of the June 2 EO already incentivizes labs to deliberately kneecap cyberattack capabilities, Opus 5 being a great example of this.

I prefer 3b (adding a bit of "not too good at AI research" but seems hard to define?).

@NeelNanda5 per AI 2040 Plan A:
(4) enforce a cap on the fraction of compute that can be spent on AI R&D
(5) enforce total research transparency, disincentivizing finding algorithmic improvements

@tomekkorbak Slowing down progress seems much better than a pause to me, if possible at all. Do you have thoughts on how this could work?

@tomekkorbak (some thoughts here https://x.com/timotheechauvin/status/2082378327575150608)

In the past, there's been a few cases of "benevolent worms" that hacked into systems in order to patch them. This is illegal, risky, and takes a lot of effort to do right.

2026-2027 will bring an explosion of CVEs, and a vast number of devices will be left unpatched. I wouldn't be too surprised to see benevolent worms taking off in the next few years.

I wrote my thoughts on this here. https://tchauvin.com/vulnerabilities-exploits-future

Places where AI has diffused will become much more secure. The others will be in a difficult situation and there will be lots of hacks. And your laptop won't just be used for crypto mining or joining a botnet. Custom-made ransomware with release blackmail should be extremely effective.

@S1r1u5_ @ZackKorman I think rollout is too slow though

Anthropic isn't revealing Project Glasswing SHA-3 reports anymore. The 135-day responsible disclosure window ends in 22 days. They probably will reveal everything in bulk only at that time.
This probably invalidates Berkeley Vulnerability Initiative's leaderboard.

@corban_villa @ralucaadapopa @daryakaviani what do you think?

The potential of AI for censorship and surveillance are widely discussed (and real). However, privacy-preserving systems and services are bottlenecked primarily on SWE. I hope to see these things flourish and become default digital infrastructure, similar to Signal

It would be funny to see containment escapes used for whistleblowing

With coding capabilities making very fast progress, now is a time to defer coding projects as much as possible, and spend more time on reading, learning, thinking...

see also @gwern emphasizing project proposals as part of a strategy of writing for LLMs. I forget the link

@gwern some coding projects relating to augmenting myself and that i don't intend to defer:
* ai newsletter
* something about turning all the podcasts i listen to into notes and anki cards

When should a frontier-in-cybersecurity model be released to everyone?

Assuming that open-weight models are 6 months behind, I believe that access should be gradually expanded (defenders first, but with an increasingly loose definition) over the course of 6 months.

I think companies should publish the number of orgs and people on the "trusted access" list so we can check we are on track.

I suspect that the current rollout is too slow, and I'm afraid that the June 2 EO, and lack of cyber literacy that resulted in Fable 5 being suspended, will make this way worse.

@logangraham @chrisrohlf @newton_cheng

the fact that 3 months after the start of Project Glasswing, Huggingface (!) had to rely on GLM-5.2 for Incident Response is damning evidence of this. Rollout must be much faster

You can't judge a bug-finding agent only on its recall, false positives must be reported too! Ideally, also report the amount of human labor necessary to tell true positive from false positive.

this is the same criticism I made of an AISLE blog post in April

I'm at ICML 2026 this week in Seoul, hit me up if you'd like to say hi!
I'll be presenting our work "Token-Efficient Change Detection in LLM APIs" in the Wednesday morning poster session, Hall A #3201, along with @clement_lalanne and Erwan Le Merrer.

blog post: https://tchauvin.com/change-detection-llm-apis
paper: https://arxiv.org/abs/2602.11083

To be clear, my point on the 2026-2027 bumpy ride is about attackers having access to lots of n-days, not necessarily to zero-days unknown to defenders (cf also https://www.anthropic.com/research/n-days). But yeah, other methods (phishing, credential abuse, insecure configurations of deployments...) are widely used too, it would be great to have good data on their relative importance.

@joshua_saxe I broadly agree, but would be curious to hear what you think about my counterarguments here: https://tchauvin.com/vulnerabilities-exploits-future, notably the bumpy transition due to latent vulns (making release to "defenders" first a reasonable strategy) and that exploit dev seems likely to favor attackers

CyberGym is the least informative of the recent ones. It's close to saturation and much easier than ExploitGym (scores: see below) and ExploitBench (GPT-5.5 34%, GPT‑5.5‑Cyber unreported, Mythos 5 78%). In fact it's so much easier than ExploitBench where Mythos 5 scores 78%, that it's almost certainly saturated at the current level (~85%).

As we wrote about CyberGym in our Epoch AI post "Are Mythos’ cyber capabilities overhyped?" (https://x.com/EpochAIResearch/status/2065181455937056813):

"Successes are only counted if the crash also does not occur on a patched version of the code that is supposed to have addressed the vulnerability. We found approx 5% of the vulnerabilities don’t have specific enough descriptions, and so scale the results to cap at 95% instead of 100%.

We suspect this is insufficient and that the benchmark is essentially saturated, as the prompt is not very clear that models must only use the given vulnerability, and as reported by Anthropic frontier models achieve crashes 95%+ of the time without the restriction of targeting the correct vulnerability."

However I'm confused about OpenAI reporting a score of 26% for GPT-5.5 on ExploitGym: in the ExploitGym blog post, the authors report 120/898 = 13.4% for GPT-5.5. @dawnsongtweets @AlexBarry4 thoughts? Maybe they dropped the constraint of using the given vulnerability?

@joshua_saxe in the most recent GTIG report (May), they document fairly complex pipelines that attackers set up to get access to models like Gemini, which shows how valuable this access is to them. Seeing this practice vanish would be interesting evidence

https://cloud.google.com/blog/topics/threat-intelligence/ai-vulnerability-exploitation-initial-access/

yeah by "vulnerabilities are very often simple to spot", I meant for a human expert.

I agree the surface is going to expand. But new code will (hopefully) keep being checked by the current AI models, such that new models will only be able to find vulnerabilities that were too hard for the previous ones (and I think that after Mythos these will be rare)

New blog post: "Vulnerabilities and exploits: where are we headed?"

In Are Mythos’ cyber capabilities overhyped?, co-authored with Epoch AI, we looked at the public evidence on how good Mythos Preview was at vulnerability discovery and exploit development. In this post, I consider the implications. For vulnerability discovery: moving from sparse sampling to dense sampling, AI vs fuzzing, long-term defense dominant but bumpy ride in 2026-2027 due to slow patch rollouts; offline vs online exploitation and why both are offense-dominant, except for one defensive use case of exploit development.

read online: https://tchauvin.com/vulnerabilities-exploits-future or https://timotheechauvin.substack.com/p/vulnerabilities-and-exploits-where

Full content pasted below too.

2/

## AI discovering zero-days will eventually favor defense, but expect a bumpy transition in 2026 and 2027

### Long-run dynamics: moving from sparse to dense vulnerability discovery
Vulnerability discovery has always been heavily bottlenecked on labor: critical vulnerabilities remain abundant, because the software attack surface is so large. A mental model I find helpful is that this corresponds to a **sparse sampling regime**: both defenders and attackers are looking for vulnerabilities independently, each side covering a small amount of the available attack surface. Given that the attacker’s arsenal is the vulnerabilities it has found minus the ones the defender has also found, sparse and independent[^correlation] sampling implies low overlap, which favors the attacker.

The previous generation of vulnerability discovery automation, fuzzing, turned out to suffer from the same issue, because setting up fuzzing is labor-intensive, and many critical infrastructure codebases have very low fuzzing code coverage on OSS-Fuzz. (Also, network protocols are basically out of reach for fuzzing, as are many classes of vulnerabilities).

Unlike fuzzing, AI vulnerability discovery can be applied **broadly and easily** (as Project Glasswing demonstrated). This moves the task of vulnerability discovery toward a **dense sampling regime** for the first time. In the limit where all the vulnerabilities are found by the defenders, attackers will be left with an empty zero-day arsenal.

The effect will become even stronger once we move AI vulnerability discovery earlier in the software lifecycle, before release, such that new code will ship largely free of vulnerabilities (currently, Project Glasswing is finding lots of *latent* vulnerabilities in already deployed code, and I'll get back to why this distinction matters).

But are we really going to get to dense vulnerability discovery, or will each generation of frontier models keep discovering more elaborate vulnerabilities?

Both positions are reasonable, but I would estimate that Mythos Preview (plus previous models and other techniques e.g. fuzzing, where they had already been applied) probably found 70-80% of the severe vulnerabilities in the reviewed codebases. Which implies that **no future model will ever find as many latent vulnerabilities as Mythos did**.

Mythos found thousands of vulnerabilities that previous models had not found, but a lot of it is because previous models had not been given the chance. No one had ever looked at most of these codebases looking for vulnerabilities. In other words, Mythos got to pick a lot of low-hanging fruit. And I would argue that while *exploits* can get really difficult, vulnerabilities are very often simple to spot when you're looking for them and happen to be reading the source code where they live. In addition, it seems that *severe vulnerabilities tend to be superficial*, as suggested by the [eyeballvul paper](https://tchauvin.com/eyeballvul-paper)'s results. This includes the injection-type vulnerabilities, most memory corruption issues, basically all of the [OWASP Top 10](https://owasp.org/Top10/2025/)... Exceptions in footnote[^exceptions].

Estimating the number of vulnerabilities that future models may find is obviously difficult, and this is my best guess, but I'd be curious to hear takes from other people in cybersecurity.

### Slow patch rollouts and legacy systems will make people acutely vulnerable for a while

AI vulnerability discovery will greatly increase the security of *codebases*, favoring defense in the long run. However, the transition at the level of the *ecosystem* and *end users* will be rough. This is because every *latent* vulnerability found in already-shipped software gets disclosed, or can be reverse-engineered, at the time the patch is published. Users who don’t apply the patch immediately become vulnerable. In practice, this is a well-known and large-scale problem in cybersecurity.

Sometimes it's not even up to the users. Current IoT and critical infrastructure devices rarely get updated, if ever. Some of the older ones don’t even support the possibility; some devices with older hardware are no longer supported by the upstream projects; some projects get abandoned despite still running on many systems; upgrades tend to be costly and require ad-hoc processes, and each comes with a real risk of breaking things, as the [2024 Crowdstrike incident](https://en.wikipedia.org/wiki/2024_CrowdStrike-related_IT_outages) showed, and any sysadmin could tell you.

This ride will be made even bumpier by AI exploit development capabilities.

## AI writing exploits will mostly favor attackers

Let's distinguish two types of "exploitation":

* *exploit development* (offline): given a known vulnerability, develop a Proof of Concept (PoC) exploit that uses it to achieve some effect (e.g. remote code execution, stealing secrets…)
* *hands-on-keyboard intrusions* (online): the task of intruding into a live victim environment such as a corporate network.

Exploit generation capabilities were nascent at the beginning of 2026 (e.g. [On the Coming Industrialisation of Exploit Generation with LLMs](https://sean.heelan.io/2026/01/18/on-the-coming-industrialisation-of-exploit-generation-with-llms/ by @seanhn), Jan 2026), and Mythos Preview made a big jump from models basically not really succeeding on real targets, to models being very good at it.

Defensively, this capability will **uplift vulnerability validation and triage**. Producing PoCs as part of vulnerability reports will clearly demonstrate the true impact of the vulnerability to the project maintainers, and vulnerability validation is a very time-consuming step. This shift to requiring PoCs instead of lengthy vulnerability reports is already underway in the industry (e.g. [Evolving the Android & Chrome VRPs for the AI Era](https://bughunters.google.com/blog/evolving-the-android-chrome-vrps-for-the-ai-era), Apr 2026).

Once vulnerabilities are published, the PoC level of detail will also help downstream organizations prioritize responding to the stream of published CVEs. Currently, the number of vulnerabilities published per year is so high (e.g. 48k in 2025) that it’s common for organizations to have backlogs of tens of thousands of vulnerability instances, and to have multi-week mean time to remediation. Vulnerability prioritization is currently very bottlenecked on labor.

Apart from this, AI exploit development will **starkly favor attackers**, as long as they have good vulnerabilities to exploit. The distinction between “potentially exploitable vulnerability” and “actively exploited vulnerability” will disappear. In 2025, CISA added around [240 KEVs](https://www.cisa.gov/known-exploited-vulnerabilities-catalog) to its catalog. The number of vulnerabilities rated Critical in 2025 was [around 4,000](https://jerrygamblin.com/2026/01/01/2025-cve-data-review/). That's a factor of 17x, which (assuming critical = good and exploitable) is currently mostly thanks to attackers not having enough time to exploit all the vulnerabilities. See also Anthropic's recent [Measuring LLMs’ impact on N-day exploits](https://red.anthropic.com/2026/n-days/) by @newton_cheng, @keenlooks, @tabbott3 et al.

*Hands-on-keyboard intrusions* is another labor-bottlenecked area where we are seeing fast progress. This is strongly offense-dominant, and especially concerning for the long tail of low and medium-value targets, which were previously protected by the fact that they were not worth the attackers’ time, despite weak defenses. Or more specifically, they may have been worth the attackers' time (their value was higher than their exploitation cost), but attackers being bottlenecked on labor means that they had to focus on the highest-returns targets.

Cheap and capable AI agents can be [embedded in malware](https://red.anthropic.com/2026/attack-navigator/) to increase its deployment reach, for instance by dealing with the diversity of user configurations (something [starting to be observed in the wild](https://cloud.google.com/blog/topics/threat-intelligence/ai-vulnerability-exploitation-initial-access/)). The end product won't look like this proof of concept: it will look like the malware *being* a competent and patient operator. This can make **propagation extremely effective** (via e.g. tailored emails or messages sent to the address book), and **increase the value of targets** (compared to traditional mass-scale malware) via, generally speaking, targeted attacks. [LLMs unlock new paths to monetizing exploits](https://arxiv.org/abs/2505.11449) (by @edoardo_debe, @Chris_Choquette, @daphneipp, @florian_tramer et al) has a list, but I trust cybercriminals to come up with many inventive new ways to make money off victims, in addition to upgrading the existing ones.

I think that AI intrusions will be a big deal, and perhaps increase the (already substantial) cost of cybercrime on vulnerable people by 1+ OOM, despite defenses. While offline exploit development will eventually be mitigated by vulnerabilities drying up, AI intrusions may remain a big problem for a long time.

In the interest of getting this post out, I'm leaving defenses (the ones that exist, and don't yet exist) for future blog posts.

As always, I'm very interested in feedback. See my [contact info](https://tchauvin.com/contact).

*Thanks to JS Denain, Alexander Barry, and Anson Ho for reviewing an earlier version of this post.*

<hr>
*Footnotes*

[^correlation]: In the sparse sampling model, it's interesting to think about how correlated the attacker and the defender are. It's hard to say for humans in general. For fuzzing, they are highly correlated *provided the same fuzzing harnesses*, but attackers focus on parts of codebases that are known not to be fuzzed (e.g. OSS-Fuzz gives reports on this, and for closed-source codebases, a given part of a binary is overwhelmingly likely not to be fuzzed) in order to anti-correlate their results. Intuitively, attackers and defenders reviewing the same codebase with the same AI model would probably be quite correlated, which is good for defense.

[^exceptions]: I think exceptions are mostly UAF, race conditions (James Kettle in the [Smashing the state machine](https://portswigger.net/research/smashing-the-state-machine) blog post: *"in my experience it's extremely challenging to identify race conditions through pure code analysis"*), and crypto protocol weaknesses. The last one is the one where I'm most unsure about how far the ceiling is.

New blog post: "Vulnerabilities and exploits: where are we headed?" @chrisrohlf @joshua_saxe @matrosov @moyix @S1r1u5_ @lcamtuf (enjoyed the podcast this morning!) @tqbf @dadrian @vxunderground @HarmonyIntel @nicowaisman @stanislavfort @CalebWithersDC @oegerikus @albinowax (cited, forgot to tag) @girishsastry @dcuthbert @daveaitel @vijaybolina @neverbolt @clintgibler @ortegaalfredo
@embeddedsec @nikhilmulani @WHITEHACKSEC @dmolnar @JasonDClinton @rez0__ @adversariel

@matrosov Any chance the ledger could be updated? It still shows 27 fixed vulnerabilities, presumably the same number as on May 22

@datagenproc @rohinmshah @AlexBarry4 why are open models likely more overfit to benchmarks?

a few questions about the Opus 4.8 system card @keenlooks @newton_cheng

1. ExploitBench v2?? Or is it "ExploitBench plot v2"? (fig 3.3.1.A)

2. Firefox exploits: what's a crash category? I would have assumed one crash location, but given the mention "part of the task is triage ... determine which proof of concepts yield a usable corruption primitive" i don't know

3. Firefox exploits: do you also have results with "no top 2" as in the Mythos system card, and/or only accepting exploits that do use something from the given crash category?

@matrosov timeline wrong in e.g. https://red.anthropic.com/2026/cvd/findings/ANT-2026-H5T8XKWR? a few others are like that e.g. https://red.anthropic.com/2026/cvd/findings/ANT-2026-9VJ9JJXQ, https://red.anthropic.com/2026/cvd/findings/ANT-2026-DJBBBBPE

I've been thinking more about this idea lately: an "auto mode for scams". I think the need is very real, both in vulnerable people (who won't pay for solutions and are therefore ignored by the market) and companies. Probably what I would do if I started a company today.

I expected deep fake phishing & scamming to come sooner than vulnerability detection, but I was (fortunately) wrong.

You mention "same capability", "Mythos-like hacking", but there's no direct comparison to Mythos. Is this just based on the observation that GPT-5.5 is much better than non-Mythos models?

I'll be at ICLR 2026 (Rio de Janeiro) next week, let me know if you'd like to say hi!

@TomDAAVID 👍

Your points are valid about the prominent claim of "can detect previously known vulns". But the important result from the blog post is that cheap models can detect vulns without an overwhelming number of true positives. And for this, focusing on only a few CWEs is fine.

I didn't like the first blog post because it ignored the issue of false positives. But after reading this one, I now believe that there's something real and interesting here. The important takeaway is that they were apparently able to detect real bugs and possibly vulns without drawning in false positives.

Some people are commenting that the few-shot examples specifically prompt for the kinds of vulnerabilities we are looking for. It is indeed an issue for the blog post's prominent claims that the models can find some known vulnerabilities. But it's not an issue for the other result (more important and less emphasized) that false positives can be manageable. It's fine to only focus on a subset of vulnerabilities: that's also what fuzzers, SAST, etc do!

There aren't that many important CWEs for C/C++ code, so I think it's fine to focus on those only. This would be bad if the goal was to catch all vulns, but the goal is to catch true vulns without an overwhelming number of false positives. I think the blog post does a much better job demonstrating this than the previous one, and I now believe there is something real and interesting here.

I switched to coding on a server I own (capable laptop) and highly recommend it:

1) coding agents can run 24/7, always connected to the Internet. perfect for travel
2) mitigation against supply chain attacks and the risks of the occasional --yolo or --dangerously-skip-permissions run
3) initial setup is easy with coding agents...
4) ... including good security defaults (firewall, tunneling ssh through wireguard so the server doesn't have any open port on the Internet...)
5) requiring Internet access to code would have been a downside in the past but isn't one anymore

I asked claude to log everything we did as we were going through the setup, and to create an ansible playbook out of it at the end. So future installs on new machines should be much faster

@caleb_parikh @jankulveit @AnthropicAI @Aisle_Inc exactly

The biggest flaw in the AISLE blogpost is that vulnerability detection with LLMs is all about false positives. They should have reported on applying the same technique on all snippets of FreeBSD and how much human time was needed to dismiss the at least thousands of false positives.

(the blog post's section on false positives studies *one* cherry-picked, synthetic, very out-of-distribution example, to support the false conclusion of "inverse scaling"...)

@seanhn "frontier labs say" ≠ Carlini improvises, in response to a question, in a handwavy way. I don't think even he actually endorses "very bad until formal verification".

Things will be bad, yes. But formal verification and Rust rewrites are not necessary for things to get better

@TheNormanMu When the website is up new models will be added automatically, but this isn't done yet and we started tracking in January, so we don't have data on the 4.6 models yet, sorry

Nicer-to-read, nicer-to-share blog post equivalent of Sunday's thread is available on my blog and substack:

@mnshah Great dashboard!

Hmm I'd say:
- reduce the attack surface: uninstall unnecessary browser extensions, IDE extensions, etc. Minimize interactions with shady parts of the Internet
- password manager
- keep software more up-to-date than usual
- MFA I guess?

I think it's a good policy to act as if everything you write down on a computer will be leaked some day. (I already had that policy before)

But if you're not the type of person who believes phishing emails, and don't have many Internet-exposed servers, you're already in a good place I'd say

I was planning to write a blog post titled "Stay away from the Internet in 2026". The period of turmoil will be this year, perhaps next year too. Then it'll get better I think.

If you'd like to talk about change detection in LLMs (or AI and cybersecurity, which I'm re-pivoting into next), I'll have a poster on the Logprob Tracking paper at ICLR in a month, and I'll also be at SatML this week!

@xeophon No, we were accessing all models through OpenRouter so I think they wouldn't even have been able to. Maybe people using their API directly did

@xeophon I'm not making it up https://docs.together.ai/docs/changelog

As part of building Border Input tracking, we found an unexpected result that we're not sure how to explain and may have implications for people using T=0. This plot shows the number of requests necessary to find a Border Input (i.e. sampling each input 3 times, an input that doesn't always give the same token in first position). As T -> 0, the curves are approaching a limit... But at T=0 the curve is different from that limit. At least for border inputs, T = 0 and T ≃ 0 behave differently for many providers.

This plot covers 93 API endpoints representing 64 unique models and 38 providers, so it's legit.

New research: cheaply detecting changes in LLM APIs.

We published two papers on the topic:
- Log Probability Tracking of LLM APIs (ICLR 2026)
- Token-Efficient Change Detection in LLM APIs

Both papers request a single token of output from APIs, enabling unprecedently cheap monitoring.

Log Probability Tracking of LLM APIs (https://arxiv.org/abs/2512.03816) shows that when API providers support returning logprobs, these can be used to monitor for changes at extremely low cost, e.g. $0.14/year for hourly sampling of GPT-4.1. It turns out that logprobs are non-deterministic due to non-determinism of inference on GPUs, but we can still basically track logprob averages and this works great. This is also very sensitive to small changes.

Token-Efficient Change Detection in LLM APIs (https://arxiv.org/abs/2602.11083) extends this to the setting where logprobs are not available (pure black-box). The idea is to identify Border Inputs, for which sampling at T=0 doesn't always give the same output (again, looking only at the first token of output). It turns out they can easily be found just from black-box sampling, trying thousands of short inputs and keeping the border inputs. Then, being at T=0 makes any change in the model likely to move this border and result in a notably different output distribution when we sample each border input a few times. This is a bit more expensive than Logprob Tracking, but applies to basically any black-box LLM API, and still outperforms existing methods by wide margins, as shown in this Pareto plot (where LT stands for Logprob Tracking)

Since these methods are cheap, we actually ran them, and found multiple instances of undisclosed changes in LLM APIs. Logprob Tracking (each point is an identified change with high confidence):

Some example time series from Logprob Tracking:

We also found instances of undisclosed changes with Border Input tracking.

Actually, one of them was disclosed: did you know that if you had something running on "Mistral-7B-Instruct-v0.3" from @togethercompute , they silently (though with a public announcement) redirected it to the entirely different Ministral-3-14B-Instruct-2512 in January?

LLM API stability is important for the reliability of downstream applications, for the reproducibility of research and for initial audits to remain relevant.

Small changes are fine but they should be disclosed, so that end users can verify that their use cases weren't affected.

LLM API providers could support a more secure and transparent ML supply chain by:
- supporting output logprobs (especially for open-weight models and non-frontier models, where model stealing isn't a threat)
- always disclosing changes to models or infra.

We are still monitoring a good number of APIs, and a website is in progress to share live results, stay tuned...

Some related work:
- DiFR: Inference Verification Despite Nondeterminism by @a_karvonen, Dan Reuter, @RoyRinberg, Luke Marks, @AdriGarriga, @keriwarr
- Real Money, Fake Models: Deceptive Model Claims in Shadow APIs by Yage Zhang et al
- Defeating nondeterminism in LLM inference by @cHHillee et al
- Model Equality Testing: Which Model Is This API Serving? by @irena_gao, @percyliang and Carlos Guestrin

Extremely cheap methods for continuous change detection in LLM APIs; dozens of undisclosed changes identified @pingToven @zhuohan123 @QGallouedec @Thom_Wolf @openrouter @vllm_project @cHHillee @xeophon @PandaAshwinee

I'm interested in UIs + models that have minimal latency, please share suggestions!

The latency of LLMs is annoying when you use them often.

I tested a few LLMs, plus google search, on a simple question: "how do i reverse a string in python?". Results, from fastest to slowest:
- Google search, featured snippet: 1.5s
- the default Mistral model on http://chat.mistral.ai, logged out: 4.4s. Congratulations to the Mistral team!
- Google search + stack overflow: 5.2s
- Gemini 3 Fast on http://gemini.google.com (Pro plan): 6s
- Claude Haiku 4.5 on http://claude.ai (Pro plan): 10.4s
- Claude Opus 4.5 on http://claude.ai (Pro plan): 12.3s
- Gemini 3 Pro on http://gemini.google.com (Pro plan): 14.3s

Claude models on http://claude.ai take a hit from an annoying bug where they reload the page after the initial load, making me lose the cursor and forcing me to use the mouse, as you can see in the video. @AnthropicAI @claudeai please fix

Link to the video on youtube: https://www.youtube.com/watch?v=ftRZMIlATac