Why this score Independent ML researcher (Ai2) and author of Interconnects; technically expert, transparent about methods and corrections, but a solo analyst publishing opinionated analysis without institutional editing.
Open models' demise from this seems greatly exaggerated, but I do think the previous index was flattering the open weight models too strongly.
A judgement about a measure, not yet checkable. On the rebuilt index the best released open model fell from 46 (twelve behind) to 32 (26 behind), per Artificial Analysis.
In reality, this was much more likely a safety tune of Muse Spark 1.2, which would only be a 1 point drop on Artificial Analysis to shore up safety behaviors.
Still unverified. Meta's own model card says only that the open version "has additional safety post-training" for use without system guardrails; it names no Muse Spark version it was built from.
The most likely (by far) outcome is for the status quo to continue and for the best open models to lag the best closed models by 6-9months.
Still open — resolves at end-2026. Lambert himself put the open-to-closed gap at 2-5 months on 21 September, keeping 6-9 months only for American open models. On the Artificial Analysis index the best open model trails the leader by 12 points, 46 to 58. Recheck at resolution.
It's a very good thing for the open community.
Independently corroborated: MIT Technology Review reported the same week that Chinese open models had overtaken Llama in popularity, and subsequent UK AISI and Artificial Analysis measurements consistently placed Chinese models atop the open-weight rankings through mid-2026.
Llama is no longer the open standard.
Borne out within a year: by August 2025 MIT Technology Review reported Chinese open models (DeepSeek, Kimi, Qwen) had become more popular than Llama, and by mid-2026 the independent record — UK AISI's evaluations and the Artificial Analysis index — showed every leading open-weight model was Chinese until Thinking Machines' July 2026 entry.