{"rewrite":{"id":"r_63d2890b92475dd8a5b206ae","clusterId":"c_ccd8fdd6dbaa4decd6c2d253","slug":"parsimony-benchmark-tracks-how-much-code-ai-models-add","model":"deepseek-v4-1-flash","headline":"Parsimony Benchmark Tracks How Much Code AI Models Add","summary":"An open-source benchmark called Parsimony measures the code AI adds minus the code it deletes when it modifies a program. It analyzes existing benchmark modification data using a unit that counts syntactic elements rather than lines, so line breaks and comments do not count. In its published comparison, Claude Opus 5.5(high) adds about 1.1 times as much code as other models, while GPT-6.1 Sol(high)/Codex adds about 0.7 times.","whyItMatters":"Problem-solving scores and cost have carried the comparison between coding models, and Parsimony makes code growth a comparable number, though its developer says code volume alone cannot measure readability or design quality.","webCardHtml":"\u003cp\u003eThe developer says the benchmark was built to measure bloat, since code that solves a problem can still add unnecessary processing that makes review and long-term maintenance harder. Test code, documentation, and auto-generated files are excluded from the count. Because deletions subtract from additions, a rewrite that simplifies existing processing can score a negative value, such as replacing a summing loop with Python\u0026#39;s standard \u0026#34;sum\u0026#34; function, which the developer cites as a net reduction of 10 units.\u003c/p\u003e\u003cp\u003eThe source code and measurement data are on GitHub and the results can be reproduced from that data. The developer is asking for modification data from new models and for opinions on evaluation methods.\u003c/p\u003e","blueskyPost":"Parsimony, an open-source benchmark, measures how much code an AI adds minus what it deletes. Its comparison shows models with similar intelligence scores differ widely in added code. Its developer says code volume alone cannot judge readability or design quality.","twitterPost":"Parsimony measures code an AI adds minus code it deletes. In its comparison, Claude Opus 5.5(high) adds about 1.1 times as much code as other models; GPT-6.1 Sol(high)/Codex about 0.7. Its developer says code volume alone cannot judge readability.","threadsPost":null,"newsletterBlurb":"An open-source benchmark called Parsimony measures how much code an AI model adds against what it deletes. It counts syntactic elements rather than lines, and excludes test code, documentation, and generated files. In its published comparison, higher intelligence scores did not mean less added code, though the developer says volume alone cannot measure readability or design quality.","attributionJson":"[{\"source\":\"GIGAZINE\",\"url\":\"https://gigazine.net/news/20261020-parsimony/\",\"title\":\"Benchmark \\\"Parsimony\\\" compares AI models, revealing large differences in code growth even when solving the same problem\"}]","lintFlagsJson":null,"lintHits":0,"costUsd":0,"inputTokens":4636,"outputTokens":595,"status":"published","repairAttempts":0,"nextRepairAt":null,"factsAttemptedAt":1792508833,"createdAt":"2026-10-20T15:02:34.000Z","publishedAt":"2026-10-20T15:06:07.000Z","updatedAt":"2026-10-20T15:06:07.000Z"},"cluster":{"id":"c_ccd8fdd6dbaa4decd6c2d253","canonicalTitle":"同じ問題を解決してもAIによってコードの増加量に大きな差、AIモデルを比較するベンチマーク「Parsimony」が登場","representativeArticleId":"a_c963c79bf25c4bafffdaa4cb","sourceCount":1,"writtenSourceCount":1,"writeAttempts":0,"isSolo":true,"entitiesJson":"{\"anime_titles\":[],\"manga_titles\":[],\"work_titles\":[\"Parsimony\"],\"studios\":[],\"people\":[],\"type\":\"news\",\"domain\":\"other\",\"is_roundup\":false}","contentType":"news","status":"published","firstSeenAt":"2026-10-20T14:00:00.000Z","lastSeenAt":"2026-10-20T14:00:00.000Z","updatedAt":"2026-10-20T15:06:08.000Z"},"attribution":[{"source":"GIGAZINE","url":"https://gigazine.net/news/20261020-parsimony/","title":"同じ問題を解決してもAIによってコードの増加量に大きな差、AIモデルを比較するベンチマーク「Parsimony」が登場"}],"entities":{"anime_titles":[],"manga_titles":[],"work_titles":["Parsimony"],"studios":[],"people":[],"type":"news","domain":"other","is_roundup":false},"keyFacts":null}