{"rewrite":{"id":"r_fb442c5333ac6b3b4c28d4c8","clusterId":"c_d9cbf4e6bde129eedb227ddb","slug":"opper-releases-jevman-a-pac-man-benchmark-for-decision-models","model":"deepseek-v4-1-flash","headline":"Opper Releases Jevman, A Pac-Man Benchmark For Decision Models","summary":"Opper published jevman, an open-source benchmark that has AI models play Pac-Man to compare decision-making. Six models, including Jev, Kev, Laya, Clef, and GPT-6 Luna Decisions, each play 100 games against standard-rule ghosts. Models receive maze, dot, and ghost data as JSON and return up, down, left, or right as probabilities within a 2-second limit per junction. Ranking uses the 100-game average score, with a 95 percent confidence margin, so models inside that margin are treated as tied.","whyItMatters":"By scoring decisions per junction under a hard time limit rather than grading generated text, Opper's benchmark gives probabilistic decision models a shared, measurable task where response speed is recorded as its own result alongside accuracy.","webCardHtml":"\u003cp\u003eJev is a probability model from TypeSafe AI that outputs a likelihood for each preset option instead of generating prose, which makes it suited to decisions that need to be fast. Jevman turns that property into a game: at every junction, the model gets the maze, dot placement, and ghost positions as JSON and answers with probabilities for up, down, left, or right. The game moves according to those probabilities, and any junction where the model fails to answer within 2 seconds is handled by a simple alternative algorithm. That count is recorded too, so speed is scored, not assumed.\u003c/p\u003e","blueskyPost":"Opper's jevman scores each model on 100-game averages with a 95 percent confidence margin, so close finishes become ties. A model can win on points and still not separate itself from the field.","twitterPost":"jevman's 95 percent confidence margin means GPT-6 Luna Decisions can post the top average and still be ruled tied. Score gaps inside the band do not separate models.","threadsPost":"Opper's jevman caps every decision at 2 seconds per junction, which tests models on speed as much as strategy. A slower, smarter model loses turns it might have won. The 95 percent confidence margin then flattens any lead that stays close.","newsletterBlurb":"Opper published jevman, an open-source Pac-Man benchmark that tests AI decision models at maze junctions. Each model gets the maze and ghost positions as JSON and returns probabilities for up, down, left, or right within 2 seconds; slower answers fall to a simple alternative algorithm whose use is recorded. Six models play 100 games each, ranked by average score with a 95 percent confidence margin.","attributionJson":"[{\"source\":\"GIGAZINE\",\"url\":\"https://gigazine.net/news/20261009-jevman/\",\"title\":\"jevman: A Pac-Man Benchmark Where Fast-Decision Probability Models Like Jev and GPT-6 Luna Decisions Play\"}]","lintFlagsJson":null,"lintHits":0,"costUsd":0,"inputTokens":4643,"outputTokens":636,"status":"published","repairAttempts":0,"nextRepairAt":null,"factsAttemptedAt":1791547630,"createdAt":"2026-10-09T12:01:41.000Z","publishedAt":"2026-10-09T12:06:06.000Z","updatedAt":"2026-10-09T12:06:06.000Z"},"cluster":{"id":"c_d9cbf4e6bde129eedb227ddb","canonicalTitle":"jevやGPT-6 Luna Decisionsなどの素早い判断に特化した確率モデルがパックマンをプレイするベンチマーク「jevman」","representativeArticleId":"a_e9ff38a223e65ef5caf28a80","sourceCount":1,"writtenSourceCount":1,"writeAttempts":0,"isSolo":true,"entitiesJson":"{\"anime_titles\":[],\"manga_titles\":[],\"work_titles\":[\"jevman\",\"Pac-Man\"],\"studios\":[\"Opper\"],\"people\":[],\"type\":\"news\",\"domain\":\"games\",\"is_roundup\":false}","contentType":"news","status":"published","firstSeenAt":"2026-10-09T11:00:00.000Z","lastSeenAt":"2026-10-09T11:00:00.000Z","updatedAt":"2026-10-09T12:06:06.000Z"},"attribution":[{"source":"GIGAZINE","url":"https://gigazine.net/news/20261009-jevman/","title":"jevやGPT-6 Luna Decisionsなどの素早い判断に特化した確率モデルがパックマンをプレイするベンチマーク「jevman」"}],"entities":{"anime_titles":[],"manga_titles":[],"work_titles":["jevman","Pac-Man"],"studios":["Opper"],"people":[],"type":"news","domain":"games","is_roundup":false},"keyFacts":null}