{"rewrite":{"id":"r_dcdf38aa2cdd9cf41e4bb971","clusterId":"c_64bee7ce9eb2626d0c8d57e4","slug":"ucsd-lab-s-jetspec-method-speeds-up-ai-by-up-to-9-64x","model":"deepseek-v4-flash","headline":"UCSD Lab's JetSpec Method Speeds Up AI by Up to 9.64x","summary":"Hao AI Lab at UC San Diego has developed JetSpec, a speculative decoding method that accelerates large language models. On the MATH-500 benchmark, JetSpec achieved a 9.64x speedup over standard inference when running Qwen3-8B on an NVIDIA H100. The lab released draft models, code, and a paper.","whyItMatters":"JetSpec claims to solve waste problems in both autoregressive and block-diffusion speculative decoding, reaching over 1000 tokens per second on an NVIDIA B200 with Qwen3-8B.","webCardHtml":"\u003cp\u003eHao AI Lab at UC San Diego published JetSpec, a speculative decoding method that accelerates large language model inference. The method uses parallel tree drafting to avoid waste in both autoregressive and block-diffusion approaches. On the MATH-500 math reasoning benchmark, JetSpec achieved a 9.64x speedup over standard inference when running Qwen3-8B on an NVIDIA H100, and a 4.58x speedup on the MT-Bench chat benchmark. The lab also ran Qwen3-8B on an NVIDIA B200 using a modified version of the vLLM inference engine, reaching over 1000 tokens per second.\u003c/p\u003e\u003cp\u003eHao AI Lab released JetSpec draft models for Qwen3-8B, Qwen3 30B A3B, Qwen3.6 35B A3B, gpt-oss-20b, Gemma 4 26B A4B IT, and Step 3.7 Flash. The paper and code are available on arXiv and GitHub.\u003c/p\u003e","blueskyPost":"JetSpec's 9.64x speedup on MATH-500 with Qwen3-8B suggests speculative decoding can make smaller models competitive with larger ones on math reasoning, narrowing the hardware gap for research labs.","twitterPost":"JetSpec's 9.64x speedup on MATH-500 shows speculative decoding can shrink the hardware needed for math reasoning.","threadsPost":"JetSpec hits 9.64x faster inference on MATH-500 with Qwen3-8B on an H100. The implication: smaller models with speculative decoding may match larger ones on math tasks, lowering the compute barrier for research labs. Hao AI Lab released the code and draft models, so the method is immediately testable.","newsletterBlurb":"Hao AI Lab at UC San Diego developed JetSpec, a speculative decoding method that uses parallel tree drafting to accelerate large language models. On the MATH-500 benchmark, it achieved a 9.64x speedup over standard inference. The lab released draft models for six model families and published the paper and code.","attributionJson":"[{\"source\":\"GIGAZINE\",\"url\":\"https://gigazine.net/news/20260626-jetspec-speedup-ai/\",\"title\":\"Speculative Decoding Method 'JetSpec' Developed That Speeds Up AI by Up to 9.64x\"}]","lintFlagsJson":null,"lintHits":0,"costUsd":0,"inputTokens":4325,"outputTokens":651,"status":"published","repairAttempts":0,"nextRepairAt":null,"factsAttemptedAt":1782600514,"createdAt":"2026-06-27T22:37:41.000Z","publishedAt":"2026-06-27T22:40:24.000Z","updatedAt":"2026-06-27T22:40:24.000Z"},"cluster":{"id":"c_64bee7ce9eb2626d0c8d57e4","canonicalTitle":"AIを最大9.64倍高速化する投機的デコーディング手法「JetSpec」が開発される","representativeArticleId":"a_25f1721a55667dbb61abe085","sourceCount":1,"writtenSourceCount":1,"writeAttempts":0,"isSolo":true,"entitiesJson":"{\"anime_titles\":[],\"manga_titles\":[],\"work_titles\":[],\"studios\":[],\"people\":[],\"type\":\"news\",\"domain\":\"other\",\"is_roundup\":false}","contentType":"news","status":"published","firstSeenAt":"2026-06-26T03:40:00.000Z","lastSeenAt":"2026-06-26T03:40:00.000Z","updatedAt":"2026-06-27T22:40:25.000Z"},"attribution":[{"source":"GIGAZINE","url":"https://gigazine.net/news/20260626-jetspec-speedup-ai/","title":"AIを最大9.64倍高速化する投機的デコーディング手法「JetSpec」が開発される"}],"entities":{"anime_titles":[],"manga_titles":[],"work_titles":[],"studios":[],"people":[],"type":"news","domain":"other","is_roundup":false},"keyFacts":["JetSpec achieved a 9.64x speedup over standard inference on the MATH-500 benchmark when running Qwen3-8B on an NVIDIA H100.","The method reached over 1000 tokens per second on an NVIDIA B200 with Qwen3-8B using a modified vLLM inference engine.","Hao AI Lab released JetSpec draft models for six architectures: Qwen3-8B, Qwen3 30B A3B, Qwen3.6 35B A3B, gpt-oss-20b, Gemma 4 26B A4B IT, and Step 3.7 Flash.","JetSpec uses parallel tree drafting to address waste in both autoregressive and block-diffusion speculative decoding.","The paper and code are available on arXiv and GitHub."]}