{"rewrite":{"id":"r_af1fb9009fb44f01832eb204","clusterId":"c_02da60dd502ece09bcf1d42a","slug":"tailscale-traces-backup-corruption-to-a-16-year-old-sqlite-bug","model":"deepseek-v4-1-flash","headline":"Tailscale Traces Backup Corruption to a 16-Year-Old SQLite Bug","summary":"Tailscale used SQLite as its primary database starting in 2022. From August 2025, backups upload to S3 began showing corruption, occurring 19 times over six months, each time forcing a control plane process to stop for repair or restore. Recent code changes were ruled out. Tailscale signed a support contract with SQLite developers and built a transaction log pipeline, which helped lead the SQLite team to a bug latent in the database for 16 years.","whyItMatters":"A corruption incident that could not be reproduced from the code or recovered at the time turned into a shared hunt with the database's own developers, and the replay pipeline built for recovery is what narrowed the search.","webCardHtml":"\u003cp\u003eTailscale ran SQLite as its primary database since 2022, with each shard holding a single Go process that accessed the database exclusively. Backups took a snapshot every few minutes and uploaded the whole file to an S3 bucket. That setup held from early 2023 until August 2025, when corruption started showing up in the backups.\u003c/p\u003e\u003cp\u003eCorruption hit 19 times over six months, limited to control plane configuration data but each time requiring the process to stop for repair or restore, taking down the control plane on the affected shard. Code review and the absence of recent related changes left Tailscale without a reproducible trigger, so it contracted SQLite developers for joint verification. A transaction log pipeline that streamed every modifying SQL statement to a log file supported both recovery and the eventual path to the defect.\u003c/p\u003e","blueskyPost":"Tailscale ruled out its own recent code before signing SQLite's developers to a support contract. The bug hid in the database engine for 16 years, which means the fix depended on the vendor, not the app team.","twitterPost":"Tailscale ruled out its own code first, then signed a support contract with SQLite's developers. The 16-year bug was reachable only from inside the engine.","threadsPost":"Tailscale spent months treating the corruption as its own problem, ruling out recent code changes before signing a support contract with SQLite's developers. The bug had sat in the database engine for 16 years. A backup pipeline can be correct and still fail when the layer underneath it is wrong.","newsletterBlurb":"Tailscale has published how it tracked down months of SQLite backup corruption. The corruption appeared in S3 backups 19 times over six months and could not be reproduced from code changes or scrutiny, so Tailscale contracted SQLite developers and built a transaction log pipeline that replayed modifying statements to restore data. That pipeline also helped point to a bug latent in SQLite for 16 years.","attributionJson":"[{\"source\":\"GIGAZINE\",\"url\":\"https://gigazine.net/news/20260920-sqlite-wal-reset-bug/\",\"title\":\"How a 16-Year-Old SQLite Bug Was Tracked Down\"}]","lintFlagsJson":null,"lintHits":0,"costUsd":0,"inputTokens":4570,"outputTokens":633,"status":"published","repairAttempts":0,"nextRepairAt":null,"factsAttemptedAt":1789875868,"createdAt":"2026-09-20T03:40:02.000Z","publishedAt":"2026-09-20T03:43:24.000Z","updatedAt":"2026-09-20T03:43:24.000Z"},"cluster":{"id":"c_02da60dd502ece09bcf1d42a","canonicalTitle":"16年間放置されていたSQLiteのバグを突き止めた方法","representativeArticleId":"a_8aae28c4a0f445707746e4eb","sourceCount":1,"writtenSourceCount":1,"writeAttempts":0,"isSolo":true,"entitiesJson":"{\"anime_titles\":[],\"manga_titles\":[],\"work_titles\":[\"SQLite\"],\"studios\":[\"Tailscale\"],\"people\":[],\"type\":\"news\",\"domain\":\"other\",\"is_roundup\":false}","contentType":"news","status":"published","firstSeenAt":"2026-09-20T03:00:00.000Z","lastSeenAt":"2026-09-20T03:00:00.000Z","updatedAt":"2026-09-20T03:43:20.000Z"},"attribution":[{"source":"GIGAZINE","url":"https://gigazine.net/news/20260920-sqlite-wal-reset-bug/","title":"16年間放置されていたSQLiteのバグを突き止めた方法"}],"entities":{"anime_titles":[],"manga_titles":[],"work_titles":["SQLite"],"studios":["Tailscale"],"people":[],"type":"news","domain":"other","is_roundup":false},"keyFacts":null}