Tailscale has resolved a serious stability problem in its infrastructure linked to a historical vulnerability in the database engine SQLite. A race condition bug in the Write-Ahead Logging (WAL) mechanism, which remained hidden in the source code for over 16 years, caused 19 distinct episodes of data corruption on the company’s servers over a six-month period.
The company’s coordination server architecture manages information by dividing customers into isolated shards. Each shard runs a single language process Go which accesses a local database exclusively. Since 2022 this configuration has ensured operational readiness, but the implementation of customized routines for the backup to S3 exposed anomalous storage system behavior.
In diagnostic analyzes conducted in the field, automatic verification through instruction PRAGMA integrity_check began to detect corrupted indices and structural inconsistencies. The exposure to the flaw occurred because the company chose to take direct control of file synchronization by performing a WAL file checkpoint much more frequently than the default settings.
The hunt for the race condition and the official fix, Tailscale resolves the 16-year-old flaw
To find the root of the problem, the engineers signed a support contract with the database creators and developed a custom tracing tool in the VFS (Virtual File System) layer called tmstmpvfs shim. All details on the forensic analysis and technical logs can be found in the official report published by Tailscale.
The traces have shown that the temporal overlap between a write transaction and the automatic log reset cheated the checkpoint process. The database engine considered some pages transferred to the primary file which in reality had not yet been copied, causing the irreversible loss of the updated blocks and generating discrepancies in the indices associates.
The first fix attempt distributed with the release SQLite 3.52.0 generated false integrity alerts due to a change in the rounding of floating point values present in virtual columns. The database team promptly retracted the release, publishing SQLite 3.51.3 to isolate only the race condition change in the WAL. Two months after the rollout of the patch, production telemetry intercepted and blocked the collision without any data loss, confirming the definitive resolution of the anomaly.
