README Rot Census
We ran the install steps of 70 real GitHub CLI repos on a clean machine and sorted every failure by hand.
15%of older, starred CLI repos fail their own README (7 of 48)
0 / 6maintained popular repos failed
7 / 7genuine breaks with a fix verified in a second clean sandbox
Samples
| Sample | Repos | Install steps found | No steps | Genuine breaks |
|---|---|---|---|---|
| Maintained, popular CLI repos | 10 | 6 | 4 | 0 |
| Older but starred CLI repos, batch 1 | 30 | 24 | 6 | 5 |
| Older but starred CLI repos, batch 2 | 30 | 24 | 6 | 2 |
Every genuine break and its verified fix
| Repo | What broke | Fix | Cost |
|---|---|---|---|
| donnemartin/gitsome | setuptools use_2to3 removed; pip install fails on Python 3.12+ | verified, 2 rounds | $0.043 |
| archivy/archivy | pkg_resources removed from setuptools; app crashes on import | verified, 2 rounds | $0.02 |
| freedmand/semantra-python | same pkg_resources removal | verified, 2 rounds | $0.0145 |
| dronesploit/dronesploit | --version unsupported; entry point fails to import | verified, 3 rounds | $0.028 |
| kefranabg/readme-md-generator | npx command is `readme`, not the package name | verified, 2 rounds | $0.032 |
| SpiderLabs/HostHunter | README needs Python 3.10, requirements.txt path | verified, 2 rounds (untuned) | $0.021 |
| texturedesign/texturize | install path misses conda/torch prerequisites | verified, 2 rounds (untuned) | $0.0281 |
How to read this
- Rates apply to older, starred CLI repos only. Maintained popular repos: 0 breaks in 6 evaluated.
- 5 of the 7 fixes were found on repos the loop was tuned on; the 2 fresh ones are the untuned result.
- A sandbox environment limit (no sound card) was NOT counted as a README break.
- Nemotron Nano extracts the install path from each README (commands must appear verbatim); a fresh Vercel Sandbox (Ubuntu, Python 3.14, Node 24) runs it. Every flagged break was sorted by hand.
- Evaluated 54 repos with an extractable install path, 7 broke.