Let an agent hunt flaky tests overnight in a broken CI pipeline

An engineering manager posting on Hacker News as SeanAnderson oversees teams on legacy systems more than ten years old. One team's main branch passed CI only about 40% of the time because of flaky tests, and a full run took around 40 minutes. He let Claude read the CI artifacts and asked it to write tooling that tracks a rolling 30 day pass rate. It picks the worst tests, guesses whether the fault is in the test or the product, and narrows to a minimal repro. When it cannot reproduce a bug, it tries a fix and reruns tests overnight to judge it statistically. In a few weeks the pipeline got 20 minutes faster and passed 35% more often. He discards some bad fixes.