The Daily Hallucination

SAN FRANCISCO — Engineers at issue-tracking startup Pinion spent most of the year rebuilding the company's continuous-integration pipeline after coding agents quadrupled the test suite, an effort that has reduced the time an agent waits to learn its pull request is broken from six minutes to five.

The project began when the chief technology officer assigned an engineer a ticket titled "CI costs are high," and, while he was at it, asked him to make it faster.

"Our agents open a pull request every 40 seconds," said Dario Fennimore, the engineer who received the ticket. "Each one waited six minutes to find out it broke the same test. Now it waits five. That is a minute the agent spends opening another pull request."

The company now runs 4,100 test jobs a day, 61 percent of which it attributes to agents re-running a suite that already passed "to check." Moving to faster machines cut job times by a third; a native compiler cut type-checking by 73 percent, which moved the bottleneck to the part of the pipeline that waits for the type-checker to be moved.

"Nobody on the team wrote most of the tests," Fennimore said. "The agent wrote them. Then the agent wrote code that failed them. Then it asked us why. We are the only people in the building who have not read the tests, and we are the ones on call."

A platform engineer said morale had improved. "The tests fail 34 percent quicker," she said. "You can feel it."

The CTO said the work was a success and that the company now ships more code than at any point in its history, most of it to fix the other code.

At press time, the CTO had filed a new ticket titled "CI is fast now, why is nothing shipping," and assigned it to the agent.