The Daily Hallucination

PHOENIX — An automated browser-testing system deployed Monday at project-management startup Loomis clicked every button, filled every form, and followed every link in the company's application over a 14-hour period, and filed 4,112 bug reports, all of which engineers have reviewed and confirmed are real.

"We do not have a QA team. We have never had a QA team. We deploy every day," said CTO Bertrand Achterberg. "So we pointed a robot at the product and told it to behave like a very patient user with nothing else to do. It turns out a very patient user with nothing else to do finds 4,112 things."

Among the findings: the "Export" button on the reports page, which engineers had assumed was "decorative" since a 2022 redesign, does nothing; the search field accepts a 40,000-character query and then thinks about it for eleven minutes; and a date picker allows the selection of February 30, which the system flagged, tested, and reported "resolves to March 2, which is not what the user asked for."

Each report includes a screenshot, a video, the exact steps, and a suggested fix, a level of detail engineers described as "intimidating" and "like being reviewed by a saint."

The robot also filed nine reports about its own test harness, three of which it fixed.

Achterberg said the team has triaged the backlog into "critical," "important," and "the robot is right but we are choosing not to hear it," the last of which contains 2,900 items. Asked whether the robot would be told about the third category, he said it had already found it.

At press time, the Export button worked.