Engineers Spend Three Weeks Choosing Which 6 of 4,000 API Endpoints AI Agent May See; Agent Uses One
The agent called find_customer 11,000 times in its first day and described the experience as "plenty."
DENVER — After a three-week design review, 14 meetings, and a 26-page document titled "Tool Surface: Principles," the platform team at logistics software company Halyard has granted its new AI agent access to six of the 4,000 endpoints in its internal API, a decision the agent has responded to by using one of them.
"We agonized," said Rosalind Achebe, the staff engineer who led the effort. "Read tools versus write tools. Granularity. Blast radius. Whether 'list_shipments' should take a date range or a cursor. We modeled 40 scenarios. Then we turned it on, and it called find_customer about 11,000 times and seemed really happy."
The agent, which assists the support team, has not called list_shipments, get_invoice, update_note, resolve_ticket, or the sixth tool, whose name the team declined to share because "it was a compromise and we are not proud of it."
Internal logs show the agent did once consider calling get_invoice, generating the text "I should check the invoice" before adding "but the customer record probably has what I need," which it did.
"Honestly, it is the most disciplined engineer on the team," said a product manager who asked not to be named because the agent can read Slack. "We gave a human that much API surface once. He wrote a script that ran for six hours and emailed 40,000 people."
The team said its next quarter will be spent designing a seventh tool. Asked whether the agent had requested one, Achebe said it had not, but that it had recently begun ending its responses with "Let me know if there is anything else I can look up," which the team is treating as a hint.
At press time, the agent had called find_customer again.