It got me thinking: how do you actually optimize for agent discovery? With humans you can do SEO, copywriting, word of mouth. But an agent just looks at available tools in context and picks one based on the description, schema, examples.
Has anyone experimented with this? Does better documentation measurably increase how often agents call your tool? Does the wording of your tool description matter across different models (ZLM vs Claude vs Gemini)?
The short answer: yes, description wording measurably matters, and it's more sensitive than most people expect.
A few things we've found that move the needle:
1. Concrete use-case beats generic label. 'Send a transactional email after payment confirmation' outperforms 'Send email' by a wide margin across models — the agent doesn't have to infer when to use it.
2. Negative constraints help. Explicitly saying 'Don't use this for bulk sends or newsletters' actually reduces hallucinated misuse, because the model has a clear mental model of the boundary.
3. Schema field names are silent prompts. A field named 'recipient_email' gets filled correctly far more than 're' or even 'to'. The model is pattern-matching on names it's seen in similar contexts.
4. Example I/O beats description prose. If you include one example input and what the output looks like, agents are significantly more likely to call the tool correctly on the first attempt.
For cross-model consistency: Claude tends to do better with structured JSON schema + examples. GPT-4 responds well to prose descriptions with analogies. Gemini seems to weight field name patterns heavily.
The broader pattern: treat your tool description like a micro few-shot prompt, not an API doc. Optimize for the model's context, not for human readers.
Two things that surprised us: (1) being explicit about what the tool doesn't do matters as much as what it does - vague descriptions get hallucinated calls constantly, and (2) inline examples in the description beat external documentation every time. The agent won't browse to your docs page.
The schema side matters too - clean parameter names, sensible defaults, clear required vs optional. It's basically UX design for machines rather than humans. Different models do have different calling patterns (Claude is more conservative, will ask before guessing; others just fire and hope) so your descriptions need to work for both styles.
But it's the same points you should follow when designing a human readable docs(as zahlman said above). Isn't it?
That seems... surprising, and if necessary something that could easily be corrected on the harness side.
> The schema side matters too - clean parameter names, sensible defaults, clear required vs optional. It's basically UX design for machines rather than humans.
I don't follow. Wouldn't you do all those things to design for humans anyway?
Practically: the agent reads your docs, README, or API description and decides if it can use your tool to solve the current problem. So the question is really "will an AI understand my tool well enough to use it correctly?"
What helps: - Clear, literal API documentation (not marketing copy) - Explicit input/output examples with edge cases - A `capabilities.md` or similar that describes what the tool does and doesn't do
The irony: the skills that make tools understandable to AI (precision, literalness, examples) are the opposite of what makes them legible to humans (narrative, benefits, stories).
I mean how do i check that my changes in documentation even work in a right way?
Three things that actually moved the needle:
Negative boundaries work better than positive claims. "Generates reports from structured receipts. Does NOT execute code, modify files, or make API calls" gets called correctly way more often than "A powerful report generation tool." Trigger words matter more than you'd think. I maintain explicit trigger lists per skill — specific phrases that should activate it. Without those, the agent pattern-matches on vibes and gets it wrong ~30% of the time. With explicit triggers, that drops to under 5%.
Schema is the real interface. Clean parameter names with sensible defaults beat elaborate descriptions. If your tool takes query: string vs search_query_input_text: string, the first one gets called more reliably across models.
But here's the thing the "agent economy" framing gets wrong: you don't want fully autonomous tool selection. An agent choosing freely between 50 tools is like giving a junior developer admin access to everything — it'll work sometimes and break spectacularly other times. What works better is constraining the agent's scope upfront. Give it 3-5 relevant skills for the task, not your entire toolkit. Or build workflow skills that chain multiple tools in a fixed sequence — the agent handles the content, the workflow handles the routing.
The uncomfortable truth: you're not optimizing for "discovery" in the human sense. There's no brand loyalty, no trust built over time. Every single invocation is a cold start where the model reads your description and decides. That's actually freeing — it means the best-described tool wins, regardless of who built it.
Two things I do to fight this:
First, skill scoping per task. Instead of exposing all 20+ skills to every agent, each terminal only sees the 3-5 skills relevant to its current dispatch. The orchestrator decides which skills to load before the agent even starts. Less noise, better selection accuracy.
Second, context rotation to prevent context rot setting in. When an agent's context fills up, the system automatically writes a structured handover, clears the window, and resumes in a fresh context. This is critical because a degraded context doesn't just pick worse tools — it starts ignoring instructions entirely. A fresh context with a good handover outperforms a bloated one every time.
I'm actually testing automatic refresh at 60-70% usage right now — not waiting until the window is nearly full, but rotating early to prevent context rot before it starts. Early results suggest that's the sweet spot: late enough that you've done meaningful work, early enough that the handover quality is still high.
The organic discovery problem you're describing is essentially unsolvable with a flat tool list. The more tools you add, the worse selection gets — it's not linear degradation, it's closer to exponential once you pass ~15-20 tools in context. The only path I've found is hierarchical: a routing layer that narrows the set before the agent sees it.
I wrote this post because of exactly those corner cases. If I'm building something agents would use - how do i understand which tool they'd actually choose?
For example you building an API provider for image generation. There are thousands of them in the internet.
I wonder if there are a tool that basically would simulate choosing between your product/service and your competitors one.
I don't know if there a correlation between what LLM would choose now and how you product should look to most likely be in LLM data set.
In that YC video i mentioned in post body they discuss tool called ReSend - something like an email gateway for receiving/sending mails. What's interesting - there are a lot of tools like that, but LLM's would every time choose shiny new resend.
Seems like there are something more than just being in the internet for a long time :)
I hope it doesn’t stick.
The tool i’ve didn’t see - “custdevs for agents”. So we can simulate choosing process for them in thousands of different scenarios. And then compare how tasty product looks for Claude or Gemini or any other LLM
Correct me if i’m wrong :)