Skip to content

PlaybookAgentic OSPart 3 of 7

Most common agent errors

We’ve helped 100+ eng orgs reduce agent errors and token spend. We compiled a list of what they were doing wrong.

Wrong and missing context is extremely costly to agents, and extremely easy to produce. Here are some examples of common omissions and errors we’ve seen across the work of 2000+ engineers:

  • The drift detection tool drifts. The most common omission is not making your CI checks (or other drift detection tools) self-improving. They wind up carrying outdated context and can potentially cause regressions elsewhere.
  • Instruction-improvement workflows don’t pass deployment gates, blocking the improvement PRs and defeating the purpose of these automations.
  • Nothing compares the live GitHub ruleset to the checked-in file. Even if you write governance as a file, sometimes that file is not what’s actually enforced. Until you’ve diffed it against what the platform is really doing, your file is a proposal.
  • Documentation checks are not required. If a check is only a flag, your team will get used to ignoring or skipping it in a week. We recommend a small Markdown file in the repo that you run with an agent on every push (a script can’t help check prose). For each file the change touches, it should check whether it falsifies a concrete claim in your documentation.
  • Relying on running a skill. People will forget. Automate it.

These are the principles we suggest to avoid producing wrong context or letting context go stale:

  1. One command per lifecycle step. Install, verify, run and deploy are each documented, idempotent commands that are loud on failure.
  2. No dependencies outside your repo. If it lives in a dashboard/wiki, an agent cannot see it.
    • E.g., branch protection is .github/rulesets/main.json instead of a settings page.
    • The most common failure comes from setups that depend on one person’s laptop: scheduled workflows that are locally configured, or MCP servers connected only in someone’s client. Search everything you’ve written for your agents for file paths that start in somebody’s home folder.
  3. Single source of truth. Keep a single canonical instruction set, and have every other agent get copies or symlinks with enforced sync.
    • 9/10 times, not having a copy is better than syncing one.
      • A notable exception: our CLAUDE.md → AGENTS.md copy is hook-synced and CI-gated. Skills are symlinked into .agents/skills/. We do this instead of a pointer, because otherwise we’d have to hope every session follows conventions that are not baked into the agent (model- and harness-specific optimization can change this principle).
  4. Instructions are code. Context files should be versioned (depending on your memory approach), reviewed in PRs, and pruned. And when they’re wrong, we call it a bug. Every error here is costly, because every session will spend tokens finding out it’s running the wrong instructions, and risk improvising another approach.