How to maintain a team of AI agents

The learning loop: five layers

1. Per task. Every bot ends every task with one line: LESSON: what it would do differently. Appended to its own playbook file.
2. Corrections. When you correct a bot, it writes a FEEDBACK entry: what you said verbatim, why, how to apply. These outrank everything else.
3. Friday retro. The learning officer reads all new lesson and feedback lines, dedupes them, and for anything that recurred twice either rewrites the affected skill, teaches a new one, or proposes a new Auto Review rule. You approve skill changes.
4. Monthly skills audit. Every saved skill is re-run on a real task; stale ones are paused.
5. Quarterly drift audit. The reviewer checks live assets (pages, profiles, campaigns, workflows) against the current brand context and opens an incident for each mismatch.

Grok Bot saves skills in plain language and lets you teach one by demonstrating a workflow for up to ten minutes (skills, routines and automations). Skills are shared across bots on a workspace, which means one good skill taught once is everyone’s.

The weekly access audit

Ten minutes every Friday, done by the learning officer and read by you: installed plugins and connectors, sites the Agent Computer is signed into, active routines and whether each is still needed, tokens by name and age. Anything that finished gets signed out, uninstalled, or paused the same day. The platform’s own guidance is the same: sign out of services when access should end, remove temporary files, and delete connectors or revoke authorization when no longer needed (approvals, security and privacy).

Deleting a bot does not delete the shared computer’s files or its browser sessions. Cleanup is a separate step, every time.

Metrics that tell you the truth

• Approval-without-edit rate. Share of packets you approved as written. Target 90 percent by week six. Below that, the packet format or the bot’s judgment needs work.
• Denials and reasons. Every deny is a lesson; log why.
• Tasks done versus reopened. Reopened means the “done when” line was vague or the work was wrong.
• Incidents. Injection attempts caught, credentials requested, links clicked, anything reported under the incident rule.
• Routine failures. Routines that errored or produced nothing. Long-unused routines get paused automatically by the platform, so review before assuming they still run.
• Cost. Weekly usage against the seat allowance, and any on-demand spend.

Rotate what can be rotated

Every token, application password and API key the bots use should carry the team’s name and a date, so you can revoke one without touching your own access. Rotate quarterly, or the same day anything leaks into a chat. Never paste a secret into a chat with a bot; a secret pasted into a chat lives in that history from then on. Use the masked secure-request field or sign in yourself in the Agent Computer.

The kill switch, rehearsed

1. Pause all routines on every bot.
2. Open the Agent Computer, sign out of every site, uninstall connectors.
3. Suspend the bot account in your identity provider; that kills mail delegation, drive access and anything tied to it at once.
4. Revoke the bot-named tokens: application passwords, API keys, CRM users, platform access.
5. Reset the computer from Settings; it returns to the last durable snapshot.
6. Ask the reviewer for an incident report: what happened, how it got in, which rule should have caught it.

Run this once during setup so it takes under five minutes when it matters. A framework such as the NIST AI Risk Management Framework is useful if you need to show a client or auditor that the process exists and is exercised.

When a bot drifts

Drift looks like a bot doing more than it was asked, quoting prices that are not in the approved list, using phrases the brand never says, or skipping the packet format. The cause is almost always one of three things: the charter was edited in chat and the change contradicted an earlier rule, the knowledge files were updated but the bot was not told to re-read them, or a new skill overrode an old one. The fix is to re-paste the charter, re-seed the knowledge, and re-test the skill. If the drift touched a live asset, the reviewer’s drift audit and the pod room’s change log tell you what to roll back.

Maintenance is lighter when the team is structured well from the start; that structure is described in How to make AI agents work together, and the pillar guide How to build a team of AI agents with Grok Bot has the full phased rollout.