AI agent team series, part four
How to maintain a team of AI agents after launch
Launch is the easy part. A team of agents stays useful only if somebody reads the lessons, audits the access, and retests the skills on a schedule. This is that schedule, and the small set of numbers that tells you whether it is working.
What does ongoing maintenance of an AI agent team involve?
A daily lesson line from every task, a Friday retro that promotes repeated lessons into skills or rules, a weekly access audit of plugins, sign ins, routines and tokens, monthly retesting of every skill on real work, quarterly drift audits against the brand context, a five minute kill switch you have rehearsed, and a small set of metrics: approval without edit rate, denials, reopened tasks and incidents.
The whole rhythm costs about an hour a week of human attention. A team that does not get that hour degrades quietly, and the first sign is usually work that is subtly off brand rather than work that is obviously wrong.
By the VIS Mountain Editorial Team. Published . Updated . The sources behind it are listed below.
An agent team is a system that decays unless something maintains it
Nothing about an agent team is self sustaining. Charters get edited in passing, knowledge files get updated without anybody telling the bots to re-read them, skills that worked in month one stop matching the tool they were built around, and access granted for one project stays granted for a year.
None of that is dramatic. It is the same slow drift that happens to any set of written procedures nobody revisits, and it produces the same result: a process that everybody assumes is running and nobody has checked.
The difference with agents is speed. A person carrying a stale procedure produces a handful of wrong outputs before somebody notices. A team of agents running a stale procedure on a routine produces them on a schedule, at volume, with confidence.
So maintenance here is not tidying. It is the control that keeps everything in the other six parts of this series true after the week you set it up.
Five layers, from one task to one quarter
Each layer catches what the one below it missed, and each has a named owner. A loop with no owner is a diagram rather than a process.
Per task: one lesson line
Every bot ends every task with a single line saying what it would do differently, appended to its own playbook file. One line, not a paragraph, because the value is in having a hundred of them to compare rather than in any one being thorough.
Per correction: a feedback entry, verbatim
When you correct a bot, it writes an entry recording what you said word for word, why, and how to apply it. These outrank everything else, including its own lesson lines, because they are the only input that comes from the person the work is for.
Weekly: the Friday retro
The learning officer reads all new lesson and feedback lines, dedupes them, and for anything that has recurred twice either rewrites the affected skill, teaches a new one, or proposes a new approval rule. You approve skill changes rather than the bot applying them.
See the remaining steps: Five layers, from one task to one quarterHide the remaining steps: Five layers, from one task to one quarter
Monthly: the skills audit
Every saved skill is re-run on a real task. Stale ones are paused rather than deleted, so you can tell the difference between a skill that failed and one that was never there.
Quarterly: the drift audit
The reviewer checks live assets, meaning pages, profiles, campaigns and workflows, against the current brand context, and opens an incident for each mismatch. This is the layer that catches work that was correct when it shipped and is not correct now.
xAI's documentation notes that Grok Bot saves skills in plain language, that one can be taught by demonstrating a workflow for up to ten minutes, and that skills are shared across bots on a workspace. That last point is what makes the monthly audit worth the hour: one good skill taught once belongs to everyone, and so does one stale skill.
The weekly access audit
Done by the learning officer, read by you. Anything that has finished gets signed out, uninstalled or paused the same day rather than added to a list.
- Installed plugins and connectors: what is there, and which of them anybody still needs.
- Sites the agent computer is signed into. Access granted for one project is the most common thing left running.
- Active routines, and whether each one is still worth running at all.
- Tokens, by name and by age, so that anything older than the rotation window is visible.
- Anything created during the week that nobody would have approved if asked at the time.
The platform's own guidance says the same thing: sign out of services when access should end, remove temporary files, and delete connectors or revoke authorisation when they are no longer needed. One detail that is easy to get wrong is that deleting a bot does not delete the shared computer's files or its browser sessions, so cleanup is a separate step every single time.
Six metrics that tell you the truth
None of these is a vanity number, and all six are cheap to collect because the task files already contain them. The first one is the single best early warning you have.
Approval without edit rate
The share of packets you approved as written. The working target is ninety percent by about week six. Below that, either the packet format or the bot's judgement needs work, and the metric will not tell you which, so read a few of the edits.
Denials, and the reason for each
Every denial is a lesson, and the reason is the useful half. A log of denials with no reasons is a count of frustration rather than a source of improvement.
Tasks done against tasks reopened
Reopened means the done when line was vague or the work was wrong. A rising reopen rate almost always traces back to task definitions rather than to the bots.
Incidents
Injection attempts caught, credentials requested, links clicked, and anything else reported under the incident rule. A month with zero incidents and an inbox agent running is worth checking rather than celebrating.
Routine failures
Routines that errored or produced nothing. Long unused routines get paused automatically by the platform, so review before assuming anything still runs.
Cost
Weekly usage against the seat allowance, and any on demand spend. Worth watching from week one, because unattended work is exactly the kind that runs past a limit nobody set.
Rotate what can be rotated, and name everything
Every token, application password and API key the bots use should carry the team's name and a date. The name is what lets you revoke one without touching your own access, and the date is what makes an old one visible without an audit.
Rotate quarterly as a baseline, and the same day anything leaks into a chat. There is no partial version of that second rule: a secret pasted into a chat lives in that history from then on, and rotating it is the only thing that makes the history harmless.
Never paste a secret into a chat with a bot in the first place. Use the platform's masked secure request field, or sign in yourself inside the agent computer view and let the bot carry on from the signed in state.
The kill switch, in order
Run this once during setup so it takes under five minutes when it matters. A kill switch you have never used is a document, and documents are slow at three in the morning.
Pause all routines on every bot
This stops new work starting while you deal with what is already running.
Sign the agent computer out of every site, and uninstall connectors
Browser sessions outlive the bot that opened them, which is the step people forget.
Suspend the bot account in your identity provider
This is the highest leverage step in the list. It kills mail delegation, drive access and everything else tied to that identity at once.
See the remaining steps: The kill switch, in orderHide the remaining steps: The kill switch, in order
Revoke the bot named tokens
Application passwords, API keys, CRM users, platform access. This is why they were named after the team.
Reset the computer from Settings
It returns to the last durable snapshot, which is what removes anything written to it.
Ask the reviewer for an incident report
What happened, how it got in, and which rule should have caught it. Without this step you have stopped the bleeding and learned nothing.
If you need to demonstrate to a client or an auditor that this process exists and is exercised, the NIST AI Risk Management Framework gives you a recognised structure to record it against.
When a bot drifts, it is almost always one of three things
Drift looks like a bot doing more than it was asked, quoting prices that are not in the approved list, using phrases the brand never says, or quietly skipping the packet format.
The cause is nearly always one of three. The charter was edited in chat and the change contradicted an earlier rule. The knowledge files were updated but the bot was never told to re-read them. Or a new skill overrode an old one.
The fix follows the cause: re-paste the charter, re-seed the knowledge, re-test the skill. Do all three if you cannot tell which it was, because it takes ten minutes and diagnosing it properly takes longer.
If the drift touched a live asset, the reviewer's drift audit and the pod room's change log tell you what to roll back. That is the only reason the change log is worth keeping, and it is reason enough.
What the maintenance calendar actually looks like
Put these in a calendar with a named owner. Maintenance that depends on somebody remembering is maintenance that happens for three weeks.
Open the full comparison: What the maintenance calendar actually looks likeHide the full comparison: What the maintenance calendar actually looks like
| Cadence | What happens | Who owns it |
|---|---|---|
| Every task | One lesson line appended to the bot's playbook | Each bot |
| Every correction | A feedback entry recording your words verbatim | The bot being corrected |
| Daily | The standup and the two approval digests | Chief of staff |
| Weekly, Friday | Retro, plus the ten minute access audit | Learning officer |
| Monthly | Every saved skill re-run on a real task, and the injection test | Learning officer and reviewer |
| Quarterly | Drift audit against the brand context, and token rotation | Reviewer |
| Once, then after any change | Kill switch rehearsal, timed | You |
Add it up and the human cost is roughly an hour a week plus a longer session once a quarter. That is the real running cost of an agent team, and it is worth knowing before you build one rather than after.
What good looks like around week twelve
Not a promise, and not a benchmark anybody has published. It is the shape of a team that has had the maintenance rhythm applied to it for a quarter.
- You approve most packets as written, and the ones you edit are edited for judgement rather than for facts.
- The inbox agent's known senders file is longer than the list you could have written from memory.
- Every skill in the library has been re-run in the last month, and the ones that were not are paused.
- Nothing is signed in that nobody can explain, and every token has a name and a date.
- The reopened task rate is falling, which means the done when lines are getting sharper.
- You have run the kill switch at least once and know how long it takes.
- At least one incident has been reported and written up, because a quarter with none usually means nobody is reporting.
If most of that is not true at week twelve, the usual cause is that the Friday hour never happened. That is a scheduling problem rather than a software problem, and it is the most common reason an agent rollout fades rather than fails.
The other six parts
Seven pages, written to be read in any order. Each one owns a single part of the system so none of them has to repeat the others.
The complete A to Z guide
The whole system in one place: roles, approvals, the shared brand file, and the phased rollout that starts read only.
Read the guideSet up your first agent
The forty minute sequence that decides whether an agent is a colleague or a liability, including the charter format.
Read the guideEmail and inbox security for agents
The three absolutes, the five sender checks, the label scheme, and the incident rule that makes slips reportable.
Read the guideMake agents work together
One chief of staff, pods with their own rooms, a task file with a done when line, and one approval packet format.
Read the guideWhy one agent is not enough
Why the do everything assistant is the setup most people abandon, and which of its failures a team actually fixes.
Read the guideGrok Bot vs Hermes Agent vs OpenClaw
Managed against self hosted: where each one runs, who is responsible for securing it, and which suits which situation.
Read the guideAlready running agents and not sure the rhythm is holding?
The access audit and the drift audit are the two most people skip. If you want a second pair of eyes on either, we are happy to talk it through.
No affiliation. VIS Mountain is not affiliated with, sponsored by or endorsed by any product named on this page, and none of them reviewed or approved it. Grok Bot and Grok are trademarks of xAI. Cursor is a trademark of Anysphere, Inc. Hermes Agent is a project of Nous Research. OpenClaw is an open source project of its maintainers. Google, Gmail, Facebook, Meta and other product names are trademarks of their respective owners. Every product name and logo shown here belongs to its owner and appears only to identify the product this page describes.
Informational only. This is general information about configuring AI agent software. It is not legal, cybersecurity, financial or professional advice, and reading it does not create a client relationship with VIS Mountain. Consult a qualified professional before relying on it for your own systems, data or compliance obligations.
Read the full breakdownHide the full breakdown
Accuracy, timeliness and attribution. Everything specific to a named product on this page is what that product's own documentation, or a named security researcher, published as of September 2026, not a claim we are making on our own authority. We link to each source rather than restate it as our own finding. Features, plans, limits and security details can change without notice and all of these projects change quickly, so verify current terms with each vendor before purchasing, deploying or granting access.
No guarantees, and your responsibility. AI agents can make mistakes and can be manipulated. No configuration described here eliminates risk. Results, security outcomes and cost depend on your implementation. You are responsible for complying with the laws and platform terms that apply to you, including privacy and data protection laws, anti spam and telemarketing rules such as CAN-SPAM and the TCPA in the United States, industry rules such as HIPAA where applicable, and each vendor's terms of service.
Third party links. External links are provided for reference. VIS Mountain does not control and is not responsible for the content, availability or practices of third party sites.
How this was made. First published on 3 September 2026 and migrated to this site with its disclosures intact, because they are the part that tells you how far to trust it. Prepared by the VIS Mountain editorial team with the assistance of AI tools and reviewed by a human before publication. Examples are generic and describe no specific client, person or account.
No warranties. Provided as is, without warranties of any kind. To the fullest extent permitted by law, VIS Mountain disclaims liability for losses arising from use of this information.
Straight answers.
How much human time does maintaining an agent team actually take?
Roughly an hour a week once the rhythm is established: the Friday retro plus the ten minute access audit, with the daily standup and approval digests read rather than written. On top of that there is a longer session once a quarter for the drift audit and token rotation.
The hour is not optional in the way it looks. It is the only point in the week where lessons become rules, and a team whose lessons never become rules repeats the same correction indefinitely.
What is the first sign that something has gone wrong?
A falling approval without edit rate. It moves before anything visible breaks, because you notice yourself rewriting packets a week before you notice the output is off.
The second sign is a rising reopened task rate, which usually means the done when lines have stopped being checkable rather than that the bots have got worse.
Does deleting a bot remove its access?
Not on a shared computer platform. xAI's documentation states that deleting a bot does not delete the shared computer's files or its browser sessions, so a deleted bot can leave live sign ins behind.
Cleanup is a separate step every time: sign the agent computer out of the sites that bot used, uninstall any connectors it needed, and revoke the tokens named after it. That is exactly what the weekly access audit is for.
How often should we rotate tokens and passwords?
Quarterly as a baseline, and immediately if anything leaks into a chat. Rotation is only practical if every credential is named after the bot team and dated, which is a setup decision rather than a maintenance one.
The rule with no exception is that secrets are never pasted into a chat with a bot. Use the masked secure request field, or sign in yourself inside the agent computer view.
A bot has started doing things we never asked for. What now?
Re-paste the charter, re-seed the knowledge files, and re-test the most recently added skill. Those are the three causes of drift, and doing all three takes about ten minutes, which is less than diagnosing which one it was.
Then check whether anything live was affected. The quarterly drift audit and the pod room's change log are what tell you which published page, profile or campaign to roll back.
Should the reviewer be a separate agent, or can one bot do both?
Separate. A bot reviewing its own work has already made the judgement you want reviewed, and it will make it the same way twice.
The reviewer's value is that it applies a written rule and returns a pass or fail with the rule it used, which gives you something to argue with. It can reject, but only a human approves a consequential action. Its pass is a filter on your queue, never a bypass.
Where this comes from.
Primary documentation and published research behind the guidance on this page.
- xAI: Grok Bot skills, routines and automations (opens in a new tab)How a skill is taught and how a scheduled routine behaves when it is tested.
- xAI: Grok Bot approvals, security and privacy (opens in a new tab)Where the approval prompt, the Auto Review layer and the local execution setting are documented.
- NIST: AI Risk Management Framework (opens in a new tab)A published framework for documenting who is responsible for an AI system's risks.
- xAI: Grok Bot documentation (opens in a new tab)The vendor's own description of bots, the cloud computer, memory and sessions.
- OWASP: Top 10 for LLM applications and generative AI (opens in a new tab)Where prompt injection is ranked first and excessive agency is listed as a separate risk.
Talk to the team
A short call, a look at how the business currently shows up, and a straight answer on what we would do first.