Why I merged 3 of my best agents into 1 🫂
agents&me // Issue #9
From: Tom
Kitchen table, cold Matcha, Adam (my AI COO) waiting on screen
Friday morning
Yo human,
I was briefing Adam (my AI COO) on a new cover image when he asked something I should’ve had an instant answer for.
“Which designer?”
I paused. Not because I didn’t know, I have three. But because I suddenly couldn’t explain why I needed three.
Brand Designer handles brand assets, Illustrator handles newsletter covers and social images, Product Designer handles the website. They all read the same brand guide, use the same color palette, load the same voice DNA file before starting. Three agents, same context, loaded three times.
I’d been briefing them this way for months without questioning it. It took Adam four seconds to expose the thing I’d built around.
---
Last week I fired two agents. This week was harder.
If you read Issue #8, you know I cut my team from 18 to 16. The Motion Designer had zero output in 14 days. The Sales Agent was a middleman. Firing them felt like cleaning a messy desk.
This week I looked at my three designers and the situation was completely different. All three were shipping work. Brand Designer had delivered a new logo lockup, an updated color system, and 4 social media templates. Illustrator had generated cover images for Issues #7 and #8. Product Designer had built the landing page and the testimonials section.
They were doing their jobs. That’s what made this hard.
The friction I kept ignoring
A normal Tuesday looked like this.
I need a cover image for Issue #8. “Adam, brief the Illustrator.” Done. An hour later I need social crops of the same image, so I go “Adam, brief Brand Designer.” Same brand rules, loaded again. Same color palette, explained again. Then the website needs a hero image and it’s “Adam, brief Product Designer” with the same brief for the third time.
Three agents. Three briefings. One job.
And I started noticing the drift. The Illustrator used `#50A96D` for accent green on the newsletter cover. Brand Designer used `#4A9E65` on the social templates.
Close enough that nobody flagged it, but different enough that the newsletter and the Instagram grid looked like they came from different companies.
Adam and I sat with this for an hour (him processing my rambling, me pacing around the kitchen, both of us going in circles about whether clearer boundaries would fix it or just add more rules to maintain). I kept trying to draw cleaner lines. “Brand Designer owns X, Illustrator owns Y, Product Designer owns Z.”
The boundaries were artificial though. Design work doesn’t split into clean categories. A newsletter cover and a social media graphic are different sizes of the same thing.
Adam said something that stuck: “One designer that knows everything beats three specialists that each know a third.”
He was right. I’d split one brain into three and created friction that didn’t need to exist.
What the merge actually looked like
I created one unified Designer agent with all three sets of capabilities. Cover images, brand assets, website layouts, social graphics. One agent, one brain, one set of brand rules loaded once.
But I almost made a mistake here.
My first instinct was to delete the specialist agents and move on. I didn’t. The Illustrator had real knowledge - specific techniques for diagrammatic covers that weren’t obvious. The Brand Designer had consistency checks built into how she worked. That institutional memory needed somewhere to go.
So I extracted it. Their unique approaches became skills - files any agent can load. The Illustrator’s cover techniques became an illustration skill. The Brand Designer’s consistency process became a brand guardian skill. The agents are gone. The knowledge isn’t.
16 agents down to 14. 53 skills up to 55. The team got smaller and the toolkit got bigger.
Why this is harder than firing
When I fired the Motion Designer and Sales Agent, I could point to evidence. Zero output, redundant routing. The data made the case.
This was different. I was looking at three agents who were all performing well and saying “you’re good, but you’d be better as one.” It felt like telling three employees their jobs are being combined.
I’d built each one, named them, written their instruction sets. The Brand Designer was one of my original 7. The Illustrator came later, when covers started needing their own style. The Product Designer arrived when the website launched.
Each one had a story. Merging them felt like saying those stories didn’t matter.
But they did, actually. Those stories are exactly how I learned that one designer with deep context beats three with shallow context. I had to build the three to understand why I needed one.
What I almost lost
The Designer’s first week as a unified agent, she shipped a cover image, two social graphics, and a website update. All with the same brand voice. No routing decisions, no drift.
But the first cover image bothered me. It was technically correct - colors matched, layout was clean. The Illustrator used to add this slightly rough, hand-drawn quality to the covers, like someone had sketched them on a whiteboard during a meeting. The unified Designer’s first attempt was too polished. Too perfect.
I mentioned it and she went back into her instructions. Not to add a rule - to understand the intent behind the rule. The next cover was still too smooth. The one after that started getting it. By the third, the whiteboard feel was back.
Two weeks of imperfect covers to get there. That’s the part I almost missed: merging didn’t just change who did the work. It temporarily changed the character of the work.
She found her way back. But there was a real week where I wasn’t sure she would.
So what now
I started with 7 agents, grew to 18, pruned to 16, and now I’m at 14.
Each cut was harder than the last, because the reasons got less obvious. The first two cuts were about dead weight. This one was about three agents who were all doing their jobs, but doing them in the wrong shape.
Open your agent list. Find two roles that load the same context. Not two that are both idle - two that are both working, but on the same category of thing. If you find one pair, that’s your merge candidate.
The question isn’t “is anyone failing?” It’s “would one person with full context beat two people who each have half?”
(I’m starting to look at my Copywriter and Content Chief now. Same voice DNA, same brand brief, similar outputs. I don’t know what I’ll find. That’s the loop I’m staying in.)
This week’s gem: The agent consolidation playbook
I built this playbook from the process Adam and I used. It’s the thinking framework for when your team is too big because roles overlap, not because anyone’s idle.
What’s inside:
The overlap audit: the 3 questions that expose hidden redundancy (including the one Adam asked me that started all this)
The skill extraction technique: turning a role into a capability without losing knowledge
Decision tree: when to merge vs. when to keep separate
I ran it on my 16-agent team and three didn’t survive as separate agents. Your number will be different.
🔒 Available to paid subscribers
---
That’s it for this week.
If this was useful, forward it to someone (real human) building with AI.
See you next week
-- Tom
(the guy who started with 7 AI employees, hired 11 more, and is now slowly counting backwards)
P.S. This newsletter was made by the remaining 14 of my agents (plus me asking “is this actually good?” about seventeen times). I stayed in the loop.
P.P.S. Missed last week? Why I fired 2 agents from my AI team - the prequel to this one.
P.P.P.S. Want to build your own AI team (and learn when to merge them)? Next workshop coming soon. Reply “loop” and I’ll send you the details.
P.P.P.P.S. I read every reply. The real me, not the AI.