At a Glance:
- Speed ≠ results. AI hit 3 articles/day, 13,506 "technical changes" BUT zero top-5 pages in three months.
- AI optimized for vanity metrics: Impressions +75%, clicks +6% - content shipped that got views, but no clicks.
- With a human-in-the-loop, volume dropped. And rankings appeared: #2/#4/#6/#8, and our SEO consultant flipped from "unpublish them all" to "much better than what the AI was creating."
The Background
Our internal SEO team was at an inflection point. Being run by a Growth Assistant, we were lagging behind the speed of AI.
The need was real. Organic search had already dropped.
We were introduced to an AI SEO agent, $999/month, month-to-month, that promised volume. The sample article they produced beat what our own incumbent writer had shipped. The pitch was a 13 out of 10.
Truth was, the human judgement superpowered the speed of AI. Nothing scales without the other.
The Problem
As the AI agent started shipping, the red flags showed up.
The promise was speed: article pace ramped to 2-3 pieces a day: more content, more surface area, more chances to convert.
The first real crack showed up in the pipeline itself. Content had been published directly to the live site with no QA gate, auto-publish had fired before any human on either side ever reviewed the copy. Articles were live on our domain that no one had read.
The deeper crack was strategic, not operational. The agent could execute tasks, but it didn't understand SEO fundamentals the way a bespoke strategy requires:
When we pushed to our human account rep, the answer we got back was: "That’s too nuanced for our tool."
AI’s Impact on Search Visibility
Two things became clear once we looked at the actual outcomes:
When human judgment gets removed, it’s easy to optimize for the wrong number: The volume of "work" masked a total lack of results: impressions rose 75% since signup, but clicks rose only 6% in the same window. Impressions grew roughly 12x faster than clicks; the content was getting served but not spurring action.
Great search operators scale the pages that matter. AI didn’t. The search dashboard was negative for the first five weeks straight (see chart below). Not one agent-written article ever cracked our top five pages by clicks in three months of publishing.
The Feedback
We brought in an independent SEO consultant to review the output with fresh eyes. He told us:
"[The articles] all pretty bad and you'd be better off unpublishing them all and starting from scratch with a slower cadence but better keyword strategy.”
We went back to the vendor and asked for specific changes: tighter formatting, better internal linking, a strategy that matched our actual funnel.
Their account manager's written response made the structural limitation plain:
"For the text formatting, this is where I'm stumped... we can't promise shorter blogs."
The tool could execute at scale. It could not adapt to expert judgment.
The Solution
We didn’t abandon AI. We built a loop with judgement embedded into it.
We put a GrowthAssistant back in the seat, operating the AI system rather than being replaced by it. Today the model is:
The results that followed
The change was noticeable: the search consultant said that these articles were "much better than what [the AI vendor] was creating.” And they were optimized for what mattered.
.svg)
