I Built a System to Measure Whether AEO Was Working. The Proof Showed Up First.
by Meghan Brenner
This week I finally did something I should have done months ago: set up a real way to measure whether our AEO work was actually doing anything. That meant sitting down with GA4, Google Analytics 4, and getting honest about how little I'd actually used it.
The Tool I Technically Already Had
I'd had GA4 running on our site for a while, the same way I "have" a gym membership. It was there. I wasn't really using it.
For anyone in the same boat: GA4 is the thing that tells you where your website visitors came from. Did they click a link from Google search, from LinkedIn, from an email, from typing your address straight into the browser. It sorts all of that into buckets so you can actually see it, instead of just knowing people are showing up.
What I Actually Did This Week
I made sure GA4 was properly connected to our website. There's a piece called a "container" that has to be installed and configured correctly, and getting that right took longer than I expected. I also added UTMs, small tags on the end of a link, to our newest content. Without a UTM, a lot of traffic just lands in a generic "referral" bucket and you lose the detail. The goal was to be able to see, specifically, whether someone arrived from AI sources like Claude, ChatGPT or Perplexity, instead of an unhelpful "other."
The Signal Showed Up Out of Order
While I was in the middle of that setup, something happened that changed how I thought about the whole project. On sales call after sales call, when I asked how someone had found us, they didn't say Google. They said some version of "I asked ChatGPT" or "it came up when I searched with AI." This wasn't a single lucky week. It kept happening.
I didn't have a dashboard yet. I didn't have a single chart. And I already had proof that something was working.
I'd been thinking about measurement as the thing that comes first: you build the tracking, then you wait for the data, then you find out if it worked. Instead, the signal showed up before the system built to capture it did. A handful of conversations told me more than any report could have, and I almost missed it because I wasn't asking the right question on those calls until I happened to.
"How did you find us" is a question that's been available to ask the entire time. It costs nothing. It doesn't require GA4, UTMs, or any of the setup I spent this week on.
What the Numbers Actually Show
Once the tracking was in place, I checked. GA4 now has a separate line called "AI Assistant," traffic that comes in through tools like Claude, ChatGPT or Perplexity instead of getting buried in "referral." Over the past few weeks, that's 58 sessions. Less than one percent of our total traffic. I'm not going to dress that up as a wave.
Here's the part that got my attention. Those 58 sessions have a higher engagement rate than almost every other channel on the site, second only to organic search. People arriving through an AI tool are sticking around and doing something once they get there. It's a small number, but it isn't a hollow one.
The sales calls told me the work was landing before the spreadsheet did. Now the spreadsheet is starting to agree.
Here’s What I Actually Did, In Order
If you want to replicate this setup, here's what I did, in plain terms:
Confirmed the GA4 container was installed correctly. There's a free browser extension called Google Tag Assistant that can check this without any technical knowledge. It tells you whether the right tags are firing on your site.
Created a custom channel group for AI traffic. Inside GA4, under Admin, you can create a custom channel group that tells the system to look for traffic from specific domains, like chat.openai.com, perplexity.ai, and gemini.google.com, and label it "AI Assistant" instead of lumping it into "referral" or "unassigned."
Added UTMs to new content. When I share a link to a specific blog post or newsletter, I add a short tag to the end of the URL so GA4 knows exactly which piece of content sent that visitor. Without it, the source data gets fuzzy fast. I used the campaign creator within Google to set up the specific UTMs.
Waited, then checked Traffic Acquisition. Under Reports, Acquisition, Traffic Acquisition, I can now see a separate "AI Assistant" row. It took a few weeks before there was enough data to be meaningful.
Started asking on every sales call. Before the GA4 data was useful, I asked every prospect how they found us and wrote the answer down. That question costs nothing and doesn't require any setup at all.
The Takeaway
If you're a non-technical operator wondering whether your AEO work is doing anything, don't wait for a perfect dashboard to start paying attention. Ask everyone who contacts you how they found you, and write the answer down somewhere, even a simple spreadsheet. Check whether your existing analytics separate AI referral traffic from everything else, most platforms can show this, but it's often buried under "referral" or "unassigned" by default. The measurement system isn't what tells you the work is real. It's what tells you what to do next once you already know it's real.
I write about exactly this kind of thing, in real time, in The Muddy Middle, a newsletter for non-technical operators figuring out AI as they go. Subscribe here and I'll see you in your inbox.
Meghan Brenner is COO at JB Sales and founder of The Operator's Notebook.
