My SEO Robot Has 77 Days of Data. Here's What It Does and Doesn't Prove

Last week I wrote about the search monitor itself: the dashboard that watches my portfolio of websites, asks the AI assistants what they think of them, and now ships its own fixes through a pull request pipeline I approve with a checkmark in Discord. That post explained the machine. This post is what happens when the machine has data.

Three weeks in, the monitor holds 77 days of daily Search Console series for the sites that were running from day one, 153 changes in the change ledger, weekly AI answer runs across a couple dozen sites, and more before-and-after PageSpeed captures than I expected to need. So here is the results post, with the discipline I try to hold every number to: each figure below comes straight out of the monitor’s database, and where a number is weak I say so.

The number that matters here is year over year

If you run a business in the Florida Keys, month-over-month SEO reporting is close to useless. Charter search demand collapses after Labor Day, when the spring break crowd is a memory, the kids are back in school and hurricane season owns the weather conversation. Every weekly click line in my portfolio trends down and to the right from July to September, and almost none of that is about the websites. DirtyBoat, one of the busiest charter sites in the portfolio, peaked the week of July 20 at 393 organic clicks and came in at 141 the week of September 21. That is the season leaving. It is not the work failing.

So the monitor’s default lens is not the last 28 days against the prior 28. It is the last 28 days against the same 28 days a year ago, the same corner of the calendar, apples to apples. Comparing September to August tells you about the season. Comparing this September to last September tells you about your website.

And that lens changes the whole picture. On the sites the monitor has a full year of history on, the seasonal dip is sitting on top of a much bigger site than last year. DirtyBoat, latest 28 days versus the same 28 days a year ago:

• Clicks: 514 to 860, up 67%.
• Impressions: 20,592 to 51,259, two and a half times.
• Average position: 32.2 to 8.6. Page four to the top of page one.

That is the single strongest “the program is working” artifact in the database, and it is only visible because the comparison is year over year instead of month over month.

Two honest caveats before anyone gets excited. A twelve-month movement reflects a year of audits, iteration and now this change pipeline, not the 21 changes that shipped in the last week of September. It shows direction, not attribution for any single fix. And the click-through rate actually fell, 2.5% to 1.68%, because impressions grew faster than clicks: the site now surfaces for a much wider set of queries, and that wider net converts at a lower rate. Year over year, the reach grew far faster than the clicks. I’d rather print both sentences than pick the flattering one.

What “working” can honestly mean at three weeks

Two kinds of signals, and mixing them up is how SEO reporting becomes fiction.

Lagging indicators are clicks, bookings and revenue. They only mean something across a season, which is exactly why the monitor reads them year over year. The DirtyBoat number above is a lagging indicator done right.

Leading indicators are average position, impressions on targeted queries, click-through rate, new pages getting indexed, new queries appearing, AI citations and page speed. They show up in days, and they are how you judge last week’s changes on the newer sites in the portfolio, the ones without a year of history to compare against yet.

So here is every site where the leading indicators are moving, with the receipts.

Safe Fishing: the one that surprised me

Safe Fishing is a nonprofit charter-industry site in the portfolio with eleven fixes shipped between September 20 and 28. It joined the monitor in September, so there is no year-ago window to read yet. Here the honest signals are the leading ones, and three of them moved at once.

The rolling 28-day average position in Search Console fell from 20.8 on September 11 to 14.4 by September 26, improving nearly every single day in between. Click-through rate went from 0.97% to 2.84%, and the window’s click count, at 10 against 4, is far too small to mean anything on its own but sits well next to that position and CTR move. And in the weekly AI answer runs, the brand was mentioned in 4 out of 34 questions on September 20 and 21 out of 51 questions on September 28.

That AI jump comes with an asterisk, because the question set grew between runs. But coverage quadrupled on a bigger denominator while most portfolio sites held flat or fell. Two new news pages also got indexed and started earning impressions within days. This is the cleanest multi-metric story in the portfolio, and it’s a site I was not expecting it from.

DirtyBoat: growing impressions into a seasonal headwind

DirtyBoat got 21 shipped changes, the most of any site, and it’s the only large site where I can point at query-level receipts.

The rolling average position improved from 10.6 in the prior window to 8.6 in the current one, and the daily trend line has gone down without interruption since September 11: 9.6, 9.5, 9.3, all the way to 8.6. Impressions across the latest window rose 8.3% against the prior window, from 47,338 to 51,259, at the exact moment every other site in the fleet is losing impressions to the season.

The targeted queries did the most work. “islamorada fishing” went from 2,639 impressions in the early September window to 4,656 in the late one. The Islamorada charters hub page went from 7,853 impressions to 10,429. The swordfish season guide tripled, 496 to 1,271. Three queries that did not exist in the top 100 a month ago now do: “best islamorada fishing charters,” “fishing trips islamorada,” “florida keys fishing trip.” The AI answer reports show brand coverage in assistant answers climbing from 35% to 43% across two weekly windows.

The window-over-window asterisk: clicks against the prior 28 days are still down 16%. Of course they are, the season is walking out the door, and September click lines fall no matter what I ship. The two reads above are the durable ones: the year-over-year headline proves the program, and the query-level receipts show which pages are moving while we wait for October’s post-deploy windows to mature.

Big Beard Boats: small numbers, all pointing the same way

Big Beard is a boat repair and sales site with 17 shipped changes, and its total organic clicks since July could fit in an Uber. Absolute numbers here are almost rounding error. What’s interesting is that everything moved together: clicks 9 to 14 window over window, click-through rate 2.51% to 4.14%, average position 16.4 to 12.5, with the daily trend inflecting right around the deploy dates. And brand-new queries started appearing, like “mobile marine mechanic florida keys” going from zero impressions to 22. That’s new service content surfacing for searches it didn’t exist for last month. Small sample, right direction, worth watching.

This website: the AI numbers are the story

Kit Mobley dot com got 14 fixes and is the site with the cleanest AI visibility trend, because the question set stayed fixed at 51 questions across the last two runs. The brand was named in 7 mentions out of 34 questions on September 20, 13 out of 51 on September 21, and 19 out of 51 on September 28. Coverage went 20.6% to 37.3%. Average position sharpened from 31.0 to 25.7 in the final week. And the AI visibility monitor blog post published this month picked up 52 impressions in its first stretch, with queries like “ai crawler monitoring” showing up, which is the sound of new content getting indexed and answering a search that didn’t have me in it before.

The only real before-and-after on the board

Everything above compares rolling windows. One thing doesn’t, and it’s the most airtight artifact in the database.

The Alissa Whittle site got a performance fix on September 9. The same day, the monitor ran the homepage through mobile Lighthouse before the deploy and again after: score 59 to 85, Largest Contentful Paint from 4.15 seconds to 3.43, Total Blocking Time from 858 milliseconds to 305. Those are lab runs from my own VPS, not field Core Web Vitals, and the monitor says so in the record. A synthetic before-and-after with a timestamp on both ends is still a before-and-after.

What this data does not prove, and I’ll die on this hill

An honest results post needs the part a pitch deck would cut.

The timing is barely there. Of the 153 logged changes, essentially all shipped between September 20 and 29. Search Console data lags about three days, so the current window ends September 26. That’s days, not weeks, of post-deploy data for most of the work.

The controls also moved. Sites that received nothing during this stretch drifted too: on some, average position improved by close to a full position over the same period; on others it slid. When the quiet sites move on their own, a rising line on a changed site is evidence, not proof. Attribution here is suggestive. It deserves the word “encouraging” more than the word “proven.”

Bookings cannot carry a claim yet. DirtyBoat’s booking system fed the monitor 10 confirmed bookings in the last 28 days, 7 still active. That is not a sample, that’s a group chat.

The ledger itself is young. The change log starts September 20, so older work from the predecessor tooling has no deploy dates attached. A change without a date is a rumor.

When the honest proof arrives

The reason this post exists in three weeks instead of three months is that the instrument to settle the question is already running. The change ledger has deploy dates. The daily Search Console series has position, impressions and clicks at day resolution, and it captures the same window a year back, so the seasonal noise is subtracted out before I ever look at a chart. The AI runs have fixed question sets.

For the sites that already carry a year of history, that year-over-year view is the headline now, and the answer is yes, the sites are bigger than last year. For the newer ones it fills in as the database matures. And by mid-to-late October, the latest 28-day window for each shipped change will cover four or more weeks of post-deploy data. At that point the question stops being “do I believe the charts” and becomes bookkeeping: for each dated change, what did its window do, year over year and against the quiet sites. I’ll publish that too, red arrows included.

If you run a business and you want your website watched this way, positions, AI citations, broken fixes and all, let’s talk.

Frequently asked questions

How long does it take to prove SEO work is working?

Real traffic proof takes four to eight weeks after a change ships, because Google re-crawls and re-ranks on its own schedule and Search Console data itself lags about three days. Leading indicators like average position, impressions, indexing and AI citations move sooner and are the only honest signals in the first month.

Why can clicks fall while search positions improve?

Seasonality. In the Florida Keys, charter search demand falls sharply after Labor Day. A site can rank better for fewer searching people and show fewer clicks while the underlying visibility improves. That is why the monitor's primary lens is the latest 28-day window against the same window a year ago, not against last month. Month over month describes the calendar. Year over year describes the website.

How do you measure AI answer visibility?

Every week the monitor asks ChatGPT, Claude and Gemini the questions a real customer would ask about each business, and records which brands get named and which domains get cited. It runs the same question set each week so the mention rate and share of voice are comparable.

What is a change ledger?

A log of every change shipped to a site, with its date, what changed and where. Without it you cannot line up traffic movements with the work that caused them, and every results story is just vibes about a chart.