Lab notebook

SEO claims, checked and tested

I saved over fifty short videos full of SEO advice. Here is what each one claimed, what the creator was selling, and what held up when I checked. The claims that can be tested are being tested on real sites, and the results land here, dated, whichever way they go.

How this works

I save short SEO and AI-search videos from TikTok and Instagram. Each one gets transcribed, and every claim in it is pulled out and checked against the original source: Google’s own documentation, the study being quoted, or the tool itself. A claim that can be tested on a real site gets tested, with a page left alone for comparison, success criteria written down before the data exists, and a dated log.

Grading a claim is not grading a person. Most of these creators post useful things alongside the hype, and every row links to who said it. Where I saved the clip by link, the name goes to that exact post, so you can watch it and disagree with me. Where I saved it off my phone there is no link to the post, so the name goes to the creator’s profile instead and is marked (profile). A few clips lost their handle entirely; those stay unlinked rather than guessed at. If I got something wrong, tell me and I’ll fix it and log the change.

The sibling experiment, the one with its own page and its own log, is the Reddit interception experiment.

What the verdicts mean

Holds up

Checked against the source, and it is right.

Partly true

The core is right. A detail or the size of the effect is not.

Overstated

Something real, sold as bigger than it is.

False

The source says otherwise.

Unproven

Cannot be checked from the outside yet.

Testing

Being tested on a real site now. See the tests below.

Won’t test

Against Google’s rules or the law, so not worth risking anyone’s domain.

Being tested now

Every claim so far

23 claims so far. On a phone, scroll the table sideways.

The log

12 August 2026

Started the Reddit interception experiment, the first claim from a saved video tested on a live site. It has its own page.

7 September 2026

Went through 61 saved clips in one evening. Each was transcribed, and every claim in it was checked against its source: Google’s documentation, the study being quoted, or the tool itself. Most of the rows above come from that pass.

14 September 2026

Added the expired-domain tactic and the “Google confirmed 40%” claim.

19 September 2026

Started this page. Picked three new tests to run alongside the Reddit one: IndexNow, three-question FAQs and llms.txt.

19 September 2026

Went back through every row against the saved clip and the notes I wrote when I checked it. Swapped each creator link from their profile to the exact post, for the clips I saved by link. Twenty-three of the twenty-eight now point at the post itself. Three I had screen-recorded off my phone instead, so there is no post URL to point at: those keep a profile link, marked as such. Two lost their handle in the recording and stay unlinked rather than guessed at. No platform was wrong. Edward Sturm’s clips are on Instagram and the rest are on TikTok, which is what the rows already said.

19 September 2026

One verdict changed, and it is mine that was wrong, not his. I had Matt Diamante’s agentic-calling clip down as “Holds up”. Google’s feature is real and I had checked that properly. The part I waved through is the half that actually sells the clip: that the shop which answered the phone beat better-ranked shops. That rests on a single search filmed on a phone, with the business never named. Feature confirmed, ranking claim not, so the row is “Partly true” now. My own note from 7 September already graded it that way and I did not read my own note carefully enough.

Where it stands

The pattern so far is plain. The loudest claims (“SEO is dead”, “80% of searches are zero-click”, “Google confirmed”) fall apart when you check the source. The useful advice is quieter, and most of it agrees with what Google has already written down. The tests above will settle the rest.