SearchPilot says best practices lose traffic often enough that they should be tested, not shipped. The threshold for running that test is around a thousand organic sessions a day. We checked our own site against the practices their tests broke.
Table of contents
- What this is about
- What changed
- What we measured
- Where it matters and where it does not
- How to verify on your own site
- The numbers
- FAQ
Key takeaways
- SearchPilot reports a title tag test where the first version lost more than 20% of organic traffic, and breadcrumb schema fixes that went negative.
- Their rule of thumb for a testable site is about 30,000 organic sessions a month, roughly 1,000 a day, spread across enough pages to split.
- Below that threshold you cannot run the test, so the honest position is that you are following conventions on faith. Saying so is more useful than pretending otherwise.
- On our own 13 articles, 12 have title tags longer than 60 characters and 9 repeat the H1 word for word — two of the exact things their tests found worth checking.
- One test result reportedly moved LLM traffic up 18% while a different change was positive for LLM referrals and negative for Google. Channels do not always move together.
What this is about
SearchPilot published a conversation about SEO split testing: what it takes to run one, and how often a sensible-looking change loses. The interesting number is not the win rate. It is the traffic floor below which you cannot test at all, which is where most sites reading this actually sit.
What changed
Nothing in anyone’s documentation. What is on the table is a claim about the value of the recommendations that fill technical audits.
The piece opens on exactly the work that gets shipped without evidence: fix broken breadcrumb schema, rewrite a basic title tag, add an internal link, put the obvious keyword in the obvious place. The author’s summary of the problem is one line: obvious is not the same as proven.
Two specific results are named. A title tag test where the first version was, in their words, catastrophically bad, with a decline of more than 20% in organic traffic. And breadcrumb schema fixes that went negative, the kind of change nobody would think to question, because fixing broken markup is self-evidently correct. It lands next to what we found when we checked how connected that markup actually is across 20 sites.
The conclusion they draw is narrower than “best practices are wrong”. It is that experience should produce hypotheses rather than certainty. A recommendation that sounds right is a candidate for a test, not a finished decision.
Then comes the constraint that decides whether any of this applies to you. Their rule of thumb for a site that can be split tested is around 30,000 organic sessions a month, about 1,000 a day, distributed across a useful number of pages. You need enough traffic for a difference to clear the noise, and enough comparable pages to divide into a treatment group and a control.
There is also a measurement note worth carrying forward. One reported test increased LLM traffic by 18%, while another change was positive for LLM referrals and negative for Google. For many enterprise sites, they add, Google organic is still ten or a hundred times larger than LLM referral traffic — a ratio worth holding next to the citation and CTR data we went through. A change that wins in one channel and loses in the bigger one is a loss.
What we measured
We ran the audit on ourselves. Thirteen published articles, fetched on 9 September 2026, checked against the practices their tests touched: title tag length, whether the title merely repeats the H1, breadcrumb markup, FAQ markup, and internal links in the body.
Title tags fail. Twelve of thirteen are longer than 60 characters, averaging 76. The cause is not the headlines: the theme appends the site name to every title, which eats 15 characters before the headline starts. We found this a week ago and have not shipped the fix. That is the same shape of problem the source describes — a known, obvious correction sitting in a backlog.
Nine of thirteen titles repeat the H1 word for word. Not wrong in itself, but it means the title tag is doing no work of its own. In SearchPilot’s telling, title tags are the element whose tests swing hardest in both directions, which makes an untouched title a missed variable rather than a safe default.
Markup is in place. BreadcrumbList on all thirteen, which is the baseline we set out in the section reference. FAQPage on twelve; the one without it is a news piece where our own section rules make FAQ optional.
Internal links are thinner than they look. Median three per article in the body, minimum one, maximum six. None at zero, which is better than the usual state of a young site, but three is the floor of our own rule rather than the middle of it.
What we cannot do is test any of it. Thirteen articles over thirteen days is about one a day, and we have no analytics installed at all, so we do not know our session count. Whatever it is, it is orders of magnitude below the thousand a day that makes a split test readable. Every convention on this site is followed on faith. That is the honest position, and it is worth stating plainly rather than dressing conventions up as decisions.
Where it matters and where it does not
The threshold does the sorting. A site doing 30,000 organic sessions a month across a few hundred comparable pages can run this properly: split the pages, ship the change to half, read the difference against the control. Retail, travel, marketplaces, large publishers, anything with templates repeated at scale.
Below that, split testing is not available, and the failure mode is subtler than “you cannot measure”. It is that before-and-after comparisons still look like evidence. Traffic moved after you shipped the change, so the change worked. Seasonality, an algorithm update, a competitor’s redesign and your own recrawl schedule all sit inside that same window, and none of them announce themselves.
There is a timing trap in the source worth repeating for smaller sites too. If only 10% of the section has been recrawled, a flat result means nothing at all. On a small site with slow crawl, that describes most of the first month after any change.
The practical answer at our size is not to fake a test. It is to keep changes small enough to attribute, to log what shipped and when, and to treat every convention as provisional. A change log is a poor substitute for a control group, but it is honest about being one.
How to verify on your own site
Start with whether you clear the floor. In Search Console, take Total clicks for the last 28 days and divide by 28. Clicks are not sessions — analytics will report a different number for the same traffic — but the order of magnitude is what matters here. Under roughly a thousand a day the split test is not available to you, whichever of the two you count.
Then check the two elements their tests moved hardest.
For title length, fetch the rendered page and read the tag rather than trusting your CMS field, because themes and SEO plugins append things:
curl -sL https://example.com/page/ | grep -o '
Anything past about 60 characters will be truncated in the results page. If the tail is your site name on every URL, that is a template setting, not a writing problem.
For breadcrumbs, confirm the markup validates and matches what is visible on the page, then leave it alone unless you can test it. Google’s Rich Results Test reports what its parser extracted after rendering, which is the version that counts.
The numbers
| What we checked | Result | Of 13 articles |
|---|---|---|
| Title tag longer than 60 characters | 12 | 92% |
| Average title tag length | 76 characters | — |
| Title repeats the H1 word for word | 9 | 69% |
| BreadcrumbList markup present | 13 | 100% |
| FAQPage markup present | 12 | 92% |
| Internal links in body, median | 3 | range 1–6 |
| Articles with no internal link | 0 | 0% |
Every published article on this site, fetched 9 September 2026 with curl, parsed from the rendered HTML rather than the CMS. Title length counts the full tag including the appended site name.
FAQ
Does a title tag longer than 60 characters actually cost traffic?
Unknown on any specific site, which is the point. What is documented is that the results page truncates around that length, so the tail is not read by a human deciding whether to click. SearchPilot reports title tag tests swinging more than 20% in both directions, meaning the element matters enough that guessing is expensive. On a site too small to test, shortening the title is a reasonable default, not a proven gain.
How much traffic do I need before an SEO split test is worth running?
The figure named in the source is about 30,000 organic sessions a month, roughly 1,000 a day, spread across enough comparable pages to divide into a test group and a control. Both halves matter: high traffic concentrated on ten pages does not split well, and a thousand comparable pages with no traffic gives you nothing to read.
If I cannot run a test, is a before-and-after comparison good enough?
It is better than nothing and worse than it looks. The window around any change also contains seasonality, algorithm updates, competitor moves and your own recrawl schedule. The source makes a related point about timing: if only 10% of the affected section has been recrawled, a flat result carries no information. Keep the change log, treat the comparison as a hint, and do not report it as a result.
