Did your website changes help? What to check next

You changed a page's title three weeks ago and clicks are up. Did the title do that? Perhaps. Clicks also rise when more people search, when a competitor slips, when Google updates its ranking systems, or when a colleague edits the same page in the same week.

Most SEO measurement doesn't fail because the data is wrong. It fails because the comparison is loose: the whole site instead of the searches you targeted, "since the change" instead of a fixed window, and a result read on whichever day it looked best.

This guide sets out a fixed schedule: one baseline, five windows, a short list of searches, and a rule written down before you look. Then it walks one title change through all five windows, including the one that would have fooled you.

Start with a baseline: the 28 days before

The baseline is the 28 days ending the day before the change. Leave the day of the change out, since it's half old and half new.

Twenty-eight days is four full weeks, so it contains each day of the week exactly four times. That matters because most sites get different traffic at weekends. A baseline of, say, 30 days would carry two extra weekdays or weekend days and tilt the comparison.

Write the baseline down on the day you make the change, with the exact filters you used. Rebuilding it later, with slightly different filters, is how baselines drift.

Read five 7-day windows

After the change, measure the 7 days ending at day 7, day 14, day 30, day 60 and day 90. The day-30 window, for example, is days 24 to 30.

Seven days again contains each weekday once, so you can compare a window with the baseline divided by four. Fixed windows are better than "everything since the change", which blends the early days (when Google may not have seen the change yet) with the later ones and hides a drop that happens after a rise.

Each window has a job:

• Day 7: did anything break? Google may not have recrawled the page yet, so a flat result here means little.
• Day 14: the first fair read of a title or meta description change.
• Day 30: the first decision point for most content changes.
• Days 60 and 90: did it last? Links and larger content changes often take this long to show at all.

Search Console data arrives late: the most recent 2 to 3 days are usually missing or incomplete. So read the day-7 window on day 10 at the earliest, and every other window about three days after it closes. A window read too early looks like a drop, because its last days are only partly counted. Google's help on the performance report describes what the report counts.

Measure the searches the change targeted

A title change aimed at five searches shouldn't be judged on the page's total traffic, and certainly not on the site's. The page's other searches carry on moving for their own reasons and drown the signal.

Before you make the change, write down the searches it targets: the ones the new title, section or link is meant to win. In Search Console's performance report, filter by that page and those queries for both the baseline and every window.

Then glance at the page's other searches separately. A title that wins one search can lose another, and you want to know.

Impressions, clicks and position say different things

• Impressions: how often the page was shown. They move with demand (how many people search) and with ranking.
• Position: the average position the page was shown at. It can get worse while things improve: if the page starts appearing at #40 for new searches, the average falls even though nothing was lost. Only read it for a fixed set of searches.
• Clicks: the outcome you care about, but clicks are impressions multiplied by CTR (click-through rate), so a rise in demand raises clicks with no help from you.
• CTR: clicks divided by impressions. This is what a title or meta description mainly changes.

So judge each kind of change on the metric it's meant to move. For a title or meta description, watch CTR, with position as a check that ranking didn't slip. For a new section or FAQ, watch impressions and position first, then clicks. For internal links and backlinks, watch position and impressions over 60 and 90 days.

One simple calculation separates the title's effect from changes in demand: clicks at the old CTR = the window's impressions × the baseline CTR. It's the number of clicks you'd have got if nothing about the result had changed. The gap between that and the real clicks is what the change is worth.

A worked example

A staff rota app changes the title of its /employee-scheduling page on day 0. The change targets five searches, all variations on employee scheduling software and staff rota apps. The figures are illustrative.

The baseline for those five searches was 16,800 impressions and 336 clicks over 28 days, at an average position of 6.4. Divided by four, that's 4,200 impressions and 84 clicks a week: a CTR of 2.0%.

CTR is rounded to one decimal place; day 14, for example, is 118 ÷ 4,300 = 2.74%.

Reading it window by window:

• Day 7: CTR is up, but Google only started showing the new title partway through the week, so this is a partial read.
• Day 14 and day 30: CTR has settled at about 2.9%, from 2.0%. Position hasn't moved. The title changed how many people clicked, not where the page ranked.
• Day 60: this is the window that would fool you. Clicks are 162 against a baseline of 84, nearly double. But impressions are up by a third too (5,600 against 4,200). Six similar pages on the same site, which weren't changed, also gained about 30% more impressions that week: it's seasonal demand. At the old CTR the page would have had 112 clicks, so of the 78 extra clicks, about 28 came from demand and about 50 from the title.
• Day 90: demand is back to normal and the page still gets 37 more clicks a week than the old CTR would give, about 45% more. The effect has lasted.

The rule, written down before the change, was: keep the title if CTR on the targeted searches is at least 10% higher than the baseline at day 30 and position hasn't worsened by more than one place; revert if CTR is below the baseline at both day 14 and day 30; otherwise wait for day 60. At day 30, CTR was 2.9% against 2.0%, about 45% higher, with position slightly better. Keep.

Set decision thresholds before you look

If you choose the metric after seeing the data, you'll choose the one that looks best. Before each change, write down:

• the metric that should move (CTR for a title, position for links),
• the window where you'll decide,
• what counts as keep, revert and wait.

On pages with few clicks, add a floor in clicks as well as a percentage. A page with 10 clicks a week can move 30% on one good day. If the numbers are that small, wait for the later windows or judge several similar changes together.

Confounders: what else moved

Before crediting the change, check for these:

• Seasonality. Compare with the same weeks last year, or with a comparison group of similar pages you didn't touch, as in the example.
• Google updates. Google announces its larger ranking updates. If one lands inside a window, lean on the comparison group, because it was hit by the same update.
• Other changes to the same page. A new price, new photos, a template change, a colleague's edit. Keep a log of every change to every page with its date, and leave a page alone while its change is being measured.
• Links bought at the same time. A title change and new links to the same page in the same month can't be measured apart. Stagger them, or accept that you're measuring the two together and say so.

Before and after isn't proof

A before-and-after comparison shows what changed, not why. Without something to compare against, you can't separate the change from everything else that happened that month.

A comparison group gets you most of the way. Pick pages like the changed one (same template, similar searches) that you left alone, and measure them over the same windows. If the changed page moved and they didn't, the change is the likely cause. In the example, the comparison pages gained impressions at day 60 but their CTR didn't move, which is what points to the title.

That is strong evidence, not proof. The strongest check is to undo the change and see whether the result drops back. It costs you the gain for a while, so save it for a change you plan to repeat across hundreds of pages. It's much cheaper when every change is logged with what it replaced: Ship/Scale logs each change that way, and its rollback restores the original exactly and refuses if the page has changed since.

Ship/Scale also measures every change it makes on this schedule: against the 28 days before, at 7, 14, 30, 60 and 90 days, for position, clicks and impressions on the searches the change targeted.

Where this goes wrong

• Low-traffic pages. With a handful of clicks a week, every window swings. Group similar changes, or rely on the 60 and 90 day windows.
• Google shows a different title. Google sometimes rewrites the title it shows in results. Search the targeted queries and check that what you wrote is what searchers see.
• Google hasn't seen the change. Until the page is recrawled, you're measuring the old version. The URL Inspection tool in Search Console shows when Google last crawled the page.
• The set of searches drifts. After a change, the page may start appearing for new searches that weren't in the baseline. Report them separately as new searches rather than folding them into the comparison.
• Search Console hides some searches. Very rare searches are left out for privacy, so query-level totals are lower than page totals. Compare like with like: queries with queries, pages with pages.
• Day 90 isn't the end. Links and substantial rewrites can still be moving at 90 days, and a gain can fade later. A good result at day 90 is a reason to keep, not a reason to stop watching.

The key points

Take the 28 days before the change as the baseline, then read the 7 days ending at day 7, 14, 30, 60 and 90, each about three days after it closes. Measure only the searches the change targeted, and judge it on the metric it was meant to move, with the rule written down in advance. Check for seasonality, Google updates, other edits and links bought at the same time, ideally against a comparison group of untouched pages. Treat a before-and-after gain as evidence, not proof.