Structured data over time: testing, monitoring and regressions

Adding structured data to a site takes a few days. Keeping it correct is a job that never ends. Templates change, CMS fields get renamed, a plugin updates and duplicates the markup, engines modify the requirements for rich results. Without a checking process, after a year many sites' markup is partly broken without anyone having noticed.

Errors and warnings are not the same thing

Validation tools distinguish two levels:

Errors must always be fixed. Warnings should be assessed: adding a recommended property only makes sense if the data really exists and appears on the page. Inventing a value to make a warning disappear is worse than leaving it.

Two tools with different purposes

• Schema.org vocabulary validators: check that the markup is syntactically correct and that types and properties exist in the vocabulary.
• Search engines' rich result tests: check whether the markup meets that engine's specific requirements for a given result type.

Markup can be perfectly valid for Schema.org and eligible for no rich result at all, because the engine requires additional properties or doesn't support that type. It's worth using both: the first for correctness, the second for the effect in search.

The most common regressions

In my work, when markup that used to work stops working, the cause is almost always one of these:

1. empty CMS field: the template outputs "price": "" or "author": {"name": ""} because an editor left a field blank;
2. duplicate markup: a theme and a plugin both generate a Product or Organization block, with different values;
3. changed format: a date that switches from 2026-03-12 to 12/03/2026, a price with the currency symbol inside the number;
4. data no longer visible: reviews or prices removed from the page in a redesign but left in the JSON-LD;
5. relative or outdated URLs: images and logo pointing to the previous domain after a migration;
6. unhandled characters: quotes or line breaks in text that break the JSON if the template doesn't encode them.

Automated tests in the build

The most reliable way to prevent regressions is to check the markup before it reaches production. A simple approach:

1. choose a representative sample of URLs, one page per template;
2. in the build pipeline or a nightly test, download the HTML of those pages from staging;
3. extract the <script type="application/ld+json"> blocks and check they are valid JSON;
4. check that the essential properties for each type are present and not empty;
5. make the test fail if anything is missing.

A minimal check takes just a few lines:

import json, re, sys, urllib.request
html = urllib.request.urlopen(sys.argv[1]).read().decode()
blocks = re.findall(r'<script type="application/ld\+json">(.*?)</script>', html, re.S)
for b in blocks:
    data = json.loads(b)
    if data.get("@type") == "Product":
        assert data.get("name"), "Product without name"
        assert data.get("offers", {}).get("price"), "Product without price"
print("ok", len(blocks), "blocks")

It doesn't replace the official validators, but it catches the most frequent breakages immediately.

Monitoring in production

Webmaster tools offer reports for each detected structured data type, with the number of valid items, items with warnings and invalid ones. It's worth:

• checking them after every release that touches templates;
• watching the trend over time: a sudden drop in valid items signals a regression;
• using email notifications for new errors.

Consistency with visible content

Structured data must describe what the user sees on the page. Ratings that aren't shown, prices different from those displayed, events that have already ended, reviews written by the site about itself: these are guideline violations that can lead to the loss of rich results or to a manual action. The most useful automated check, where possible, is comparing the value in the JSON-LD with the one extracted from the visible HTML.

Changing requirements

Engines periodically update the rich result types they support and their requirements; some have been reduced or retired over time. Correct markup that is no longer used does no harm, but it shouldn't be regarded as a guarantee of visibility. Review the official documentation for the types you use at least once a year.

In short

• Errors exclude items from rich results; warnings should be assessed without inventing data.
• Use both a Schema.org validator and the engine's tests: they measure different things.
• Regressions come from empty fields, duplicates, formats and redesigns.
• Add an automated test on the JSON-LD of the main templates.
• Keep markup consistent with visible content and review the requirements every year.