If you run a website that serves EU or Californian users, regulators may already have scanned it, and you would not know. This article explains how regulators use automated scanning to find violations: which tools they use, what those tools actually detect, and how to read your own site the way an enforcement crawler does. Knowing this lets you fix what is visible from the outside before a complaint or a sweep turns it into a case file.
From Complaints to Crawlers: How Enforcement Changed
For years, website enforcement depended on complaints. A user noticed something, wrote to a data protection authority, and an investigator looked at the site by hand. That model still exists, but it no longer sets the pace.
The change began with tools that collect evidence in a repeatable way. In 2019 the European Data Protection Supervisor released the open-source Website Evidence Collector. It is built on Node.js and Puppeteer, loads a page in a headless browser and records the cookies, local storage entries and third-party requests that appear before any user interaction. In 2024 the EDPB published its own open-source website auditing tool for supervisory authorities. These tools produce timestamped, reproducible evidence, and that kind of evidence holds up in proceedings.
Consumer protection works the same way. The European Commission’s EU eLab, launched in 2022, gives national consumer authorities a shared environment for crawling, capturing and analysing websites. Coordinated CPC network “sweeps” now examine hundreds of sites at once. In the 2023 sweep on dark patterns, authorities checked 399 online shops and found at least one manipulative practice on 148 of them.
What Automated Scanning Actually Detects
Most people picture a crawler that reads privacy policies. Enforcement scanners are interested in behaviour more than text. The checks that come up most often are these:
– Cookies or identifiers set before consent, especially analytics and advertising cookies from Google, Meta or TikTok
– Network requests to third-party domains that fire on page load, regardless of banner state
– Whether rejecting cookies is as easy as accepting them, measured by clicks and button presence on the first layer
– Missing or broken links to the privacy policy, imprint or accessibility statement
– Whether opt-out signals such as Global Privacy Control are honoured
The rejection-button check is a good example of how this becomes enforcement. In January 2022 the French CNIL fined Google €150 million and Facebook €60 million because refusing cookies took more steps than accepting them. That is exactly the asymmetry a scripted browser can count.
In the US, the California Attorney General’s 2022 settlement with Sephora, $1.2 million, turned partly on the company’s failure to treat Global Privacy Control as a valid opt-out. Testing GPC is trivial to automate. You send the Sec-GPC: 1 header and watch whether ad-tech requests still fire.
Accessibility monitoring is the oldest structured example. Commission Implementing Decision (EU) 2018/1524, under the Web Accessibility Directive, requires member states to run “simplified monitoring” on large samples of public sector sites, mostly with automated tools, alongside a smaller in-depth review. Since the European Accessibility Act began applying on 28 June 2025, market surveillance authorities have had similar reasons to scale up checks of private ecommerce sites.
The Myth That Small Sites Fly Under the Radar
The most persistent misconception is that regulators only look at big brands. Automated scanning removes the cost of looking. Once a crawler is configured, checking 10,000 domains costs little more than checking 10.
Advocacy groups showed this first. In May 2021 the privacy NGO noyb scanned thousands of European websites with an automated tool and sent 560 draft complaints about non-compliant cookie banners, many to mid-sized companies. The EDPB set up its cookie banner taskforce in September 2021 to coordinate the follow-up and published its report in January 2023.
Size still affects how high a fine is. It no longer decides whether you get noticed.
A Realistic Scenario: The Tag Manager Change Nobody Reviewed
Here is a pattern compliance teams see all the time. A regional retailer passes a consent audit in spring. In September, the marketing team adds a new conversion pixel through Google Tag Manager and sets it to fire on “All Pages” instead of waiting for the consent trigger. Nobody on the legal side sees that change.
Three weeks later, a national authority runs a sweep of retailers in that sector. The crawler loads the homepage with an empty profile, records a Meta Pixel request carrying a hashed identifier before any banner interaction, and saves a HAR file with a timestamp. The retailer’s first sign of trouble is a letter asking for an explanation within 30 days.
The banner looked perfect the whole time. What changed was the firing order of a third-party tag. Teams that review third-party scripts and their hidden data flows catch this kind of problem. Teams that only look at the banner miss it.
How to Scan Your Own Site Like a Regulator
An experienced privacy engineer replicates the regulator’s method rather than guessing at it. The process takes about an hour per key template:
Step 1: Open a fresh browser profile with no cookies, ideally from an EU IP address. A cached consent state makes every result worthless.
Step 2: Load the homepage, a product or service page and the checkout or contact form. Record network traffic in Chrome DevTools or run the Website Evidence Collector from the command line.
Step 3: Before you touch the banner, list every cookie and every request to a third-party domain. Anything that is not strictly necessary is a finding.
Step 4: Click “Reject” and reload. Verify that marketing tags stay silent on later pages, not just the first one.
Step 5: Repeat with the GPC signal enabled, and on a mobile viewport. Mobile templates often load a different tag configuration.
Save the output with dates. If a regulator does make contact, being able to show the documentation regulators actually request changes the tone of the conversation.
Common Mistakes When Preparing for Automated Enforcement
Testing with a logged-in or consented browser. Developers check their own site from a profile that accepted cookies months ago. The scan comes back clean, and it is wrong.
Trusting the CMP dashboard instead of the network log. OneTrust, Cookiebot and Usercentrics report what they are configured to block. Hard-coded scripts in the theme, or tags added outside the consent manager, will not appear there. Only the network tab shows what really fires, which is the gap described in technical versus visual cookie consent failures.
Auditing once a year. A clean audit in March tells you nothing about the deployment in August. Regulator sweeps do not follow your audit calendar.
Some exceptions apply. Purely B2B intranets behind authentication are rarely crawled. Sites serving only non-EU, non-Californian markets face different, often lighter, scanning pressure, although Brazil’s ANPD and the UK ICO have both signalled interest in cookie enforcement. Sites for children or in regulated sectors such as health and finance should expect closer scrutiny.
FAQ
Can a regulator open an investigation based only on an automated scan?
Yes. Automated evidence such as timestamped cookie logs and HAR files is routinely used to start inquiries. Formal decisions usually follow manual verification and a chance for the company to respond.
How often do authorities run these sweeps?
The EU CPC network runs coordinated sweeps roughly once a year on a chosen theme. Individual data protection authorities such as the CNIL run their own campaigns, often announced in their annual priorities.
Do regulators scan mobile apps too?
Increasingly, yes. The California Attorney General ran a sweep of mobile apps in 2023, and app traffic analysis uses similar network-interception techniques.
Summary: Fix What Is Visible From the Outside
Regulators and advocacy groups now find violations with headless browsers, network logs and reproducible scripts, not with complaints alone. What they detect is behaviour: tags firing before consent, reject buttons hidden on a second layer, ignored opt-out signals and missing legal pages. Run the same fresh-profile scan on your key templates after every significant deployment, keep the dated results, and you will see what a regulator’s crawler sees before they do.