A/B Testing a Consent Banner Measures the Banner, Not Consent
A team runs an A/B test on its consent banner, variant B lifts the accept rate by ten points, and the test is declared a win. The problem is what the test measured. It measured which banner gets more people to click accept, and a banner that makes accept big, bright, and one click, while reject hides behind a second screen, will always win that test.
That is not a finding about consent. It is a finding about friction. GDPR Article 7 requires consent to be freely given, specific, informed, and unambiguous, and the European Data Protection Board's Guidelines 05/2020 say the choice has to be genuine. A banner that wins by making one option harder to choose is not measuring preference. It is measuring the design.
The number that lies
A fair test holds the choice constant and varies the presentation honestly. Both options get equal prominence: same size, same colour weight, same number of clicks. The test changes the wording, or the order, or the timing, and measures whether people understand the choice better, not whether more of them pick the option the business prefers.
Measure the reject rate alongside the accept rate, and measure the time people take to decide. A banner that people decide on in two seconds is a banner people did not read. A banner that takes ten seconds and still gets a clear choice is a banner that did the job. Speed is not the metric. Understanding is.
What a fair test looks like
Regulators have been clear that equal prominence is not optional. CNIL guidance treats a banner that makes refusal harder than acceptance as a failure of freely given consent, and enforcement has followed banner design, not just banner absence. The fine is not for having a banner. It is for having a banner that steers.
That changes the risk calculation for an A/B test. A variant that lifts accept rates by making reject harder is not just a bad test. It is evidence, written down, in your own experiment log, that the team knew the design steered people and shipped it anyway. An experiment log is a document. It can be read.
Measure the rate at which people reach a settings screen and make a granular choice, because that is the behaviour of someone who understood the question. Measure the complaint rate and the withdrawal rate after the test, because a banner that tricks people into accepting generates withdrawals later. Measure the share of visitors who choose at all, rather than closing the banner or ignoring it.
The regulators have noticed
The metric that matters most is the one no dashboard shows by default: whether the consent you collected would survive being read back to the person who gave it. If the answer is no, the accept rate is not a win. It is a liability with a chart attached.
Keep reading: Consent withdrawal in one click: the Article 7 test, Consent banners on subdomains: one banner or one per subdomain.
What to measure instead
Frequently asked questions
Can I A/B test a consent banner?
Yes, but hold the choice constant and vary the presentation honestly. Both options need equal prominence, the same size, the same colour weight, and the same number of clicks. A test that makes accept easier than reject measures friction, not preference.
What does freely given consent mean for banner design?
Under GDPR Article 7, consent must be freely given, specific, informed, and unambiguous. The EDPB Guidelines 05/2020 say the choice has to be genuine, which means refusing must be as easy as accepting.
Why is a high accept rate not always good?
Because it may show the banner steered people rather than informed them. A banner that makes accept big and bright and hides reject behind a second screen will always lift accept rates, and regulators treat that steering as a failure of consent.
What should I measure instead of the accept rate?
The reject rate alongside it, the share of visitors who make a granular choice in settings, the time people take to decide, and the withdrawal rate afterwards. A banner that tricks people into accepting generates withdrawals later.