Blocking robots: what you gain and what you lose
Fix guide · Access to your site
Lots of council websites block robots to stay safe. Done well, that keeps bad robots out. Done badly, it also blocks Google, screen readers and helpful checks. This guide helps you get the balance right.
Why it matters
Blocking robots keeps bad ones out. But it can also stop Google, screen readers and helpful checks from seeing your pages.
What the problem is
Your website shows some visitors an "Are you human?" page, or an "Access denied" page, instead of the page they asked for. It is there to stop bad robots. But it can catch good visitors too.
What you gain
- Bad robots cannot copy your pages or fill in your forms with spam.
- Your site stays fast when lots of robots visit at once.
- AI tools you have not agreed to cannot take your words.
What you can lose
- Google or Bing can be blocked by mistake. Then fewer people find your pages.
- "Are you human?" pages can be hard for people who use a screen reader, or a slow phone.
- People on a work or school network, or a VPN, can be locked out.
- Checks that could help you, like link checkers and our free monthly check, cannot see your pages.
Would you like your council in our monthly check? Talk to David. He can share how our checker works, so your web team can let it in safely.
Why it happens
- A security tool was switched on to stop attacks or spam, with strict settings.
- The tool blocks anything that does not look like a person using a normal browser.
- Nobody checked who else it stops, such as search engines, screen readers, people on work networks or helpful checks.
A quick check you can do now
Open your page on a phone, on a work network and with a screen reader. If you see an "Are you human?" page, ask your web team who it stops.
Read the whole guide free
Sign in with your email. It takes a minute. You get:
- how to find the gaps on your own pages
- how to fix them, step by step
- how to do it in Drupal and LocalGov Drupal
- councils that get this right, to copy
We send you the monthly newsletter too. You can stop it any time.