401 vs 403: What the Difference Means for SEO

Disclosure: some links on this page are affiliate links. If you sign up through one we may earn a commission at no extra cost to you. We only recommend tools we would genuinely use.

From the outside, a 401 and a 403 look identical: the visitor asks for a page and the server says no. The difference decides whether that refusal is a request for credentials or a flat denial, and Google reads both in ways that catch site owners out. This guide covers the 401 vs 403 distinction in plain English: definitions, Googlebot’s treatment, diagnosis, fixes, and when each is the right choice.

401 vs 403: what does each status code mean?

A 401 Unauthorized response means the server does not know who you are: the request lacks valid credentials, and it may succeed once you sign in. A 403 Forbidden response means the server understood the request, may know exactly who you are, and still refuses to authorise it. In short, 401 says “prove your identity” and 403 says “no, regardless”.

Typical 401 situations: a staging site behind HTTP authentication, a portal with an expired session, an API called without a token. Typical 403 situations: a logged-in user opening an admin URL without the right role, a firewall blocking an IP range, hotlink protection on images, or file permissions denying a directory.

	401 Unauthorized	403 Forbidden
Meaning	No valid credentials supplied	Request understood, access refused
Server’s message	”Sign in, then try again"	"The answer is no”
Required header	WWW-Authenticate challenge	None
Does retrying help?	Yes, with valid credentials	No, not until permissions change
Typical causes	HTTP auth, login walls, expired tokens	Firewall rules, permissions, IP or geo blocks
Google’s treatment	Content treated as non-existent	Content treated as non-existent

What does the HTTP specification say about 401 and 403?

RFC 9110, the current HTTP semantics standard, defines 401 Unauthorized as a request that “lacks valid authentication credentials for the target resource”, and requires the server to send a WWW-Authenticate header telling the client how to authenticate. It defines 403 Forbidden as meaning the server “understood the request but refuses” to authorise it.

That matters for one practical reason: a status code is a claim, not a guarantee. Plenty of servers and CDNs return a 403 status for what is really an authentication problem, so when you diagnose an access error, trust what the server is actually doing over what the code implies.

How does Google treat 401 and 403 status codes?

Google treats 401 and 403 the same way as almost every other client error: as a sign the content does not exist. Google’s documentation states that all 4xx errors except 429 are treated alike, the crawlers tell the indexing pipeline the content is gone, and a previously indexed URL is removed from the index.

Crawling of the affected URLs then gradually decreases, and a misfiring firewall can quietly deindex sections of a site while everything looks fine in a browser. Googlebot also never provides credentials, as Google’s Search Console documentation puts it: it does not fill in login forms or answer a WWW-Authenticate challenge, so anything behind a 401 stays out of Google permanently. Ideal for staging, disastrous on a money page.

One warning from Google itself: in a February 2023 Search Central blog post, Gary Illyes asked site owners and CDNs to stop returning 403s and 404s to slow Googlebot down. 4xx codes other than 429 have no effect on crawl rate; they only get content removed. If you need Googlebot to back off, return a 429.

How do you diagnose a 401 or 403 error?

To diagnose a 401 or 403, check three places: a curl request for the live response code, Google Search Console for what Googlebot received, and your server logs for who is being refused and by which layer. A browser check alone is not enough, because bot protection often treats crawlers differently from humans.

Start with curl from a terminal:

curl -I -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" https://www.example.co.uk/page/

The first line of the output shows the status code, and a 401 should also show a WWW-Authenticate header. Many firewalls verify Googlebot by IP address though, so a spoofed user agent from your own machine may be treated differently from the real crawler. The URL Inspection live test in Search Console is the tiebreaker: it fetches with Google’s own infrastructure.

In Search Console, the Page indexing report groups these URLs under “Blocked due to access forbidden (403)” and “Blocked due to unauthorized request (401)”. A sudden jump in either bucket usually points at a firewall or CDN rule change rather than anything you edited on the site.

How do you fix an unwanted 401 or 403?

Fixing an unwanted 401 or 403 almost always means fixing a security layer, not the page itself. Work through the checks below in order, then use the URL Inspection live test to confirm Googlebot receives a 200 before requesting reindexing in Search Console:

• Firewall or CDN bot rules. Verify Googlebot against Google’s published IP ranges and allow it explicitly; user-agent allowlisting alone is unreliable because anyone can fake the string.
• HTTP authentication left switched on. A password prompt from a launch or migration sometimes survives on a live directory. Remove it from anything public.
• File permissions and deny rules. Check .htaccess or server config for deny directives catching more paths than intended.
• Country-level blocking. If your firewall blocks traffic by geography, confirm the rule is not catching search engine crawlers.
• Hotlink protection. Overly strict referrer rules can serve images a 403, which hurts image indexing.

When should you use a 401 or 403 deliberately?

Use a 401 for anything that genuinely requires a login, and a 403 for anything refused outright no matter who asks. A staging environment behind HTTP authentication is the classic deliberate 401: the whole tree returns 401 to crawlers and visitors alike, and pre-launch content never reaches the index.

A 403 is the honest answer for blocked IPs, abusive traffic and resources that should never be served to that requester. It is not a substitute for noindex: if a page should stay public for users but out of search results, use a noindex robots meta tag, because a 403 locks humans out too. And never use either code for rate limiting; that job belongs to 429.

What else do people ask about 403 vs 401?

The questions below cover the 401 and 403 points site owners raise most often: SEO impact, deindexing, crawl rate, Search Console labels and staging setups. Each answer stands on its own, and the sections above hold the fuller detail behind them.

Is a 403 status bad for SEO?

Only when it is unintentional. Google treats a 403 as content that does not exist, so an indexed URL that starts returning it will drop out of the index. A deliberate 403 on private or blocked resources is correct behaviour.

Does a 401 remove a page from Google’s index?

Yes. Google’s documentation says all 4xx responses except 429 signal that the content does not exist, and previously indexed URLs are removed. A page behind a 401 will fall out of the index and stay out.

Should I return 401 or 403 to slow Googlebot down?

Neither. Google stated in 2023 that 4xx codes other than 429 have no effect on crawl rate and only lead to removal from Search. Return a 429 if the server is genuinely overloaded.

Why does Search Console say “Blocked due to access forbidden (403)”?

It means Googlebot received a 403 for those URLs, even if the pages open normally in your browser. The usual culprit is a firewall, CDN or bot-protection rule. Verify Googlebot by IP and allow it through.

Is returning a 403 the same as adding noindex?

No. Both keep a page out of Google, but a 403 blocks human visitors as well, while noindex leaves the page usable and simply asks search engines not to index it. For public pages you want excluded from search, noindex is the right tool.

Which code should a staging site return, 401 or 403?

A 401, delivered by HTTP authentication. It keeps every crawler and passer-by out while your team signs in with a username and password, and nothing on the staging tree can reach the index.

---

Status codes change silently when a firewall rule or plugin update goes wrong, so regular crawls beat one-off checks. SE Ranking’s Website Audit reports which HTTP status codes your crawled pages respond with and flags 4xx URLs as issues (as of July 2026, per SE Ranking’s feature documentation). Start Your Free Trial