How a lowercase redirect breaks indexing of URLs with Cyrillic and other non-ASCII characters

“Redirect all URLs to lowercase” is a common piece of technical SEO advice. It removes duplicates like /Page/ and /page/, and for plain Latin URLs it works. I tested it on a Cyrillic URL, and in a week Googlebot never reached the page, even though it opened in a browser without any trouble.

Below I break down how this loop works, show how to set up a lowercase redirect without it, and how to check your own site. The full test data with server logs is in a separate test card.

How non-ASCII characters get into a URL

Cyrillic, Georgian or Latin letters with diacritics can’t sit in a URL as they are. The browser turns each character into UTF-8 bytes and writes each byte as % followed by two hex digits. The Cyrillic letter “т” becomes %D1%82, and é becomes %C3%A9.

The hex digits A–F can be written in upper or lower case. Under RFC 3986, %D1%82 and %d1%82 mean exactly the same thing. The standard recommends uppercase, and its section on normalization says to convert other spellings to uppercase.

The first byte of any non-ASCII character in UTF-8 has a value from C2 to F4. So the encoding of every such character contains at least one hex letter. The same goes for encoded symbols such as a comma %2C or a colon %3A.

How the simplest lowercase redirect works

The simplest implementation takes the raw request string and converts it to lowercase. In nginx this is the $request_uri variable, which keeps the URL exactly as the client sent it. If the lowercased string differs from the original, the rule responds with a 301.

For a Latin URL without encoding everything is fine: /Page/ becomes /page/. With non-ASCII URLs the rule behaves differently. Here is what it does with three examples:

Request	Server response	Result
/%D1%82%D0%B5%D1%85… (“технічний”, standard encoding)	301 to /%d1%82%d0%b5%d1%85…	Same letters, only the hex case changed
/%D0%A2%D0%B5%D1%85… (“Технічний”, capitalised)	301 to /%d0%a2%d0%b5%d1%85…	The capital “Т” stayed capital
/caf%C3%A9/	301 to /caf%c3%a9/	Same letters, only the hex case changed

The table shows two problems at once. The rule doesn’t lowercase Cyrillic letters, because they are encoded, so it misses its own goal. Instead it changes the case of the hex digits and creates a redirect where the URL was already correct.

The cause-and-effect chain

In this story the server only answers with a redirect. The loop is closed by a crawler that normalizes URLs according to RFC 3986 before each request. For Googlebot, events go like this:

1. Googlebot requests the standard URL /%D1%82… and gets a 301.
2. The Location header holds the URL with lowercase hex /%d1%82….
3. Before the next request Googlebot normalizes it and gets /%D1%82… again.
4. The server sees the same URL and answers 301 again.
5. After a few attempts Googlebot stops without ever getting the content.

A browser behaves differently. In my test, Chrome took the URL from Location as is and didn’t change the hex case. It followed one redirect, got a 200 and showed the page, so a person doesn’t notice the problem.

What the test showed

I reproduced this rule for one page and collected nginx logs for a week. I gave Google the lowercase-hex URL in three ways: through URL Inspection, through a separate sitemap and through the Location header. Here are the key facts:

• Google requested the page 36 times, and all 36 requests went to the uppercase-hex form;
• in URL Inspection each pass through the loop took six requests, separately for the mobile and desktop agent, and each regular Googlebot visit took three;
• Google never requested the lowercase-hex form, even when it took the URL from the sitemap;
• Chrome, ClaudeBot and QlyzeBot got the page with a 200.

A table of all requests and sample log lines is in the test card.

How I checked that it really was Google

A User-Agent is easy to fake, so I checked the IP addresses the way Google recommends. Reverse DNS for each address returns a name in the googlebot.com domain, and a forward lookup of that name returns the same IP. One “bingbot” request failed this check, so I left it out.

How to set up a lowercase redirect safely

A lowercase redirect isn’t harmful in itself. The problem is that the simplest rule works on the encoded string. A safe implementation decodes the path first, then lowercases the letters, and only then encodes the result back.

Here is that logic in JavaScript. It is easy to port to server middleware in another language:

const decoded = decodeURI(path);       // "/Технічний-аудит/"
const lower = decoded.toLowerCase();   // "/технічний-аудит/"
if (lower !== decoded) {
  redirect(301, encodeURI(lower));     // "/%D1%82%D0%B5…", uppercase hex
}

The encodeURI function always returns the standard form with uppercase hex digits. So a URL that differs only in hex case gets no redirect, and a URL with a capital Cyrillic letter gets a proper redirect to the lowercase one. If your URLs contain encoded reserved characters such as %2F, this logic needs more work: decodeURI doesn’t decode them, and encodeURI will encode their % a second time.

What to check after setting it up

The rule alone doesn’t guarantee that Googlebot gets the page on the first request. The URL has to be the same everywhere a search engine sees it. Check a few things:

• canonical, the sitemap and internal links use the standard form with uppercase hex;
• the uppercase-hex URL answers 200 or leads to a 200 page in one step;
• the rule doesn’t touch query parameters if they contain case-sensitive values.

That way every page has one standard URL, and Googlebot gets it on the first request.

How to check your site

The check takes a few minutes and needs no server access. Take a page with Cyrillic, diacritics or an encoded symbol in its URL. Then follow these steps:

1. Request it with curl -sI in the standard encoding with uppercase hex.
2. Look at the response code and the Location header.
3. If the response is a 301 and Location contains lowercase hex, your site has this problem.

For example, for my test page the command curl -sI "https://marko-f.com/%D1%82%D0%B5%D1%85%D0%BD%D1%96%D1%87%D0%BD%D0%B8%D0%B9-%D0%B0%D1%83%D0%B4%D0%B8%D1%82/" returns a 301 and a Location with lowercase hex. Server logs help confirm the problem. If Googlebot requests the same URL several times in a row and gets a 301 each time, it’s the same loop.

Sources

The mechanics rely on the URI standard and Google’s documentation. I took the behaviour of the browser and crawlers from my own server’s logs. Here are the primary sources the text refers to:

• RFC 3986: section 2.1 on percent-encoding and section 6.2.2.1 on normalizing the case of hex digits;
• Google: Redirects and Google Search: how Google handles 301 redirects;
• Google: Verifying requests from Google’s crawlers: reverse and forward DNS lookup;
• MDN: encodeURI: how JavaScript encodes URLs.

The test data was collected on the marko-f.com server from 17 to 24 October 2026.