Decoding URL Syntax: John Mueller Clarifies That Raw and Percent-Encoded Brackets Create Entirely Different URLs for Google Search

By Search Industry News Desk
Published: October 2023

Main Facts

In a recent clarification that has sent ripples through the technical search engine optimization (SEO) community, Google Search Advocate John Mueller revealed a critical nuance regarding URL path and query string parsing. According to Mueller, raw square brackets ([ and ]) and their percent-encoded equivalents (%5B and %5D) are treated by web standards and search engine crawlers as fundamentally different URLs.

The revelation came via a discussion on the decentralized social media platform Bluesky, prompted by a technical inquiry from former Google employee Vanessa Fox. Mueller candidly admitted that he had previously operated under the assumption that raw and encoded brackets were functionally equivalent, noting that he had "probably gave people the wrong answer on this in the past."

While Googlebot will crawl both variations if they are exposed to the web, the search engine relies on its internal deduplication systems. If Google discovers that the raw-bracket version and the percent-encoded version serve identical content, it will attempt to consolidate the signals, choose a preferred canonical version, and drop the other from its active index. However, leaving this choice up to Google’s automated algorithms introduces unnecessary risk.

Mueller strongly advises webmasters, developers, and SEO professionals to adopt a strict, uniform URL format across all site assets—including internal links, XML sitemaps, rel="canonical" tags, and structured data markup. Furthermore, this logic extends beyond brackets: identical rules regarding soft-equivalence and deduplication apply to spaces, plus signs (+), and their percent-encoded counterpart (%20).

Chronology of the Discovery

The technical debate surrounding URL encoding did not happen in a vacuum; it unfolded through a series of public exchanges and newly launched video series:

• The Genesis (Early October 2023): Google launched a targeted video series titled Non-Canonical Cabinet to address esoteric, edge-case technical SEO questions. The first episode tackled how Google handles emojis and non-English characters within URL paths.
• The Follow-Up Question: Following the release of the first episode, Vanessa Fox—famed for developing Google’s original Webmaster Central—raised a related question in the comments section regarding how Google processes bracketed query strings, such as those used in programmatic filtering (?filter[]=123 versus ?filter%5B%5D=123).
• The Bluesky Verification (Monday): Taking the conversation to Bluesky, Fox formally asked whether Google treats percent-encoded URLs differently from raw-bracketed URLs. She highlighted a JavaScript equality check demonstrating that comparing the two literal strings returns false.
• The Official Correction: John Mueller responded publicly on Bluesky, confirming the distinction based on internet engineering standards. He acknowledged his past misconceptions and laid out the exact mechanisms by which Google crawls, checks checksums, and de-duplicates these pages.
• Ongoing Developments: As of late October 2023, the SEO community awaits subsequent installments of the Non-Canonical Cabinet series, with the next episode slated to explore complex edge cases surrounding robots.txt directives.

Supporting Data and Technical Standards

To understand why Google treats raw and encoded brackets differently, one must look directly at the underlying internet architecture standards established by the Internet Engineering Task Force (IETF).

The Authority of RFC 3986

Mueller’s technical justification is anchored in RFC 3986, the official URI (Uniform Resource Identifier) Generic Syntax specification. Under Section 2.2 of the standard, certain characters are designated as "reserved." These characters—which include brackets, plus signs, question marks, and hashes—serve as delimiters within a URI structure.

RFC 3986 explicitly dictates that if a URI swaps a reserved character for its percent-encoded hexadecimal representation, the resulting URIs are not equivalent.

Invalid/Raw Query String:    ?filter[]=123
Valid/Encoded Query String:  ?filter%5B%5D=123

The IPv6 Exception

Adding to the complexity, RFC 3986 strictly restricts the use of raw square brackets within a URL to a very specific context: the hostname component, where they are utilized exclusively for wrapping raw IPv6 addresses (e.g., http://[2001:db8::1]/). Using raw brackets inside a query string or a standard URL path is technically a violation of URI syntax standards.

Case Sensitivity in Hex Encoding

Mueller also clarified a minor detail regarding hexadecimal encoding syntax. While the distinction between raw and encoded characters matters immensely, the letter case within the percent-encoding itself does not. Therefore, %5B and %5b are treated as entirely equivalent because they strictly adhere to RFC 3986 rules governing hexadecimal characters.

Official Responses and Expert Insights

The dialogue between industry veterans John Mueller and Vanessa Fox highlights the subtle gaps that often exist between theoretical web standards and practical implementation.

John Mueller’s Guidance on Crawling and Deduplication

Detailing how Googlebot processes sites that utilize both formats, Mueller explained:

"That said, maybe your server makes them ‘soft-equivalent’ by serving the same content. We’d crawl both versions, if we see the same content (maybe even checksum level), we’d de-dupe them and pick one."

Despite this automated safety net, Mueller warned against complacency. Browsers often automatically encode raw brackets before transmitting a user’s request, but client-side code—such as JavaScript applications, API calls, or specific internal linking structures—may inject raw brackets directly into the request stream. Mueller’s golden rule for developers is straightforward: "Always encode it, never show []."

Google’s Existing Documentation

Google’s official public documentation on URL structure and crawling already aligns with Mueller’s statements. The documentation specifies that Google Search fully supports URLs as defined by IETF STD 66 (which houses RFC 3986), explicitly commanding webmasters that “Characters defined by the standard as reserved must be percent encoded.”

The Reality of Legacy Implementations

When Vanessa Fox pointed out that many established enterprise websites rely on invalid raw bracket notations throughout their filtering architectures—and yet appear to index just fine—Mueller offered pragmatic reassurance.

He noted that he would not anticipate catastrophic ranking or indexing penalties for a mature site utilizing this structure, provided the server responds correctly. However, he maintained that he "wouldn’t set up a new site like this." For legacy sites burdened by raw brackets, Mueller advised performing rigorous audits to ensure internal links, XML sitemaps, canonical tags, and structured data objects are kept completely uniform to prevent mixed signals.

Implications for SEO Professionals and Web Developers

Mueller’s clarification carries several vital takeaways for technical SEOs, site architects, and engineering teams managing large-scale websites, particularly e-commerce platforms and dynamic directory sites that heavily utilize parameter filtering.

1. The Danger of Internal Link Fragmentation

If a website’s templates generate raw brackets in some navigation menus (e.g., ?category[]=shoes) while JavaScript or pagination components generate percent-encoded brackets elsewhere (e.g., ?category%5B%5D=shoes), the site creates internal link fragmentation. Even though Google may eventually recognize the pages as soft-duplicates, mixed signals force the search engine to waste valuable crawl budget discovering and processing redundant URL variations.

2. Cross-Crawler Vulnerabilities

Mueller issued a broader warning extending beyond Googlebot. Drawing on Google’s historical engineering challenges over the past decade, he noted that alternative search engine crawlers, third-party log analyzers, uptime monitors, and indexing programs may not possess Google’s sophisticated logic for handling semantic "soft-equivalencies." If a secondary crawler fails to equate raw and encoded brackets, it may experience crawling failures, fail to index important filter pages, or report false-positive duplicate content issues.

3. Actionable Best Practices for Technical SEOs

To future-proof web architectures against URL normalization discrepancies, technical SEOs should implement the following protocols immediately:

• Enforce Uniform Encoding Rules: Configure web frameworks, content management systems (CMS), and routing engines to systematically percent-encode all reserved characters in query parameters and paths.
• Audit Canonical Implementations: Ensure that the URL declared within rel="canonical" tags matches the exact format of the requested URL down to the character encoding. Never point a raw-bracket URL to an encoded-bracket canonical tag (or vice versa) if it can be avoided.
• Monitor Server-Side Redirects: If transitioning a site from raw brackets to percent-encoded brackets, implement explicit 301 permanent redirects to guide users and search bots seamlessly from the legacy format to the standardized format.
• Clean Up XML Sitemaps: Ensure that every URL injected into an XML sitemap uses the standardized, percent-encoded format to signal a clear preference to Googlebot from the earliest stages of discovery.

As search engine algorithms grow increasingly strict regarding clean, standards-compliant web architecture, Mueller’s insights serve as a timely reminder that adherence to fundamental internet standards like RFC 3986 remains an indispensable pillar of advanced technical SEO.