Introduction
URLs connect nearly every web interaction, but their syntax assigns special meaning to characters such as ?, &, #, / and =. A user value containing one of these symbols can accidentally change the structure of a request unless it is represented safely.
Percent-encoding converts bytes that cannot appear literally into a percent sign followed by two hexadecimal digits. Browsers, API clients, routers and OAuth systems use it constantly. The utily.tools URL Codec provides a quick way to encode or inspect these values.
Reserved characters and UTF-8 bytes
RFC 3986 divides URI characters into unreserved, reserved and percent-encoded forms. Letters, digits, hyphen, period, underscore and tilde are generally unreserved. Reserved characters act as delimiters and should only remain literal when their structural meaning is intended.
Non-ASCII characters are first encoded as UTF-8 bytes. Each byte is then written as %HH. The character ç becomes the UTF-8 bytes C3 A7 and therefore appears as %C3%A7. Decoding reverses the hexadecimal byte representation and then interprets the byte sequence as UTF-8.
Technical foundations and behavior
Percent-encoding is component-dependent. A complete URI contains structural delimiters that must remain recognizable, whereas an individual path segment or query value treats many of those same characters as data. Applying the wrong character set can either destroy the URI structure or allow a value to alter it.
Component-aware encoding
Path segments, query names, query values and fragments occupy different grammatical positions. Each value must be encoded before it is assembled with structural delimiters; applying one rule to the complete address can escape required separators, while leaving data delimiters unescaped can enable parameter injection.
Spaces and form encoding
Generic percent-encoding represents a space as %20. The application/x-www-form-urlencoded format commonly uses + for spaces and percent-encodes a literal plus sign. Confusing these conventions can change values during decoding.
Double encoding
Encoding an already encoded percent sign transforms %20 into %2520. Systems should define exactly one encoding boundary and avoid repeatedly normalizing input, especially around redirects and signature calculations.
Real-world applications
Search parameters
Search text containing ampersands or hash signs must remain one query value instead of being interpreted as another parameter or fragment.
OAuth redirects
A callback URL is often itself embedded as a query parameter, so its delimiters must be encoded at the correct nesting level.
Internationalized content
Product names, addresses and user-generated text use UTF-8 percent-encoding when included in paths or query strings.
Standards and deeper technical reference
URI syntax and percent-encoded octets
RFC 3986 defines a URI as scheme, authority, path, query and fragment components. Percent-encoding represents one byte as percent followed by two hexadecimal digits. For non-ASCII text, the producer normally encodes the character as UTF-8 and percent-encodes each resulting byte. The unreserved set is ASCII letters, digits, hyphen, period, underscore and tilde.
Reserved characters have syntactic meaning. Gen-delimiters include colon, slash, question mark, number sign, square brackets and at sign; sub-delimiters include punctuation such as ampersand and equals. Decode only after splitting the URI into components, or an encoded delimiter can be reinterpreted as structure.
| Scope | Intended input | Important behavior |
|---|---|---|
| Complete URI | Scheme, authority, path, query and fragment together | Preserves structural delimiters needed to identify components |
| Component value | One path segment, query name or query value | Escapes delimiters that would otherwise change the surrounding grammar |
| Structured URL model | Separately represented URL components | Parses and serializes according to a defined URL standard |
| Form query model | Name-value pairs in form data | Uses application/x-www-form-urlencoded rules, including plus for space |
RFC URI rules versus the WHATWG URL standard
RFC 3986 provides generic syntax and normalization principles, while the state-machine-based WHATWG URL Standard defines interoperable handling of web URLs. Their models are related but not identical. Systems should parse components structurally instead of rebuilding addresses through unvalidated string concatenation.
The application/x-www-form-urlencoded format is another distinct layer: it traditionally represents spaces with plus signs. A literal plus sign must therefore be percent-encoded when form decoding is expected, whereas generic URI percent-decoding does not inherently translate plus into space.
Double decoding and canonicalization attacks
Encoding or decoding the same data twice can change meaning: %252F becomes %2F and then slash. Differences between a proxy, framework and application decoder can bypass routing or authorization checks. Normalize exactly once at a defined boundary, reject malformed percent triplets and run security validation on the same canonical representation used to access the resource.
Do not decode an entire URL before parsing it. Encoded slash, dot segments, NUL and path traversal sequences require special care when a URL is mapped to a filesystem. Host names use IDNA processing rather than ordinary percent-encoding.
Primary specifications and references
- RFC 3986: Uniform Resource Identifier Generic SyntaxIETF / RFC Editor
- WHATWG URL StandardWHATWG
- HTML form URL-encoded serializationWHATWG
Conclusion
URL encoding protects the boundary between URI structure and user data. The correct function depends on whether the input is an entire address, a path segment, a query value or form data.
In my opinion, the safest approach is to build URLs with structured APIs and use manual encoding tools for inspection and debugging. Explore values with the utily.tools URL Codec, then read more articles on the site about the formats and security mechanisms carried by modern web requests.