ns_percentencode - Encoding and Decoding Percent-Encoded Strings
This group of commands lets you encode and decode strings by escaping unsafe characters as percent-escape sequences ("%xx"). ns_percentencode and ns_percentdecode are the low-level percent-encoding primitives. They operate on a single string according to a selected encoding scheme. The supported schemes are query, path, cookie, and oauth1. ns_urlencode and ns_urldecode are URL-oriented convenience commands. They use URL parts rather than percent-encoding schemes. In addition to the historical query and path parts, ns_urlencode supports higher-level URL construction helpers such as fullpath and fragment. Use ns_percentencode when you need a primitive percent encoder for one string. Use ns_urlencode when constructing URL query strings, path components, full paths, or fragments from application data.
Percent‑encodes a single string. All characters not in the selected scheme’s unreserved set are replaced by "%" and two hex digits.
Reverses the percent‑encoding in a string. Under query scheme only, "+" is converted back to space. If decoding to UTF‑8 fails and -fallbackcharset is given (and -charset is not), a second decode attempt uses the fallback charset instead of raising an exception.
Encodes URL data according to the selected URL -part. With -part query, a single argument is encoded as one query value, preserving the historical behavior. With multiple arguments, the arguments are interpreted as alternating name/value pairs. Names and values are encoded and joined as a query string using "=" and "&". Repeated names are preserved when the argument list contains repeated names, for example when using ns_set data via ns_set array. With -part path, each argument is treated as one URL path component and multiple arguments are joined with "/". This is the historical behavior of ns_urlencode for path data. With -part fullpath, the single argument is treated as a full URL path. Slash characters in the argument are preserved as path separators, and each path segment is encoded individually. With -part fragment, the single argument is encoded as a URL fragment component. The cookie and oauth1 parts are accepted for backward compatibility, but are deprecated for ns_urlencode. Use ns_percentencode with -scheme cookie or -scheme oauth1 instead.
Decodes percent-escape sequences in string according to the selected URL -part. The supported part names match ns_urlencode. For decoding, the selected part primarily controls the treatment of plus signs. With -part query, plus signs are decoded as spaces. With path, fullpath, fragment, cookie, and oauth1, plus signs remain literal plus signs. Percent escape sequences are decoded in all parts. The cookie and oauth1 parts are accepted for backward compatibility, but are deprecated for ns_urldecode. Use ns_percentdecode with -scheme cookie or -scheme oauth1 instead.
The URL parts accepted by ns_urlencode and ns_urldecode are query, path, fullpath, fragment, cookie, and oauth1. The cookie and oauth1 parts are retained for compatibility only and are deprecated in favor of the corresponding ns_percentencode and ns_percentdecode schemes.
query (default): As a percent-encoding scheme, query encodes URL query values and form data (application/x-www-form-urlencoded). Spaces are encoded as "+". Unsafe characters from RFC 3986 query components are percent encoded. As an ns_urlencode URL part, query has two forms. With one argument, the argument is encoded as one query value. With multiple arguments, the arguments are interpreted as alternating name/value pairs and encoded as a query string.
% ns_urlencode -part query "Gustaf & NaviServer" Gustaf+%26+NaviServer % ns_urlencode -part query a 1 b "Gustaf & NaviServer" a=1&b=Gustaf+%26+NaviServer
Query pair construction works naturally with ns_set values, including repeated keys:
% set s [ns_set create query]
% ns_set put $s a 1
% ns_set put $s a 2
% ns_set put $s q "Gustaf & NaviServer"
% ns_urlencode -part query {*}[ns_set array $s]
a=1&a=2&q=Gustaf+%26+NaviServer
76 characters remain unescaped by the query scheme:
! $ ' ( ) * + - . / 0 1 2 3 4 5 6 7 8 9 : ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ a b c d e f g h i j k l m n o p q r s t u v w x y z ~
path: As a percent-encoding scheme, path encodes one URL path component according to RFC 3986 path-segment rules. Spaces are encoded as "%20"; slashes embedded in path components are encoded as "%2F". As an ns_urlencode URL part, path preserves the historical behavior: each argument is treated as one path component and multiple arguments are joined with "/".
% ns_urlencode -part path docs "NaviServer ADP" docs/NaviServer%20ADP % ns_urlencode -part path "docs/NaviServer ADP" docs%2fNaviServer%20ADP
Use fullpath when the input is already a complete URL path and slash characters should be preserved as path separators. 76 characters remain unescaped by the path scheme:
! $ & ' ( ) * + , - . 0 1 2 3 4 5 6 7 8 9 : @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ a b c d e f g h i j k l m n o p q r s t u v w x y z ~
fullpath: URL part for ns_urlencode. The input is treated as a complete URL path. Slash characters are preserved as path separators, and each path segment is encoded individually. Leading, trailing, and repeated slashes are preserved.
% ns_urlencode -part fullpath "/docs/NaviServer ADP/index" /docs/NaviServer%20ADP/index % ns_urlencode -part fullpath "/a//b c/" /a//b%20c/
fullpath is intentionally different from path. The path part treats each argument as one path component, while fullpath treats its argument as an already structured path.
fragment: URL part for ns_urlencode. The input is encoded as a URL fragment component, i.e. the part after "#". Spaces are encoded as "%20". Slash and question mark characters are preserved, as allowed by RFC 3986 fragment syntax.
% ns_urlencode -part fragment "section 1#details" section%201%23details % ns_urlencode -part fragment "chapter/1?view=full" chapter/1?view=full
cookie: Encoding scheme for cookie-value segments per RFC 6265. Use this scheme with ns_percentencode and ns_percentdecode. The ns_urlencode -part cookie form is accepted for backward compatibility, but is deprecated. 89 characters remain unescaped:
! # $ & ' ( ) * + - . / 0 1 2 3 4 5 6 7 8 9 : < = > ? @
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ ] ^ _ `
a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~
oauth1: Encoding scheme for OAuth 1.0 signature base strings (RFC 5849 §3.6). Spaces are encoded as "%20"; only unreserved chars per RFC 3986 remain literal. Use this scheme with ns_percentencode and ns_percentdecode. The ns_urlencode -part oauth1 form is accepted for backward compatibility, but is deprecated. 66 characters remain unescaped:
- . 0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ a b c d e f g h i j k l m n o p q r s t u v w x y z ~
This scheme is also used for AWS signature encoding.
ns_urlencode encodes unencoded application data into URL parts. It does not preserve existing percent escapes as escapes; a literal "%" character is encoded when it is unsafe for the selected part. Do not pass already percent-encoded URL parts to ns_urlencode unless double encoding is intended.
Encode form data as an application/x-www-form-urlencoded query string or POST body: (https://www.w3.org/TR/html401/interact/forms.html#h-17.13.4):
set data {
first_name John
second_name Doe
data "Hello World"
}
set post_data [ns_urlencode -part query {*}$data]
## output: first_name=John&second_name=Doe&data=Hello+World
Encode path segments per W3C URI recommendations (examples 1 and 2 of URI Recommendation):
set e1 {albert bertram marie-claude}
ns_urlencode -part path {*}$e1
## output: albert/bertram/marie-claude
set e2 {albert bertram/marie-claude}
ns_urlencode -part path {*}$e2
## output: albert/bertram%2fmarie-claude
ns_urlencode -part fullpath "/albert/bertram marie-claude"
## output: /albert/bertram%20marie-claude
Encode a URL fragment:
ns_urlencode -part fragment "section 1#details" ## output: section%201%23details
By default escapes use lowercase, but you can request uppercase:
% ns_percentencode -scheme path "a/b c" ## output: a%2fb%20c % ns_percentencode -scheme path -uppercase "a/b c" ## output: a%2FB%20c
Use the cookie scheme to percent‑encode only those bytes unsafe in cookie segments:
ns_percentencode -scheme cookie "key=value; path=/" ## output: key%3dvalue%3b%20path%3d%2f
Construct an OAuth 1.0 signature base string
ns_percentencode -scheme oauth1 -uppercase "Ladies + Gentlemen" ## output: Ladies%20%2B%20Gentlemen
If your percent‐encoded input isn’t valid UTF‑8, you can retry with a different charset:
ns_percentdecode "%C3%A9t%C3%A9" ## output: été ns_percentdecode "%E9" ## output: input string '%E9' cannot be converted to UTF-8 ns_percentdecode -charset iso8859-1 "%E9" ## output: é
With the fallback charset, a valid UTF-8 string and a latin-1 string can be decoded.
ns_percentdecode -fallbackcharset iso8859-1 "%C3%A9t%C3%A9" ## output: été ns_percentdecode -fallbackcharset iso8859-1 "%E9" ## output: é