Why Text Becomes café: Debug UTF-8, HTML Entities and URL Encoding

You enter café into a form, save it, and later see café. Or a query string containing C++ becomes C . Or an HTML page displays & instead of an ampersand. These problems look related because they all involve special characters, but they come from different layers of text representation.

Correcting the wrong layer can make the damage worse. Replacing every é with é might temporarily improve one page, for example, but it cannot repair a broken encoding pipeline or handle every other Unicode character.

This guide separates character encodings, HTML entities, URL percent encoding and JSON representation, then shows how to locate the actual fault using browser tools, cURL and JavaScript.

Four layers that are easy to confuse

A character is not the same thing as its transmitted bytes or the way it is written inside markup. Consider this short text:

café & tea

There are several valid ways to represent it, depending on the context:

UTF-8 is a character encoding. Percent encoding is a way to represent bytes in a URL. HTML character references are part of HTML syntax. JSON escaping is part of JSON syntax. They are not interchangeable converters.

The Web2Generators HTML Encoder and Decoder is designed for character references such as & and <. The separate URL Encoder and Decoder handles percent encoding, form values and query parameters.

Problem 1: café is usually a character-decoding mismatch

The letter é is encoded in UTF-8 as the two bytes C3 A9. If those bytes are mistakenly interpreted as Windows-1252 characters, they are displayed as é. The bytes did not inherently contain the wrong word. They were interpreted with the wrong encoding.

This kind of corrupted-looking text is often called mojibake. It can occur when a database connection, file importer, response header or legacy service assumes a different encoding from the component that produced the bytes.

Check the full path of the data:

1. How was the original text entered or imported?
2. Which encoding was used to write it into a file, database or message?
3. Was it converted again when read or exported?
4. What bytes does the server actually send?
5. How does the recipient choose the decoding charset?

An HTML document should normally declare UTF-8 early in the document:

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Text example</title>
</head>
<body>
  <p>café &amp; tea</p>
</body>
</html>

For a server-delivered HTML page, also check its HTTP response header, for example:

Content-Type: text/html; charset=utf-8

An explicit server charset can override a document declaration, and an already damaged database string will not become correct merely because the response declares UTF-8. The declaration describes the bytes being served; it does not repair them.

Problem 2: The text is already corrupted before the browser receives it

Start with the browser’s Network panel. Open the request that supplies the affected content, inspect the response headers and compare the raw response with the rendered page. The Web2Generators browser Network inspection guide walks through this process in Chrome, Firefox, Edge and Safari.

For a reproducible server-side check, use cURL to save headers and response bytes:

curl --compressed --silent --show-error \
  --dump-header headers.txt \
  --output response.bin \
  'https://example.com/api/profile'

--compressed permits cURL to decode a supported HTTP content compression format. This is distinct from decoding the response’s character encoding. For a JSON endpoint, inspect whether the response body is actually valid UTF-8 and whether the server returned the expected media type.

Save the following as check-response.mjs and run node check-response.mjs to inspect those bytes with Node.js:

import { readFileSync } from "node:fs";

const bytes = readFileSync("response.bin");
try {
  const text = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
  console.log(text.slice(0, 200));
} catch (error) {
  console.error("Response contains invalid UTF-8 byte sequences", error);
}

A successful UTF-8 decode proves that the byte sequence is well-formed. It does not prove that the text was never corrupted earlier: a string already containing café can itself be correctly encoded as UTF-8. For this reason, compare a known sample at each storage and transport boundary.

If you need to build a repeatable request with custom headers, use the cURL Command Builder before inspecting the actual result in your terminal. The builder prepares a command but does not fetch or validate the response itself.

Problem 3: A plus sign or space changes inside a URL

URL encoding has its own rules. In JavaScript, encodeURIComponent is useful for one query value, not for an entire URL:

const value = "café & tea";
console.log(encodeURIComponent(value));
// caf%C3%A9%20%26%20tea

A common mistake is constructing the full URL by hand:

// Fragile: '&' becomes a query separator.
const bad = "/search?q=" + "café & tea";

// Safer: the URL API encodes and manages the query parameter.
const url = new URL("https://example.com/search");
url.searchParams.set("q", "café & tea");
console.log(url.toString());
// https://example.com/search?q=caf%C3%A9+%26+tea

Why a plus sign? URLSearchParams uses form-style query serialization, where spaces become +. In contrast, encodeURIComponent uses %20 for spaces. A literal plus sign in a form-style value should become %2B:

const query = new URLSearchParams({ language: "C++" });
console.log(query.toString()); // language=C%2B%2B

Decode values according to the format that actually produced them. When working with a full URL, let URL and URLSearchParams handle its structure. See the site’s URL Encoder, Decoder and Query Editor announcement for examples involving repeated parameters and the difference between plus signs and spaces.

Do not repeatedly decode a signed URL or rebuild it casually. Changing percent-encoding details or parameter order can invalidate signatures that depend on the URL’s exact representation.

Problem 4: & appears on the page instead of &

The HTML source Tom & Jerry normally renders as Tom & Jerry. If the user sees the literal five-character sequence &, the data might have been escaped twice, or it may have been inserted into the page through a text-only API after already being escaped.

For example, this is correct when value is ordinary user-entered text:

const value = "Tom & Jerry";
document.querySelector("#result").textContent = value;

Do not first replace the ampersand with & and then assign that encoded string to textContent. The browser will treat the & sequence as literal text, not interpret it as markup.

Conversely, never switch to innerHTML merely to make an entity decode if the content can be controlled by a user. That can introduce cross-site scripting. If content is meant to be displayed literally, use textContent; if it is permitted rich HTML, apply a context-appropriate, maintained sanitizer. The HTML Escaping vs Sanitizing guide explains the security distinction.

The HTML tool is useful for experimenting with examples, but the right production fix is usually to escape at the appropriate output boundary, once, using your rendering framework or a context-aware library.

Problem 5: JSON, databases and forms disagree

JSON is a data format, not an alternative to UTF-8. A JSON response can contain a literal é or a Unicode escape such as \u00e9; after correct parsing, both represent the same character.

For most modern APIs, send UTF-8 encoded JSON with an appropriate Content-Type: application/json header and validate the body as JSON. Do not URL-encode a JSON string unless an API explicitly expects JSON inside a URL or form field.

Form submissions can use application/x-www-form-urlencoded or multipart/form-data, which have different serialization rules. If a server expects JSON but you send form-encoded values, the issue is a request-format mismatch, not necessarily a broken accent. The JSON vs Form Data vs Multipart guide provides side-by-side examples.

Database defaults and connection settings matter too. Check both the stored column data and the connection’s client/server encoding, rather than relying on a page’s meta charset. If imports have permanently replaced unrecognized characters with the replacement symbol �, restoring the original may require backups or an untouched source copy.

A quick diagnostic decision table

A reliable workflow for international text

Create a tiny regression fixture containing accented Latin text, non-Latin text, an emoji, a literal plus sign and an ampersand. Test that fixture at every boundary: form, server, database, API response, browser DOM and exported file. Keep expected outputs as actual Unicode strings, and preserve raw bytes while diagnosing an incident.

If your application also sends multilingual text to an AI model, character encoding and tokenization are different concerns. The Web2Generators AI Token Counter can illustrate token splits for different text, but it does not validate or repair UTF-8 transport problems.

Further reading

• WHATWG Encoding Standard 
• WHATWG URL Standard 
• MDN: TextDecoder 

For a quick hands-on diagnosis, take one broken value and inspect it separately in the HTML Encoder and Decoder and URL Encoder and Decoder. Identifying the representation first is often the fastest route to a clean fix.