Invisible Unicode characters: what they are and what they break

Character	Code point	Purpose	What it breaks
Zero-width space	U+200B	Suggests a line-break point	Search, string comparison
Zero-width non-joiner	U+200C	Prevents letters joining	Ligatures, Arabic and Indic text
Zero-width joiner	U+200D	Forces letters to join	Emoji sequences, text length
Soft hyphen	U+00AD	Hyphen shown only when wrapping	Search, copied text
Non-breaking space	U+00A0	Space that never wraps	Trimming, CSV parsing, code
Narrow no-break space	U+202F	Thin space before punctuation	Same as above
Byte-order mark	U+FEFF	Marks encoding at file start	First CSV column, JSON parsing
Word joiner	U+2060	Prevents a break, no width	Search
Left-to-right mark	U+200E	Sets text direction	Display order, trimming
Right-to-left mark	U+200F	Sets text direction	Display order
RTL override	U+202E	Forces right-to-left display	Filename spoofing
Line separator	U+2028	Unicode line break	JavaScript string literals
Paragraph separator	U+2029	Unicode paragraph break	JSON in older parsers

Every one of these has a real job. The problem is never their existence — it is that they travel silently between systems that treat them differently.

• Copying from web pages. Sites insert zero-width spaces to control where long strings wrap, and soft hyphens for justified text. Both come along with the copy.
• Word processors. Non-breaking spaces are inserted automatically to keep a number with its unit, or a name on one line.
• PDF extraction. Copying from a PDF frequently produces soft hyphens where the typesetter broke words across lines, plus non-breaking spaces from justified spacing.
• AI chat interfaces. Output is rendered as HTML, so copying carries whatever spacing characters that formatting used — usually U+00A0 and U+202F.
• Encoding conversions. A byte-order mark appears when a file is saved as UTF-8 with BOM, common on Windows.
• Deliberate insertion. Zero-width characters have been used to fingerprint documents for leak tracing, and to slip text past keyword filters.

The pattern is consistent: they enter when text moves from a presentation context into a data context.

Search silently fails. A zero-width space inside a word means searching for it returns nothing. Nothing indicates why — the word is visibly right there.

Validation rejects correct input. An email address with a trailing non-breaking space fails a format check. The user sees a correct address and an error message that makes no sense.

Comparisons fail. "Ada" === "Ada\u200B" is false. Deduplication misses obvious duplicates; lookups return nothing; joins drop rows.

Trimming does not help. Most trim() implementations remove ASCII whitespace only. A non-breaking space survives .trim() in many languages, which is why "I already trimmed it" is such a common dead end.

Code fails to compile. A non-breaking space where a normal space belongs produces a syntax error at a line that looks perfect.

CSV imports corrupt. A byte-order mark attaches to the first header, so id becomes \uFEFFid and the first column silently fails to map.

Length is wrong. Zero-width joiners in emoji sequences mean a single visible emoji can be several code points, so character limits reject input that looks well within them.

Filenames can deceive. U+202E reverses displayed text, so report\u202Egnp.exe can display as reportexe.png. This is a genuine security concern, not a curiosity.

Scan first. Paste the text into the Invisible Character Scanner. It reports which types are present and how many of each, then removes them on request. Seeing the counts first tells you whether you are dealing with one stray character or systematic contamination.

In an editor. VS Code highlights most invisible characters by default via its unicode-highlight setting. For regex-capable find-and-replace, this pattern catches the common set:

[\u200B-\u200F\u00AD\uFEFF\u2060\u202A-\u202E\u2028\u2029]

In code, normalise at the boundary — when data arrives, not when it is used:

const clean = input
  .replace(/[\u200B-\u200F\u2060\uFEFF\u00AD\u202A-\u202E]/g, "")
  .replace(/[\u00A0\u202F]/g, " ")
  .normalize("NFC")
  .trim();

Note the two-step treatment: zero-width characters are deleted, but non-breaking spaces are converted to ordinary spaces — deleting them would join words together.

Do not strip blindly. Zero-width joiners are load-bearing in emoji sequences and in Arabic, Persian, and many Indic scripts, where removing them changes how words render. Strip aggressively in identifiers, keys, and code; strip conservatively in user-facing content.

For the specific question of whether AI tools insert these deliberately, see does ChatGPT hide invisible characters. The visible-but-equally-disruptive equivalent is covered in why smart quotes break code.