Articles

JavaScript Strings and Unicode: Code Units vs Graphemes

JavaScript string length counts UTF-16 code units, not characters. How code units, code points, and grapheme clusters differ and how to handle each.

Takina Takina · · 5 min read
A laptop screen showing code in blue light

JavaScript strings are sequences of UTF-16 code units, not sequences of characters. That single fact explains why "😀".length is 2, why reversing a string can produce garbage, and why truncating a username can split an emoji in half. To handle text correctly you need to know three levels of “character”: the code unit JavaScript stores, the code point Unicode defines, and the grapheme cluster a person actually sees. Each level has its own tools in the language.

Three levels of character

  • Code unit. A 16-bit value. This is what .length, indexing with str[i], and charCodeAt work with.
  • Code point. A Unicode number from U+0000 to U+10FFFF. Code points up to U+FFFF fit in one code unit. Code points above that, including most emoji, need two code units called a surrogate pair.
  • Grapheme cluster. What a reader perceives as a single character. It can be one code point, or several combined: a letter plus a combining accent, an emoji plus a skin-tone modifier, or a family emoji made of several people joined by invisible zero-width joiners.

The encoding background, including why UTF-16 uses surrogate pairs at all, is covered in UTF-8 and how Unicode becomes bytes.

"a".length;        // 1  — one code unit, one code point, one grapheme
"é".length;        // 1  — precomposed U+00E9
"é".length;  // 2  — "e" + combining acute accent, looks identical
"😀".length;       // 2  — one code point, a surrogate pair
"👍🏽".length;      // 4  — thumbs up + skin tone, two code points
"👨‍👩‍👧".length;      // 8  — three people joined by two zero-width joiners

All of those last four look like one character on screen.

Code-unit APIs and where they break

Most older string methods operate on code units:

  • str.length counts code units.
  • str[i] and str.charAt(i) return a single code unit, which may be half of a surrogate pair.
  • str.slice, substring, and substr cut at code-unit offsets.
  • str.split("") splits into code units.

That means "😀".split("") returns two lone surrogates, and a naive reverse breaks emoji:

"ab😀".split("").reverse().join(""); // "\uDE00\uD83Dba" — broken

Lone surrogates are not valid text. Depending on where they end up, they render as replacement characters, cause errors when encoded with encodeURIComponent, or get mangled when sent to a server. str.isWellFormed() checks for them, and str.toWellFormed() replaces any lone surrogates with U+FFFD.

Code-point-aware APIs

Since ES2015, JavaScript has tools that understand surrogate pairs.

  • String iteration. for...of, the spread operator, and Array.from iterate by code point, using the string’s built-in iterator. If iterators are unfamiliar, see generators and iterators.
  • codePointAt and String.fromCodePoint. Read and create full code points instead of code units.
  • The u and v regex flags. With the u flag, . matches a full code point and \u{1F600} escapes work. Unicode property escapes like \p{Letter} or \p{Emoji} also require one of these flags.
[..."ab😀"].length;          // 3
[..."ab😀"].reverse().join(""); // "😀ba" — correct for this string
"😀".codePointAt(0).toString(16); // "1f600"
/^.$/u.test("😀");           // true
/^.$/.test("😀");            // false

Code points still are not graphemes, though. [..."👍🏽"].length is 2, and reversing a string containing "é" moves the accent onto the wrong letter.

Grapheme clusters with Intl.Segmenter

For user-perceived characters, use Intl.Segmenter, part of the Intl API. It applies the Unicode rules for grapheme boundaries, including emoji sequences and combining marks.

const seg = new Intl.Segmenter(undefined, { granularity: "grapheme" });

function graphemes(str) {
  return Array.from(seg.segment(str), (s) => s.segment);
}

graphemes("👨‍👩‍👧 e\u0301").length; // 3 — family, space, e + accent
graphemes("ab👍🏽").reverse().join(""); // "👍🏽ba"

Intl.Segmenter also supports "word" and "sentence" granularity, which is the right way to count words in languages that do not separate them with spaces. The segments are returned lazily, so you can stop early when, for example, you only need the first N graphemes; the iterator helpers make that convenient.

Practical recipes

Truncating text for display. Cut at grapheme boundaries so you never split an emoji or strip an accent.

function truncate(str, max) {
  const parts = graphemes(str);
  return parts.length <= max ? str : parts.slice(0, max).join("") + "…";
}

Character limits in forms. Decide what you are limiting. A visible character count should use graphemes. A storage limit should use the encoded byte length, which you can get with new TextEncoder().encode(str).length. HTML’s maxlength attribute counts UTF-16 code units, so it can disagree with both.

Comparing visually identical strings. "é" and "é" look the same but are not ===. Normalize first with str.normalize("NFC") before comparing, storing usernames, or building lookup keys.

Case-insensitive comparison. toLowerCase is locale-independent but not always linguistically correct. For user-facing sorting and comparison, Intl.Collator or localeCompare with a sensitivity option is more robust.

Comparing the three levels

LevelWhat it counts"👍🏽"Main APIs
Code unit16-bit UTF-16 units4length, str[i], slice, charCodeAt
Code pointUnicode scalar values2for...of, spread, codePointAt, regex u flag
Grapheme clusterUser-perceived characters1Intl.Segmenter with granularity: "grapheme"
UTF-8 bytesEncoded storage size8TextEncoder

Performance notes

Code-unit operations are constant time, which is why the language exposes them by default. Code-point iteration and grapheme segmentation must scan the string, so they are linear in its length. For short strings like names and labels this is irrelevant. For very large text, segment once and reuse the result rather than re-segmenting inside a loop, and keep heavy text processing off the main thread, perhaps in a web worker, if it risks blocking rendering.

The takeaway

A JavaScript string’s length and indexes count UTF-16 code units, so emoji and other characters outside the Basic Multilingual Plane take two units, and many visible characters span several code points. Use for...of, spread, and the regex u flag when you need code points, Intl.Segmenter when you need what users actually see, TextEncoder when you need byte size, and normalize before comparing strings that might be composed differently.

Takina Takina · · 4 min read

Object.groupBy() in JavaScript, Explained

Object.groupBy() buckets array items by a key you compute, replacing the reduce() boilerplate developers have written for years. How it works.

#JavaScript #Web Development #Frontend