Python unicodedata.normalize() Function: Usage and Examples

Normalize Unicode strings into consistent canonical forms.

How It Works

unicodedata.normalize(form, text) supports NFC, NFD, NFKC, and NFKD. Canonical normalization can make visually identical strings compare equal; compatibility normalization may change meaningful formatting distinctions.

Step-by-Step Example

import unicodedata
combined = 'café'
decomposed = 'cafe\u0301'
print(combined == decomposed)
print(unicodedata.normalize('NFC', combined) == unicodedata.normalize('NFC', decomposed))

Understanding the Example

The expression "'café'" supplies the value stored as 'combined'. This value is then used or checked by the later statements.
The expression "'cafe\\u0301'" supplies the value stored as 'decomposed'. This value is then used or checked by the later statements.

A successful run of this example prints the following output:

False
True

Where This Fits in a Real Program

Start by writing down examples that should match and examples that must fail. Check whether you need to locate a substring, match a prefix or validate an entire field; these are different operations. Unicode and escape sequences can also change which characters are matched, so test accented text and empty input.

Practice: Try It Yourself

Normalization is not a security-purpose confusable character detector.

For this exercise, predict the result before running the changed code. Then explain the difference between the original and modified cases using the actual values, not just whether the program completed.

Common Beginner Mistakes

Assuming Unicode text that looks identical is byte-for-byte equal without normalization.
Normalize input names to NFC before a controlled comparison.

Key Takeaways

Normalize Unicode strings into consistent canonical forms.
unicodedata.normalize(form, text) supports NFC, NFD, NFKC, and NFKD.
Apply the demonstrated operation to the practice input and verify its result before depending on it in a larger program.