This guide explains Unicode Basics: Code Points, Text, and Encoding through a focused example and practical reasoning. No prior knowledge of this specific operation is required.
Understanding the Concept
Unicode assigns code points to characters, while an encoding such as UTF-8 turns text into bytes for storage or transmission. A string represents text, whereas bytes represent encoded data. One displayed symbol can contain several code points.
When You Would Use This
Inspect a label containing a letter and an accented character.
Step-by-Step Example
label = "Café"
encoded_label = label.encode("utf-8")
print(len(label))
print(len(encoded_label))
print(encoded_label.decode("utf-8"))How the Example Works
The text contains four code points, but the UTF-8 representation takes five bytes because é uses multiple bytes. Decoding restores the original text.
Example Output
4
5
CaféTry It Yourself
Inspect the UTF-8 byte length of a string containing a smiling emoji.
Common Beginner Mistakes
- Confusing byte length with character count.
- Opening text with the wrong encoding and assuming the file is corrupted.
- Assuming every visually perceived symbol is one Unicode code point.
Key Takeaways
- Unicode assigns code points to characters, while an encoding such as UTF-8 turns text into bytes for storage or transmission.
- The text contains four code points, but the UTF-8 representation takes five bytes because
éuses multiple bytes. - Avoid this mistake: Confusing byte length with character count.