A log entry contains names and symbols that are not limited to ASCII.
What You Need to Understand
Python str values are Unicode text, while bytes contain encoded data. Use explicit UTF-8 encoding and decoding when saving or transferring text; avoid treating a byte count as a character count.
When to Use It
A log entry contains names and symbols that are not limited to ASCII.
Step-by-Step Python Example
message = "Café — ready"
encoded_message = message.encode("utf-8")
restored_message = encoded_message.decode("utf-8")
print(restored_message == message)
print(len(message), len(encoded_message))How the Example Works
The round trip retains all characters. The encoded byte count can exceed the number of Unicode code points.
Example Output
True
12 15
résuméA Second Example
Compare this smaller variation with the main example, then experiment with your own values.
word = "résumé"
print(word.encode("utf-8").decode("utf-8"))Important Behavior and Edge Cases
Decoding with the wrong encoding may fail or corrupt text. Unicode code points also do not always correspond one-to-one with user-perceived characters.
Try It Yourself
Encode a short sentence in a non-Latin script and compare the string length to the UTF-8 byte count.
Common Beginner Mistakes
- Decoding with the wrong encoding may fail or corrupt text. Unicode code points also do not always correspond one-to-one with user-perceived characters.
- Applying a string method before defining whether text is intended for display, comparison, parsing, or input validation.
Key Takeaways
- Python
strvalues are Unicode text, while bytes contain encoded data. - The round trip retains all characters.
- Decoding with the wrong encoding may fail or corrupt text.