--- Claude 3 Family ---
Text:
'Hello, world!'
Token Count: 3

--- Claude 3 Family ---
Text:
'The quick brown fox jumps over the lazy dog.'
Token Count: 11

--- Claude 3 Family ---
Text:
'
Tokenization is the process of breaking down a sequence of text into smaller units, called tokens.
These tokens can be words, sub-words, or even individual characters.
The way text is tokenized can have a significant impact on the performance and cost of large language models.
Understanding token counts is crucial for managing context windows and API usage.
'
Token Count: 79

--- Claude 3 Family ---
Text:
'
def greet(name):
    return f"Hello, {name}!"

print(greet("Alice"))
'
Token Count: 22

--- Claude 3 Family ---
Text:
'This is a test with some special characters: !@#$%^&*()_+=-`~[]{}|;:\'',".<>/? and numbers 12345.'
Token Count: 45

--- Claude 3 Family ---
Text:
'Hello, 你好世界！'
Token Count: 7