mirror of
https://github.com/rspeer/wordfreq.git
synced 2024-12-23 09:21:37 +00:00
f885a60bf0
There are Unicode normalization problems with Malayalam -- as best I understand it, Unicode simply neglected to include normalization forms for Malayalam "chillu" characters even though they changed how they're represented in Unicode 5.1 and again in Unicode 9. The result is that words that print the same end up with multiple entries, with different codepoint sequences that don't normalize to each other. I certainly don't know how to resolve this, and it would need to be resolved to have something that we could reasonably call Malayalam word frequencies. |
||
---|---|---|
.. | ||
test_at_sign.py | ||
test_chinese.py | ||
test_french_and_related.py | ||
test_general.py | ||
test_japanese.py | ||
test_korean.py | ||
test_transliteration.py |