iskm/wordfreq
1
0
Fork 0
mirror of https://github.com/rspeer/wordfreq.git synced 2024-12-25 18:18:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
535 Commits 8 Branches 124 Tags 707 MiB
96a01b9685
Commit Graph

2 Commits

Author SHA1 Message Date
Rob Speer
96a01b9685 port remaining tests to pytest 2018-06-01 16:40:51 -04:00
Rob Speer
a0893af82e Tokenization in Korean, plus abjad languages (#38) 
* Remove marks from more languages

* Add Korean tokenization, and include MeCab files in data

* add a Hebrew tokenization test

* fix terminology in docstrings about abjad scripts

* combine Japanese and Korean tokenization into the same function


Former-commit-id: fec6eddcc3
 2016-07-15 15:10:25 -04:00