iskm/wordfreq
1
0
Fork 0
mirror of https://github.com/rspeer/wordfreq.git synced 2024-12-24 01:41:39 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
416 Commits 8 Branches 124 Tags 707 MiB
927d4f45a4
Commit Graph

4 Commits

Author SHA1 Message Date
Robyn Speer
4a4534c466 test_chinese: fix typo in comment 
Former-commit-id: 2a84a926f5
 2015-09-24 13:41:11 -04:00
Robyn Speer
1adbb1aaf1 add external_wordlist option to tokenize 
Former-commit-id: 669bd16c13
 2015-09-10 18:09:41 -04:00
Robyn Speer
f0c7c3a02c Lower the frequency of phrases with inferred token boundaries 
Former-commit-id: 5c8c36f4e3
 2015-09-10 14:16:22 -04:00
Robyn Speer
a4554fb87c tokenize Chinese using jieba and our own frequencies 
Former-commit-id: 2327f2e4d6
 2015-09-05 03:16:56 -04:00