Robyn Speer 86b928f967 include data from xc rebuild		2018-07-15 01:01:35 -04:00
..
data	include data from xc rebuild	2018-07-15 01:01:35 -04:00
__init__.py	Round frequencies to 3 significant digits	2018-06-18 15:21:33 -04:00
chinese.py	Traditional Chinese should be preserved through tokenization	2018-03-08 18:08:55 -05:00
language_info.py	Handle Japanese edge cases in simple_tokenize	2018-04-26 15:53:07 -04:00
mecab.py	Fix instructions and search path for mecab-ko-dic	2018-06-21 15:56:54 -04:00
preprocess.py	Recognize "@" in gender-neutral word endings as part of the token	2018-07-03 13:22:56 -04:00
tokens.py	Recognize "@" in gender-neutral word endings as part of the token	2018-07-03 13:22:56 -04:00
transliterate.py	fix az-Latn transliteration, and test	2018-03-08 16:47:36 -05:00