Rob Speer 3ec92a8952 Handle Japanese edge cases in simple_tokenize		2018-04-26 15:53:07 -04:00
..
test_chinese.py	Traditional Chinese should be preserved through tokenization	2018-03-08 18:08:55 -05:00
test_french_and_related.py	Use langcodes when tokenizing again (it no longer connects to a DB)	2017-04-27 15:09:59 -04:00
test_japanese.py	Handle Japanese edge cases in simple_tokenize	2018-04-26 15:53:07 -04:00
test_korean.py	Tokenization in Korean, plus abjad languages (#38)	2016-07-15 15:10:25 -04:00
test_transliteration.py	fix az-Latn transliteration, and test	2018-03-08 16:47:36 -05:00
test.py	Handle Japanese edge cases in simple_tokenize	2018-04-26 15:53:07 -04:00