Commit Graph

22 Commits

Author SHA1 Message Date
Robyn Speer
d99ac1051a fix version, update instructions and changelog 2021-02-18 18:25:16 -05:00
Robyn Speer
f71acec2d7 work with langcodes 3.0, without language_data 2021-02-09 17:27:22 -05:00
Robyn Speer
a8915d67f7 update the changelog 2020-10-01 16:12:41 -04:00
Robyn Speer
becf94f767 update version and changelog 2020-04-28 15:24:24 -04:00
Robyn Speer
3b7382d770 update CHANGELOG for 2.3.1 2020-04-22 11:12:02 -04:00
Robyn Speer
258670b823 update changelog for 2.3 2020-04-16 15:51:20 -04:00
Lance Nathan
86e988b838 Fix code affected by a breaking change in msgpack 1.0
The msgpack readme explains: "Default value of strict_map_key is changed to
True to avoid hashdos. You need to pass strict_map_key=False if you have data
which contain map keys which type is not bytes or str."

chinese.py loads SIMPLIFIED_MAP from disk.  Since it is a str.translate
dictionary, its keys are numbers.  And since it's a dictionary we created
ourselves, there's no hashdos concern, so we can load it with
strict_map_key=False.
2020-02-28 13:02:45 -05:00
Robyn Speer
0931f1297d update changelog for v2.2.1 2019-02-05 15:58:10 -05:00
Rob Speer
d9fc6ec42c update the changelog for version 2.2 2018-07-23 16:38:39 -04:00
Rob Speer
0644c8920a Update README to describe @ tokenization 2018-07-23 11:21:44 -04:00
Rob Speer
ca9cf7d90f update the CHANGELOG for MeCab fix 2018-06-26 11:31:03 -04:00
Rob Speer
1dc763c9c5 update README and CHANGELOG 2018-06-18 15:21:43 -04:00
Rob Speer
52aae3459d fix typo in previous changelog entry 2018-06-18 10:52:28 -04:00
Rob Speer
0a95d96b20 update CHANGELOG for 2.0.1 2018-05-01 14:47:55 -04:00
Rob Speer
a759f38540 update the changelog 2018-03-14 17:56:29 -04:00
Rob Speer
e3352392cc v1.7: update tokenization, update data, add bn and mk 2017-08-25 17:37:48 -04:00
Andrew Lin
f28a193015 Clarify the changelog. 2017-02-14 13:09:12 -05:00
Rob Speer
e6114bf0fa Update README with new examples and URL 2017-01-09 15:13:19 -05:00
Rob Speer
3f9c8449ff Mention that multi-digit numbers are combined together 2017-01-05 19:24:28 -05:00
Rob Speer
a05a1c8d5c mention tokenization change in changelog 2017-01-05 19:19:31 -05:00
Rob Speer
803ebc25bb Update documentation and bump version to 1.6 2017-01-05 19:18:06 -05:00
Rob Speer
0ba563c99c Add a changelog 2016-08-22 12:41:39 -04:00