iskm/wordfreq
1
0
Fork 0
mirror of https://github.com/rspeer/wordfreq.git synced 2024-12-25 02:05:24 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Note on next languages to support

Browse Source 
Former-commit-id: 531db64288
This commit is contained in:
Rob Speer 2015-09-04 01:50:15 -04:00
parent bf88f97744
commit 25e24f9c32
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
wordfreq_builder/wordfreq_builder/config.py
View File

@ -8,6 +8,11 @@ CONFIG = {
'sources': { 'sources': {
# A list of language codes (possibly un-standardized) that we'll # A list of language codes (possibly un-standardized) that we'll
# look up in filenames for these various data sources. # look up in filenames for these various data sources.
#
# Consider adding:
# 'th' when we get tokenization for it
# 'hi' when we stop messing up its tokenization
# 'tl' because it's probably ready right now
'twitter': [ 'twitter': [
'ar', 'de', 'el', 'en', 'es', 'fr', 'id', 'it', 'ja', 'ko', 'ms', 'nl', 'ar', 'de', 'el', 'en', 'es', 'fr', 'id', 'it', 'ja', 'ko', 'ms', 'nl',
'pt', 'ru', 'tr' 'pt', 'ru', 'tr'