iskm/wordfreq
1
0
Fork 0
mirror of https://github.com/rspeer/wordfreq.git synced 2024-12-24 01:41:39 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

describe the use of lang in read_values

Browse Source
This commit is contained in:
Rob Speer 2015-09-22 17:22:38 -04:00
parent 7c12f2aca1
commit f224b8dbba
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
wordfreq_builder/wordfreq_builder/word_counts.py
View File

@ -42,6 +42,9 @@ def read_values(filename, cutoff=0, lang=None):
If `cutoff` is greater than 0, the csv file must be sorted by value If `cutoff` is greater than 0, the csv file must be sorted by value
in descending order. in descending order.
If `lang` is given, it will apply language-specific tokenization to the
words that it reads.
""" """
values = defaultdict(float) values = defaultdict(float)
total = 0. total = 0.