iskm/wordfreq
1
0
Fork 0
mirror of https://github.com/rspeer/wordfreq.git synced 2024-12-26 18:38:51 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
142 Commits 8 Branches 124 Tags 707 MiB
9aa773aa2b
Commit Graph

142 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Joshua Chin
78c5b589c5 splits emoji from text 2015-06-24 16:50:28 -04:00
Joshua Chin
2346a0535a word_frequency no longer returns None if it does not detect tokens 2015-06-24 14:47:26 -04:00
Joshua Chin
0cac6dfda1 Merge pull request #9 from LuminosoInsight/centibels 
Switch to a more precise centibel scale
 2015-06-23 12:55:55 -04:00
Rob Speer
d81070a13c fix redundant mention of the header 2015-06-23 12:53:57 -04:00
Rob Speer
7862a4d2b6 Switch to a more precise centibel scale. 2015-06-22 17:36:30 -04:00
Rob Speer
28c9edf135 Update the copyright year in the license 
I don't know when you need to update the year, but the code has substantially changed since 2013.
 2015-06-18 18:55:59 -04:00
Rob Speer
cd3270168f Rename MIT-LICENSE to MIT-LICENSE.txt 
This makes the filename match what README.txt refers to.
 2015-06-18 18:55:18 -04:00
Joshua Chin
90fc3970c6 updated data with a clean build 2015-06-18 11:38:57 -04:00
Joshua Chin
35f472fcf9 updated test because the new tokenizer removes URLs 2015-06-18 11:38:28 -04:00
Rob Speer
981cc249f1 copy-edit some docstrings 2015-06-17 14:42:38 -04:00
Rob Speer
13988f8e3d Merge pull request #8 from LuminosoInsight/newbuild-refactor 
Refactored the newbuild branch, in response to the preliminary review notes
 2015-06-17 14:35:37 -04:00
Joshua Chin
68b1c121bd updated dB_to_freq docstring 2015-06-17 14:34:03 -04:00
Joshua Chin
a289ab7f8b clearified the docstrings for random_words and random_ascii_words 2015-06-17 14:26:06 -04:00
Joshua Chin
9f288bac31 corrected available_languages to return a dict of strs to strs 2015-06-17 12:43:13 -04:00
Joshua Chin
6cc962bfea changed yield to yield from in iter_wordlist 2015-06-17 12:38:31 -04:00
Joshua Chin
9b30da4dec updated db_to_freq docstring 2015-06-17 12:24:23 -04:00
Joshua Chin
7e808bf7c1 added docstrings 2015-06-17 12:20:50 -04:00
Joshua Chin
053e4da3e6 removed temporary variable 2015-06-17 12:16:02 -04:00
Rob Speer
ed19d79c5a clearer error on py2 2015-05-28 14:05:11 -04:00
Rob Speer
0f4ca80026 add installation instructions to the readme 2015-05-28 14:02:12 -04:00
Rob Speer
611a6a35de update Japanese data; test Japanese and token combining 2015-05-28 14:01:56 -04:00
Rob Speer
05cf94d1fd Work on making Japanese tokenization use MeCab consistently 2015-05-27 18:10:25 -04:00
Rob Speer
0e5156e162 Merge branch 'master' into newbuild 
Conflicts:
	setup.py
	wordfreq/build.py
	wordfreq/config.py
 2015-05-21 20:41:47 -04:00
Rob Speer
84e5edcea1 rebuild data 2015-05-21 20:36:15 -04:00
Rob Speer
410912d8f0 remove old tests 2015-05-21 20:36:09 -04:00
Rob Speer
b42594fa5f allow more language matches; reorder some parameters 2015-05-21 20:35:02 -04:00
Rob Speer
df863a5169 tests for new wordfreq with full coverage 2015-05-21 20:34:17 -04:00
Rob Speer
dd41e61c57 update README, another setup fix 2015-05-13 04:09:34 -04:00
Rob Speer
f13cca4d81 update dependencies 2015-05-12 12:30:01 -04:00
Rob Speer
bb18f741e2 restore missing line in setup.py 2015-05-12 12:24:18 -04:00
Rob Speer
35aec061de add new data files from wordfreq_builder 2015-05-11 18:45:47 -04:00
Rob Speer
9b63e54471 WIP: burn stuff down 2015-05-08 15:28:52 -04:00
Lance Nathan
e8a1548d93 Tweak to previous variable name fix 2015-05-06 17:57:10 -04:00
Lance Nathan
4632ffb177 Merge pull request #6 from LuminosoInsight/ftfy4 
Clean data with ftfy v4
 2015-05-06 17:32:45 -04:00
Lance Nathan
5f05b52fe5 Merge pull request #5 from LuminosoInsight/dutch-201504 
Better Dutch surface-form data
 2015-05-06 17:15:21 -04:00
Rob Speer
506073030a fix reused variable name 2015-05-06 17:06:37 -04:00
Rob Speer
2f3bb955d1 set version number to 0.8 2015-05-05 12:05:00 -04:00
Rob Speer
24a7c73e6d Merge branch 'dutch-201504' into ftfy4 
Conflicts:
	setup.py
 2015-05-05 12:04:44 -04:00
Rob Speer
70b2c678ea require ftfy 4 2015-05-05 12:04:13 -04:00
Rob Speer
873ace87db v0.7: make a proper Dutch 'surfaces' list 2015-04-30 13:01:24 -04:00
Rob Speer
6cf46ee5aa Merge branch 'master' into dutch-201503 
Conflicts:
	wordfreq/build.py
 2015-04-29 14:36:24 -04:00
Rob Speer
af5f65b328 start a new multilingual wordlist called 'stems' 
So far, this wordlist is only in Dutch.
 2015-03-31 15:59:30 -04:00
Rob Speer
3507d8b630 Fix Dutch lists 
- Use surface forms consistently, not stems
- Count all instances of words on Wikipedia, not one per article
 2015-03-12 16:00:03 -04:00
Andrew Lin
cfe58cd899 Merge pull request #3 from LuminosoInsight/variable_name_fix 
Fix a variable name for clarity.
 2015-03-11 14:10:53 -04:00
Rob Speer
377336bcdc new Dutch data, bump version to 0.6 2015-03-03 15:54:45 -05:00
Andrew Lin
434c603798 Fix a variable name for clarity. 2015-03-03 11:59:46 -05:00
Andrew Lin
5a4d3a87d5 Merge pull request #2 from LuminosoInsight/new-twitter-lists 
New twitter lists
 2015-02-17 15:36:13 -05:00
Rob Speer
ffdaa82b11 add surface forms from Twitter 2014 data 2015-02-17 15:06:11 -05:00
Rob Speer
b6f246ecbb stop running 'remove_unsafe_private_use' unnecessarily 2015-02-17 14:02:36 -05:00
Rob Speer
6ab72201cd add twitter-stems-2014 wordlist data 2015-02-11 13:29:32 -05:00