Commit Graph

14 Commits

Author SHA1 Message Date
Joshua Chin
b3a008f992 added arabic tests
Former-commit-id: f83d31a357
2015-07-07 15:10:59 -04:00
Joshua Chin
21c809416d changed default to minimum for word_frequency
Former-commit-id: 9aa773aa2b
2015-07-07 15:03:26 -04:00
Joshua Chin
9c741bb341 updated tests
Former-commit-id: ca66a5f883
2015-07-07 14:13:28 -04:00
Robyn Speer
9615b9f843 test and document new twitter wordlists
Former-commit-id: 14cb408100
2015-07-01 17:53:38 -04:00
Robyn Speer
a9b9b2f080 update data using new build
Former-commit-id: f9a9ee7a82
2015-07-01 11:18:39 -04:00
Robyn Speer
4997d776b9 case-fold instead of just lowercasing tokens
Former-commit-id: 638467f600
2015-06-30 15:14:02 -04:00
Joshua Chin
fbd15947bb revert changes to test_not_really_random
Former-commit-id: bbf7b9de34
2015-06-30 11:29:14 -04:00
Joshua Chin
9b02abb5ea changed english test to take random ascii words
Former-commit-id: a49b66880e
2015-06-29 11:05:01 -04:00
Joshua Chin
d10109bb38 changed japanese test because the most common japanese ascii word keeps changing
Former-commit-id: 5ed03b006c
2015-06-29 11:04:19 -04:00
Joshua Chin
fa89956df3 Japanese people do not 'lol', they 'w'
Former-commit-id: 17f11ebd26
2015-06-29 11:01:13 -04:00
Joshua Chin
a0b7211451 updated tests for emoji splitting
Former-commit-id: 3bcb3e84a1
2015-06-25 11:25:51 -04:00
Robyn Speer
f3958d63ae Switch to a more precise centibel scale.
Former-commit-id: 7862a4d2b6
2015-06-22 17:36:30 -04:00
Joshua Chin
4706a38c7a updated test because the new tokenizer removes URLs
Former-commit-id: 35f472fcf9
2015-06-18 11:38:28 -04:00
Robyn Speer
26517c1b86 tests for new wordfreq with full coverage
Former-commit-id: df863a5169
2015-05-21 20:34:17 -04:00