10 Commits

Author	SHA1	Message	Date
Rob Speer	f9a9ee7a82	update data using new build	2015-07-01 11:18:39 -04:00
Rob Speer	638467f600	case-fold instead of just lowercasing tokens	2015-06-30 15:14:02 -04:00
Joshua Chin	bbf7b9de34	revert changes to test_not_really_random	2015-06-30 11:29:14 -04:00
Joshua Chin	a49b66880e	changed english test to take random ascii words	2015-06-29 11:05:01 -04:00
Joshua Chin	5ed03b006c	changed japanese test because the most common japanese ascii word keeps changing	2015-06-29 11:04:19 -04:00
Joshua Chin	17f11ebd26	Japanese people do not 'lol', they 'w'	2015-06-29 11:01:13 -04:00
Joshua Chin	3bcb3e84a1	updated tests for emoji splitting	2015-06-25 11:25:51 -04:00
Rob Speer	7862a4d2b6	Switch to a more precise centibel scale.	2015-06-22 17:36:30 -04:00
Joshua Chin	35f472fcf9	updated test because the new tokenizer removes URLs	2015-06-18 11:38:28 -04:00
Rob Speer	df863a5169	tests for new wordfreq with full coverage	2015-05-21 20:34:17 -04:00