iskm/wordfreq
1
0
Fork 0
mirror of https://github.com/rspeer/wordfreq.git synced 2024-12-23 09:21:37 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

stop running 'remove_unsafe_private_use' unnecessarily

Browse Source
This commit is contained in:
Rob Speer 2015-02-17 14:02:36 -05:00
parent 6ab72201cd
commit b6f246ecbb
1 changed files with 1 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
wordfreq/util.py
View File

@ -1,6 +1,5 @@
# coding: utf-8
from unicodedata import normalize
from ftfy.fixes import remove_unsafe_private_use
def standardize_word(word):
@ -21,4 +20,4 @@ def standardize_word(word):
that the capitalized versions will not share a word count with the
lowercase versions.
"""
return normalize('NFKC', remove_unsafe_private_use(word)).lower()
return normalize('NFKC', word).lower()