iskm/wordfreq
1
0
Fork 0
mirror of https://github.com/rspeer/wordfreq.git synced 2024-12-23 17:31:41 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Document the NFKC-normalized ligature in the Arabic test.

Browse Source
This commit is contained in:
Andrew Lin 2015-08-03 11:09:44 -04:00
parent 6d40912ef9
commit 41e1dd41d8
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
tests/test.py
View File

@ -149,6 +149,6 @@ def test_ar():
)
eq_(
tokenize('\ufefb', 'ar'),
['\u0644\u0627']
tokenize('\ufefb', 'ar'), # An Arabic ligature...
['\u0644\u0627'] # ...that is affected by NFKC normalization
)