Allow single quotes "'" and hyphens "-" (#3)

- Allow single quotes "'" and hyphens "-" (a5796832d706e2ccafe86bdf823d50b75b4a6b2c)


Co-authored-by: Sanchit Gandhi <sanchit-gandhi@users.noreply.huggingface.co>
This commit is contained in:
Arthur Zucker 2022-11-28 08:31:03 +00:00 committed by system
parent f8fb469f66
commit 5ab69dbb40
1 changed files with 0 additions and 2 deletions

View File

@ -50,12 +50,10 @@
"suppress_tokens": [ "suppress_tokens": [
1, 1,
2, 2,
6,
7, 7,
8, 8,
9, 9,
10, 10,
12,
14, 14,
25, 25,
26, 26,