Add tokenizer configuration

This commit is contained in:
Thomas Wolf 2020-11-25 15:12:03 +01:00
parent 93a53c79f3
commit 182a12c8b0
1 changed files with 3 additions and 0 deletions

3
tokenizer_config.json Normal file
View File

@ -0,0 +1,3 @@
{
"do_lower_case": true
}