cl-tohoku
/
bert-base-japanese-whole-word-masking
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update tokenizer_config.json

This commit is contained in:
system 2020-06-07 12:26:07 +00:00
parent f9b8d28251
commit 30931391d3
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
tokenizer_config.json Normal file
View File

@ -0,0 +1,5 @@
{
"do_lower_case": false,
"word_tokenizer_type": "mecab",
"subword_tokenizer_type": "wordpiece"
}