Upload processor

This commit is contained in:
Niels Rogge 2023-02-08 20:20:53 +00:00 committed by huggingface-web
parent 54b789ef16
commit 4c921565bf
4 changed files with 100365 additions and 50269 deletions

View File

@ -1,4 +1,4 @@
#version: 0.2
#version: 0.2 - Trained by `huggingface/tokenizers`
Ġ t
Ġ a
h e

100363
tokenizer.json Normal file

File diff suppressed because it is too large Load Diff

View File

@ -19,7 +19,6 @@
},
"errors": "replace",
"model_max_length": 1000000000000000019884624838656,
"name_or_path": "facebook/opt-2.7b",
"pad_token": {
"__type": "AddedToken",
"content": "<pad>",

50268
vocab.json

File diff suppressed because one or more lines are too long