42189dea11
change hebrew code from iw (deprecated) to he ( #6 )
...
- change hebrew code from iw (deprecated) to he (2f637251702bb9799886af508a2863d3584bbb1a)
Co-authored-by: Altryne <altryne@users.noreply.huggingface.co>
2023-01-31 20:11:57 +00:00
a1790163fb
Changes for fast tokenizer ( #5 )
...
- Add tokenizer.json (dcca07232bfb1028e499333730f868b87fd3d043)
- Update unknown token (8852c40b30c9b7b981faf4fa77167fd862fd5fdb)
- Move <|endoftext|> from added_tokens.json to vocab.json (3e9581879a6134abfb58f5788096027dd1756a63)
Co-authored-by: Jonatan Kłosko <jonatanklosko@users.noreply.huggingface.co>
2023-01-25 10:17:42 +00:00
0ce71a9bbd
Upload tokenizer
2022-10-05 16:27:05 +00:00
c65924544a
Upload tokenizer
2022-09-28 08:15:02 +00:00
8e72aa6038
Upload tokenizer
2022-09-27 09:55:50 +00:00
1dff7eb67e
Upload processor
2022-09-27 07:04:56 +00:00
25da2ff16c
Upload processor
2022-09-26 06:50:41 +00:00
Arthur Zucker