salti
/
bert-base-multilingual-cased-finetuned-squad
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
salti/bert-base-multilingual-cased-finetuned-squad is a forked repo from huggingface. License: None
18 Commits 1 Branch 0 Tags 2 GiB
Text 100%
Go to file
Patrick von Platen 0368c75d05 upload flax model 2021-05-19 01:26:36 +00:00
.gitattributes allow flax 2021-05-19 01:26:15 +00:00
README.md add datasets 2021-03-10 01:57:48 +00:00
config.json Update model 2021-03-10 00:22:40 +00:00
flax_model.msgpack upload flax model 2021-05-19 01:26:36 +00:00
pytorch_model.bin Update model 2021-03-10 00:22:40 +00:00
special_tokens_map.json Update special_tokens_map.json 2020-05-16 02:06:05 +00:00
tf_model.h5 Add TF model 2021-03-10 00:52:39 +00:00
tokenizer_config.json Update model 2021-03-10 00:22:40 +00:00
training_args.bin Update model 2021-03-10 00:22:40 +00:00
vocab.txt Update vocab.txt 2020-05-16 02:06:07 +00:00

README.md

language datasets
multilingual
squad
arcd
xquad

Multilingual BERT fine-tuned on SQuADv1.1

WandB run link

GPU: Tesla P100-PCIE-16GB

Training Arguments

max_seq_length              = 512
doc_stride                  = 256
max_answer_length           = 64
bacth_size                  = 16
gradient_accumulation_steps = 2
learning_rate               = 5e-5
weight_decay                = 3e-7
num_train_epochs            = 3
warmup_ratio                = 0.1
fp16                        = True
fp16_opt_level              = "O1"
seed                        = 0

Results

EM F1
81.731 89.009

Zero-shot performance

on ARCD

EM F1
20.655 48.051

on XQuAD

Language EM F1
Arabic 42.185 57.803
English 73.529 85.01
German 55.882 72.555
Greek 45.21 62.207
Spanish 58.067 76.406
Hindi 40.588 55.29
Russian 55.126 71.617
Thai 26.891 39.965
Turkish 34.874 51.138
Vietnamese 47.983 68.125
Chinese 47.395 58.928