salti/bert-base-multilingual-cased-finetuned-squad is a forked repo from huggingface. License: None

Go to file

Patrick von Platen 161c56eeb8 allow flax		2021-05-19 01:26:15 +00:00
.gitattributes	allow flax	2021-05-19 01:26:15 +00:00
README.md	add datasets	2021-03-10 01:57:48 +00:00
config.json	Update model	2021-03-10 00:22:40 +00:00
pytorch_model.bin	Update model	2021-03-10 00:22:40 +00:00
special_tokens_map.json	Update special_tokens_map.json	2020-05-16 02:06:05 +00:00
tf_model.h5	Add TF model	2021-03-10 00:52:39 +00:00
tokenizer_config.json	Update model	2021-03-10 00:22:40 +00:00
training_args.bin	Update model	2021-03-10 00:22:40 +00:00
vocab.txt	Update vocab.txt	2020-05-16 02:06:07 +00:00

README.md

language

datasets

multilingual

squad

arcd

xquad

Multilingual BERT fine-tuned on SQuADv1.1

WandB run link

GPU: Tesla P100-PCIE-16GB

Training Arguments

max_seq_length              = 512
doc_stride                  = 256
max_answer_length           = 64
bacth_size                  = 16
gradient_accumulation_steps = 2
learning_rate               = 5e-5
weight_decay                = 3e-7
num_train_epochs            = 3
warmup_ratio                = 0.1
fp16                        = True
fp16_opt_level              = "O1"
seed                        = 0

Results

EM	F1
81.731	89.009

Zero-shot performance

on ARCD

EM	F1
20.655	48.051

on XQuAD

Language	EM	F1
Arabic	42.185	57.803
English	73.529	85.01
German	55.882	72.555
Greek	45.21	62.207
Spanish	58.067	76.406
Hindi	40.588	55.29
Russian	55.126	71.617
Thai	26.891	39.965
Turkish	34.874	51.138
Vietnamese	47.983	68.125
Chinese	47.395	58.928