diff --git a/README.md b/README.md index 63097ff..e3ab06b 100644 --- a/README.md +++ b/README.md @@ -4,11 +4,14 @@ language: datasets: - squad - --- # Multilingual BERT fine-tuned on SQuADv1.1 +[**WandB run link**](https://wandb.ai/salti/mBERT_QA/runs/wkqzhrp2) + +**GPU**: Tesla P100-PCIE-16GB + ## Training Arguments ```python @@ -28,6 +31,30 @@ seed = 0 ## Results -| EM | F1 | -| :---: | :---: | +| EM | F1 | +| :----: | :----: | | 81.731 | 89.009 | + +## Zero-shot performance + +### on ARCD + +| EM | F1 | +| :----: | :----: | +| 20.655 | 48.051 | + +### on XQuAD + +| Language | EM | F1 | +| :--------: | :----: | :----: | +| Arabic | 42.185 | 57.803 | +| English | 73.529 | 85.01 | +| German | 55.882 | 72.555 | +| Greek | 45.21 | 62.207 | +| Spanish | 58.067 | 76.406 | +| Hindi | 40.588 | 55.29 | +| Russian | 55.126 | 71.617 | +| Thai | 26.891 | 39.965 | +| Turkish | 34.874 | 51.138 | +| Vietnamese | 47.983 | 68.125 | +| Chinese | 47.395 | 58.928 |