From 290cdf1029f6e8eeaad3bc8ddc08825b690608f3 Mon Sep 17 00:00:00 2001 From: Moritz Laurer Date: Sun, 5 Dec 2021 16:51:38 +0000 Subject: [PATCH] Update README.md --- README.md | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 8988478..29c656f 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,31 @@ +--- + --- language: +- multilingual - en tags: -- text-classification - zero-shot-classification +- text-classification +- nli +- pytorch metrics: - accuracy +datasets: +- mnli +- xnli +- anli +license: mit pipeline_tag: zero-shot-classification - +widget: +- text: "De pugna erat fantastic. Nam Crixo decem quam dilexit et praeciderunt caput aemulus." + candidate_labels: "violent, peaceful" +- text: "La película empezaba bien pero terminó siendo un desastre." + candidate_labels: "positivo, negativo, neutral" +- text: "La película empezó siendo un desastre pero en general fue bien." + candidate_labels: "positivo, negativo, neutral" +- text: "¿A quién vas a votar en 2020?" + candidate_labels: "Europa, elecciones, política, ciencia, deportes" --- # Multilingual mDeBERTa-v3-base-mnli-xnli ## Model description @@ -51,7 +69,11 @@ training_args = TrainingArguments( ### Eval results The model was evaluated using the matched test set and achieves 0.90 accuracy. +average | ar | bg | de | el | en | es | fr | hi | ru | sw | th | tr | ur | vu | zh +---------|----------|---------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|----------|---------- +0.808 | 0.802 | 0.829 | 0.825 | 0.826 | 0.883 | 0.845 | 0.834 | 0.771 | 0.813 | 0.748 | 0.793 | 0.807 | 0.740 | 0.795 | 0.8116 +{'ar': 0.8017964071856287, 'bg': 0.8287425149700599, 'de': 0.8253493013972056, 'el': 0.8267465069860279, 'en': 0.8830339321357286, 'es': 0.8449101796407186, 'fr': 0.8343313373253493, 'hi': 0.7712574850299401, 'ru': 0.8127744510978044, 'sw': 0.7483033932135729, 'th': 0.792814371257485, 'tr': 0.8065868263473054, 'ur': 0.7403193612774451, 'vi': 0.7954091816367266, 'zh': 0.8115768463073852} ## Limitations and bias Please consult the original DeBERTa-V3 paper and literature on different NLI datasets for potential biases.