From 68499bf5d67d0ad8f7329c90af430dd4495d1b2c Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Fri, 11 Dec 2020 22:24:07 +0100 Subject: [PATCH] Migrate model card from transformers-repo Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755 Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/distilbert-base-uncased-finetuned-sst-2-english-README.md --- README.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..d33b586 --- /dev/null +++ b/README.md @@ -0,0 +1,19 @@ +--- +language: en +license: apache-2.0 +datasets: +- sst-2 +--- + +# DistilBERT base uncased finetuned SST-2 + +This model is a fine-tune checkpoint of [DistilBERT-base-uncased](https://huggingface.co/distilbert-base-uncased), fine-tuned on SST-2. +This model reaches an accuracy of 91.3 on the dev set (for comparison, Bert bert-base-uncased version reaches an accuracy of 92.7). + +# Fine-tuning hyper-parameters + +- learning_rate = 1e-5 +- batch_size = 32 +- warmup = 600 +- max_seq_length = 128 +- num_train_epochs = 3.0