Update README.md
This commit is contained in:
parent
f507f5c180
commit
3a1c032540
25
README.md
25
README.md
|
@ -11,4 +11,27 @@ metrics:
|
|||
|
||||
# DistilBERT base model (uncased)
|
||||
|
||||
This model is the Multi-Genre Natural Language Inference (MNLI) fine-turned version of the [uncased DistilBERT model](https://huggingface.co/distilbert-base-uncased).
|
||||
This is the [uncased DistilBERT model](https://huggingface.co/distilbert-base-uncased) fine-tuned on [Multi-Genre Natural Language Inference](https://huggingface.co/datasets/multi_nli) (MNLI) dataset for the zero-shot classification task. The model is not case-sensitive, i.e., it does not make a difference between "english" and "English".
|
||||
|
||||
## Training
|
||||
|
||||
Training is done on a [p3.2xlarge](https://aws.amazon.com/ec2/instance-types/p3/) AWS EC2 instance (1 NVIDIA Tesla V100 GPUs), with the following hyperparameters:
|
||||
|
||||
```
|
||||
$ run_glue.py \
|
||||
--model_name_or_path distilbert-base-uncased \
|
||||
--task_name mnli \
|
||||
--do_train \
|
||||
--do_eval \
|
||||
--max_seq_length 128 \
|
||||
--per_device_train_batch_size 16 \
|
||||
--learning_rate 2e-5 \
|
||||
--num_train_epochs 5 \
|
||||
--output_dir /tmp/distilbert-base-uncased_mnli/
|
||||
```
|
||||
|
||||
## Evaluation results
|
||||
|
||||
| Task | MNLI | MNLI-mm |
|
||||
|:----:|:----:|:----:|
|
||||
| | 82.0 | 82.0 |
|
Loading…
Reference in New Issue