| language |
license |
tags |
datasets |
widget |
model-index |
| en |
mit |
| sagemaker |
| bart |
| summarization |
|
|
|
| name |
results |
| bart-large-cnn-samsum |
| task |
dataset |
metrics |
| type |
name |
| summarization |
Summarization |
|
| name |
type |
| SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization |
samsum |
|
| type |
value |
name |
| rogue-1 |
42.621 |
Validation ROGUE-1 |
|
| type |
value |
name |
| rogue-2 |
21.9825 |
Validation ROGUE-2 |
|
| type |
value |
name |
| rogue-l |
33.034 |
Validation ROGUE-L |
|
| type |
value |
name |
| rogue-1 |
41.3174 |
Test ROGUE-1 |
|
| type |
value |
name |
| rogue-2 |
20.8716 |
Test ROGUE-2 |
|
| type |
value |
name |
| rogue-l |
32.1337 |
Test ROGUE-L |
|
|
|
| task |
dataset |
metrics |
| type |
name |
| summarization |
Summarization |
|
| name |
type |
config |
split |
| samsum |
samsum |
samsum |
test |
|
| type |
value |
name |
verified |
verifyToken |
| rouge |
41.3282 |
ROUGE-1 |
true |
eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZTYzNzZkZDUzOWQzNGYxYTJhNGE4YWYyZjA0NzMyOWUzMDNhMmVhYzY1YTM0ZTJhYjliNGE4MDZhMjhhYjRkYSIsInZlcnNpb24iOjF9.OOM6l3v5rJCndmUIJV-2SDh2NjbPo5IgQOSL-Ju1Gwbi1voL5amsDEDOelaqlUBE3n55KkUsMLZhyn66yWxZBQ |
|
| type |
value |
name |
verified |
verifyToken |
| rouge |
20.8755 |
ROUGE-2 |
true |
eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWZiODFiYWQzY2NmOTc5YjA3NTI0YzQ1MzQ0ODk2NjgyMmVlMjA5MjZiNTJkMGRmZGEzN2M3MDNkMjkxMDVhYSIsInZlcnNpb24iOjF9.b8cPk2-IL24La3Vd0hhtii4tRXujh5urAwy6IVeTWHwYfXaURyC2CcQOWtlOx5bdO5KACeaJFrFBCGgjk-VGCQ |
|
| type |
value |
name |
verified |
verifyToken |
| rouge |
32.1353 |
ROUGE-L |
true |
eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWNmYzdiYWQ2ZWRkYzRiMGMxNWUwODgwZTdkY2NjZTc1NWE5NTFiMzU0OTU1N2JjN2ExYWQ2NGZkNjk5OTc4YSIsInZlcnNpb24iOjF9.Fzv4p-TEVicljiCqsBJHK1GsnE_AwGqamVmxTPI0WBNSIhZEhliRGmIL_z1pDq6WOzv3GN2YUGvhowU7GxnyAQ |
|
| type |
value |
name |
verified |
verifyToken |
| rouge |
38.401 |
ROUGE-LSUM |
true |
eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGI4MWY0NWMxMmQ0ODQ5MDhiNDczMDAzYzJkODBiMzgzYWNkMWM2YTZkZDJmNWJiOGQ3MmNjMGViN2UzYWI2ZSIsInZlcnNpb24iOjF9.7lw3h5k5lJ7tYFLZGUtLyDabFYd00l6ByhmvkW4fykocBy9Blyin4tdw4Xps4DW-pmrdMLgidHxBWz5MrSx1Bw |
|
| type |
value |
name |
verified |
verifyToken |
| loss |
1.4297215938568115 |
loss |
true |
eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzI0ZWNhNDM5YTViZDMyZGJjMDA1ZWFjYzNhOTdlOTFiNzhhMDBjNmM2MjA3ZmRkZjJjMjEyMGY3MzcwOTI2NyIsInZlcnNpb24iOjF9.oNaZsAtUDqGAqoZWJavlcW7PKx1AWsnkbhaQxadpOKk_u7ywJJabvTtzyx_DwEgZslgDETCf4MM-JKitZKjiDA |
|
| type |
value |
name |
verified |
verifyToken |
| gen_len |
60.0757 |
gen_len |
true |
eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTgwYWYwMDRkNTJkMDM5N2I2MWNmYzQ3OWM1NDJmODUyZGViMGE4ZTdkNmIwYWM2N2VjZDNmN2RiMDE4YTYyYiIsInZlcnNpb24iOjF9.PbXTcNYX_SW-BuRQEcqyc21M7uKrOMbffQSAK6k2GLzTVRrzZxsDC57ktKL68zRY8fSiRGsnknOwv-nAR6YBCQ |
|
|
|
|
|
|
bart-large-cnn-samsum
If you want to use the model you should try a newer fine-tuned FLAN-T5 version philschmid/flan-t5-base-samsum out socring the BART version with +6 on ROGUE1 achieving 47.24.
This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
For more information look at:
Hyperparameters
{
"dataset_name": "samsum",
"do_eval": true,
"do_predict": true,
"do_train": true,
"fp16": true,
"learning_rate": 5e-05,
"model_name_or_path": "facebook/bart-large-cnn",
"num_train_epochs": 3,
"output_dir": "/opt/ml/model",
"per_device_eval_batch_size": 4,
"per_device_train_batch_size": 4,
"predict_with_generate": true,
"seed": 7
}
Usage
from transformers import pipeline
summarizer = pipeline("summarization", model="philschmid/bart-large-cnn-samsum")
conversation = '''Jeff: Can I train a 🤗 Transformers model on Amazon SageMaker?
Philipp: Sure you can use the new Hugging Face Deep Learning Container.
Jeff: ok.
Jeff: and how can I get started?
Jeff: where can I find documentation?
Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
'''
summarizer(conversation)
Results
| key |
value |
| eval_rouge1 |
42.621 |
| eval_rouge2 |
21.9825 |
| eval_rougeL |
33.034 |
| eval_rougeLsum |
39.6783 |
| test_rouge1 |
41.3174 |
| test_rouge2 |
20.8716 |
| test_rougeL |
32.1337 |
| test_rougeLsum |
38.4149 |