Compare commits

...

10 Commits

Author SHA1 Message Date
Philipp Schmid e49b3d60d9 Update README.md 2022-12-23 19:48:57 +00:00
Philipp Schmid 32d5883cf0 Update README.md 2022-12-23 19:48:13 +00:00
Philipp Schmid 20cced441c Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator (#10)
- Add verifyToken field to verify evaluation results are produced by Hugging Face's automatic model evaluator (0a4d5e89fe6716df5491f488ce43ad0a9ee508a1)


Co-authored-by: Evaluation Bot <autoevaluator@users.noreply.huggingface.co>
2022-12-05 13:32:46 +00:00
Philipp Schmid 18b336aea4 Update README.md 2022-11-09 12:30:52 +00:00
philschmid 78e20b3792 Align task name and type with Hub taxonomy (#3)
- Align task name and type with Hub taxonomy (f608325cb62c5a840be48b6ad7e2ddbe7f2673fe)


Co-authored-by: Lewis Tunstall <lewtun@users.noreply.huggingface.co>
2022-07-04 13:10:54 +00:00
philschmid cc1cf6c22f change nlp to summarizer to match earlier code (#2)
- change nlp to summarizer to match earlier code (87dc0aaeefda339b47f7ac6f610fac751d6c844f)


Co-authored-by: Nicholas Broad <nbroad@users.noreply.huggingface.co>
2022-07-01 19:38:36 +00:00
philschmid 3d05f288d9 Add evaluation results on samsum dataset (#1)
- Add evaluation results on samsum dataset (09b79fe340dd38f362d80c7392f8886251d42ff8)


Co-authored-by: Evaluation Bot <autoevaluator@users.noreply.huggingface.co>
2022-06-24 11:26:48 +00:00
Philipp Schmid 01473c6b7b Update README.md 2021-12-13 13:22:44 +00:00
Philipp Schmid accef2cc45 Update README.md 2021-04-01 14:19:26 +00:00
Philipp Schmid 438ac1476d Update README.md 2021-04-01 14:18:51 +00:00
1 changed files with 73 additions and 33 deletions

106
README.md
View File

@ -1,53 +1,93 @@
---
language: en
license: mit
tags:
- sagemaker
- bart
- summarization
license: apache-2.0
datasets:
- samsum
widget:
- text: "Jeff: Can I train a \U0001F917 Transformers model on Amazon SageMaker? \n\
Philipp: Sure you can use the new Hugging Face Deep Learning Container. \nJeff:\
\ ok.\nJeff: and how can I get started? \nJeff: where can I find documentation?\
\ \nPhilipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face\n"
model-index:
- name: bart-large-cnn-samsum
results:
- task:
name: Abstractive Text Summarization
type: abstractive-text-summarization
- task:
type: summarization
name: Summarization
dataset:
name: SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization
name: 'SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization'
type: samsum
metrics:
- name: Validation ROGUE-1
type: rogue-1
value: 42.621
- name: Validation ROGUE-2
type: rogue-2
value: 21.9825
- name: Validation ROGUE-L
type: rogue-l
value: 33.034
- name: Test ROGUE-1
type: rogue-1
value: 41.3174
- name: Test ROGUE-2
type: rogue-2
value: 20.8716
- name: Test ROGUE-L
type: rogue-l
value: 32.1337
widget:
- text: |
Jeff: Can I train a 🤗 Transformers model on Amazon SageMaker?
Philipp: Sure you can use the new Hugging Face Deep Learning Container.
Jeff: ok.
Jeff: and how can I get started?
Jeff: where can I find documentation?
Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
- type: rogue-1
value: 42.621
name: Validation ROGUE-1
- type: rogue-2
value: 21.9825
name: Validation ROGUE-2
- type: rogue-l
value: 33.034
name: Validation ROGUE-L
- type: rogue-1
value: 41.3174
name: Test ROGUE-1
- type: rogue-2
value: 20.8716
name: Test ROGUE-2
- type: rogue-l
value: 32.1337
name: Test ROGUE-L
- task:
type: summarization
name: Summarization
dataset:
name: samsum
type: samsum
config: samsum
split: test
metrics:
- type: rouge
value: 41.3282
name: ROUGE-1
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZTYzNzZkZDUzOWQzNGYxYTJhNGE4YWYyZjA0NzMyOWUzMDNhMmVhYzY1YTM0ZTJhYjliNGE4MDZhMjhhYjRkYSIsInZlcnNpb24iOjF9.OOM6l3v5rJCndmUIJV-2SDh2NjbPo5IgQOSL-Ju1Gwbi1voL5amsDEDOelaqlUBE3n55KkUsMLZhyn66yWxZBQ
- type: rouge
value: 20.8755
name: ROUGE-2
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMWZiODFiYWQzY2NmOTc5YjA3NTI0YzQ1MzQ0ODk2NjgyMmVlMjA5MjZiNTJkMGRmZGEzN2M3MDNkMjkxMDVhYSIsInZlcnNpb24iOjF9.b8cPk2-IL24La3Vd0hhtii4tRXujh5urAwy6IVeTWHwYfXaURyC2CcQOWtlOx5bdO5KACeaJFrFBCGgjk-VGCQ
- type: rouge
value: 32.1353
name: ROUGE-L
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWNmYzdiYWQ2ZWRkYzRiMGMxNWUwODgwZTdkY2NjZTc1NWE5NTFiMzU0OTU1N2JjN2ExYWQ2NGZkNjk5OTc4YSIsInZlcnNpb24iOjF9.Fzv4p-TEVicljiCqsBJHK1GsnE_AwGqamVmxTPI0WBNSIhZEhliRGmIL_z1pDq6WOzv3GN2YUGvhowU7GxnyAQ
- type: rouge
value: 38.401
name: ROUGE-LSUM
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGI4MWY0NWMxMmQ0ODQ5MDhiNDczMDAzYzJkODBiMzgzYWNkMWM2YTZkZDJmNWJiOGQ3MmNjMGViN2UzYWI2ZSIsInZlcnNpb24iOjF9.7lw3h5k5lJ7tYFLZGUtLyDabFYd00l6ByhmvkW4fykocBy9Blyin4tdw4Xps4DW-pmrdMLgidHxBWz5MrSx1Bw
- type: loss
value: 1.4297215938568115
name: loss
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMzI0ZWNhNDM5YTViZDMyZGJjMDA1ZWFjYzNhOTdlOTFiNzhhMDBjNmM2MjA3ZmRkZjJjMjEyMGY3MzcwOTI2NyIsInZlcnNpb24iOjF9.oNaZsAtUDqGAqoZWJavlcW7PKx1AWsnkbhaQxadpOKk_u7ywJJabvTtzyx_DwEgZslgDETCf4MM-JKitZKjiDA
- type: gen_len
value: 60.0757
name: gen_len
verified: true
verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTgwYWYwMDRkNTJkMDM5N2I2MWNmYzQ3OWM1NDJmODUyZGViMGE4ZTdkNmIwYWM2N2VjZDNmN2RiMDE4YTYyYiIsInZlcnNpb24iOjF9.PbXTcNYX_SW-BuRQEcqyc21M7uKrOMbffQSAK6k2GLzTVRrzZxsDC57ktKL68zRY8fSiRGsnknOwv-nAR6YBCQ
---
## `bart-large-cnn-samsum`
> If you want to use the model you should try a newer fine-tuned FLAN-T5 version [philschmid/flan-t5-base-samsum](https://huggingface.co/philschmid/flan-t5-base-samsum) out socring the BART version with `+6` on `ROGUE1` achieving `47.24`.
# TRY [philschmid/flan-t5-base-samsum](https://huggingface.co/philschmid/flan-t5-base-samsum)
This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
For more information look at:
@ -88,7 +128,7 @@ Jeff: and how can I get started?
Jeff: where can I find documentation?
Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
'''
nlp(conversation)
summarizer(conversation)
```
## Results