Compare commits

..

10 Commits

Author SHA1 Message Date
Guillaume Becquin 3b0952fedd Addition of Rust model 2022-02-11 19:59:44 +00:00
Patrick von Platen 3a68f56d9f upload flax model 2021-05-20 17:48:51 +00:00
Patrick von Platen 3ff5798015 allow flax 2021-05-20 17:48:33 +00:00
Julien Chaumond aa14b80177 Migrate model card from transformers-repo
Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/microsoft/codebert-base/README.md
2020-12-11 22:50:44 +01:00
system 0fef132c5f Update tf_model.h5 2020-07-10 11:33:29 +00:00
system b5e922596c Update vocab.json 2020-07-10 11:33:10 +00:00
system 9bc0ec8532 Update pytorch_model.bin 2020-07-10 11:25:59 +00:00
system 8b0f31ac5f Update merges.txt 2020-07-10 11:25:43 +00:00
system 5319141f05 Update config.json 2020-07-10 11:25:37 +00:00
system 1a3831ef57 Update special_tokens_map.json 2020-07-10 11:25:29 +00:00
7 changed files with 41 additions and 0 deletions

1
.gitattributes vendored
View File

@ -6,3 +6,4 @@
*.tar.gz filter=lfs diff=lfs merge=lfs -text *.tar.gz filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text *.ot filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text *.onnx filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text

27
README.md Normal file
View File

@ -0,0 +1,27 @@
## CodeBERT-base
Pretrained weights for [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://arxiv.org/abs/2002.08155).
### Training Data
The model is trained on bi-modal data (documents & code) of [CodeSearchNet](https://github.com/github/CodeSearchNet)
### Training Objective
This model is initialized with Roberta-base and trained with MLM+RTD objective (cf. the paper).
### Usage
Please see [the official repository](https://github.com/microsoft/CodeBERT) for scripts that support "code search" and "code-to-document generation".
### Reference
1. [CodeBERT trained with Masked LM objective](https://huggingface.co/microsoft/codebert-base-mlm) (suitable for code completion)
2. 🤗 [Hugging Face's CodeBERTa](https://huggingface.co/huggingface/CodeBERTa-small-v1) (small size, 6 layers)
### Citation
```bibtex
@misc{feng2020codebert,
title={CodeBERT: A Pre-Trained Model for Programming and Natural Languages},
author={Zhangyin Feng and Daya Guo and Duyu Tang and Nan Duan and Xiaocheng Feng and Ming Gong and Linjun Shou and Bing Qin and Ting Liu and Daxin Jiang and Ming Zhou},
year={2020},
eprint={2002.08155},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```

BIN
flax_model.msgpack (Stored with Git LFS) Normal file

Binary file not shown.

BIN
pytorch_model.bin (Stored with Git LFS) Normal file

Binary file not shown.

BIN
rust_model.ot (Stored with Git LFS) Normal file

Binary file not shown.

BIN
tf_model.h5 (Stored with Git LFS) Normal file

Binary file not shown.

1
vocab.json Normal file

File diff suppressed because one or more lines are too long