From f0937211408289ff1074afeaedf12953ab853bc1 Mon Sep 17 00:00:00 2001 From: mkshing Date: Wed, 11 May 2022 10:39:40 +0900 Subject: [PATCH] update README.md --- README.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/README.md b/README.md index 857f646..0d497e5 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,9 @@ --- language: ja thumbnail: https://github.com/rinnakk/japanese-pretrained-models/blob/master/rinna.png +license: apache-2.0 tags: +- feature-extraction - ja - japanese - clip @@ -56,3 +58,13 @@ with torch.no_grad(): print("Label probs:", text_probs) # prints: [[1.0, 0.0, 0.0]] ``` +# Model architecture +The model was trained a ViT-B/16 Transformer architecture as an image encoder and uses a 12-layer RoBERTa as a text encoder. The text encoder was trained upon the pre-trained Japanese RoBERTa model [rinna/japanese-roberta-base](https://huggingface.co/rinna/japanese-roberta-base) with the same sentencepiece tokenizer. + +# Training +The model was trained on [CC12M](https://github.com/google-research-datasets/conceptual-12m) translated the captions to Japanese. + + +# License + +[Apache-2.0 license](https://www.apache.org/licenses/LICENSE-2.0)