PygmalionAI
/
pygmalion-2.7b
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
PygmalionAI/pygmalion-2.7b is a forked repo from huggingface. License: creativeml-openrail-m
6 Commits 1 Branch 0 Tags 5.1 GiB
Text 100%
Go to file
0x000011b 9533805293 revert "convert to safetensors" 
This reverts commit e6202634ab.

As per Henk's suggestion. Safetensor support on Kobold is still a
little iffy, so let's keep the usual pickled models for now.
 		2023-01-12 19:20:02 +00:00
runs initial commit 2023-01-12 18:58:12 +00:00
.gitattributes initial commit 2023-01-12 18:58:12 +00:00
README.md Update README.md 2023-01-12 18:58:27 +00:00
config.json revert KAI chatmode config until related fixes land on stable version 2023-01-12 18:58:27 +00:00
merges.txt initial commit 2023-01-12 18:58:12 +00:00
pytorch_model.bin revert "convert to safetensors" 2023-01-12 19:20:02 +00:00
special_tokens_map.json initial commit 2023-01-12 18:58:12 +00:00
tokenizer.json initial commit 2023-01-12 18:58:12 +00:00
tokenizer_config.json initial commit 2023-01-12 18:58:12 +00:00
vocab.json initial commit 2023-01-12 18:58:12 +00:00

README.md

license language thumbnail tags inference
creativeml-openrail-m
en
text generation
conversational
false

Pygmalion 2.7B

Model description

Pymalion 2.7B is a proof-of-concept dialogue model based on EleutherAI's gpt-neo-2.7B.

Warning: This model is NOT suitable for use by minors. It will output X-rated content under certain circumstances.

Training data

The fine-tuning dataset consisted of 56MB of dialogue data gathered from multiple sources, which includes both real and partially machine-generated conversations.

Training procedure

Model weights were initialized from the uft-2.7b ConvoGPT model made available in this commit.

The model was then further fine-tuned on ~48.5 million tokens for ~5k steps on 4 NVIDIA A40s using DeepSpeed.

Intended use

The easy way

We provide a notebook with a Gradio UI for playing around with the model without having to manually format inputs. This notebook can be found here.

The manual way

The model can be used as a regular text generation model, but it'll perform best if the input prompt adheres to the following format:

[CHARACTER]'s Persona: [A few sentences about the character you want the model to play]
<START>
[DIALOGUE HISTORY]
You: [Your input message here]
[CHARACTER]:

Where [CHARACTER] is, as you can probably guess, the name of the character you want the model to portray, <START> should be used verbatim as a delimiter token to separate persona and scenario data from the dialogue, and [DIALOGUE HISTORY] is chat history so the model can have some conversational context to draw from. Ideally it'll be pairs of messages like:

[CHARACTER]: [some dialogue here]
You: [your response to the dialogue above]

Apart from chat history, you can also just add example conversations in [DIALOGUE HISTORY] to show how the character should speak - ideally at the beginning, so it doesn't get confused as to what's conversation history vs. character definition.

Known issues

We haven't played around with the model enough to enumerate them. Feel free to give us some feedback!