From b41a392439b3378e702c77da0b78bf3f8e6aa8bc Mon Sep 17 00:00:00 2001 From: Leo Gao Date: Fri, 21 May 2021 00:00:44 +0000 Subject: [PATCH] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5b5e996..710fadf 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ GPT-Neo 2.7B was trained on the Pile, a large scale curated dataset created by E ## Training procedure -This model was trained for 400,000 steps on the Pile. It was trained as a masked autoregressive language model, using cross-entropy loss. +This model was trained for 420 billion tokens over 400,000 steps. It was trained as a masked autoregressive language model, using cross-entropy loss. ## Intended Use and Limitations