A developer reported training a 3.8-billion-parameter model from scratch on eight B200 GPUs for about 43 hours at a cost of $998. It consumed roughly 65.3 billion tokens and achieved a reported CORE score of 0.384. The account covers data mixtures, Muon optimization and FP8 throughput improvements. The price describes that training run, not failed experiments or all development work. Context length also materially affected evaluation results. This is a technical account of low-cost training, not independent validation or evidence of parity with commercial frontier models.
Back to Tech
Training a 3.8-billion-parameter model for $998
The reported training run is not the project's entire development cost.