LLM360 is an initiative for comprehensive and fully open-sourced LLMs, where all training details, model checkpoints, intermediate results, and additional analyses are made available to the community. Our goal is to advance the field by inviting the community to deepen the understanding of LLMs together.
As the first step of the project LLM360, we release all intermediate model checkpoints, our fully-prepared pre-training dataset, all source code and configurations, and training details. We are committed to continually pushing the boundaries of LLMs through this open-source effort.
The Amber project includes: Amber, a 7B English language model with the LLaMA architecture; Amber-Chat, an instruction-following model finetuned from LLM360-Amber; and Amber-Safe, a safety-finetuned instruction model using LLM360-AmberChat as the base.