Models

🔓 Open-source

Gradient gives you easy access to open-source LLMs. We host the models and you can focus on inference and fine-tuning.

Ownership & Control: You own your model completely. This includes your private data used for inference and fine-tuning (i.e., the model weights generated during the fine-tuning process).
Security & Privacy: When you create your own instance of an open-source model no one else can access it unless you decide to share it. Models created within a Gradient workspace can only be accessed by members of that workspace.
More Customization: When using open-source models you have full access the model's architecture and code. This allows you to better understand the model and completely customize it to fit your needs.
Open-Source Community: Open-source comes with a built-in community of developers who are actively building and sharing resources with each other.

The Gradient API hosts a broad range of language models for inference and fine-tuning. Fine-tuning is only supported for specific models.

Model Name	Release Date	Context Length	Feature Support	License
Bloom-560	May 2022	2048	Inference; Fine-tuning	The BigScience RAIL License
Llama-2-Chat-7B	July 2023	4096	Inference; Fine-tuning	LLaMA 2 License; Cannot use LLaMA outputs to train other LLMs besides LLaMA and its derivatives
Llama-2-Chat-70B	January 2024	4096	Inference; Fine-tuning**	LLaMA 2 License ; Cannot use LLaMA outputs to train other LLMs besides LLaMA and its derivatives
Llama-3-Chat-8B	April 2024	8k	Inference; Fine-tuning**	LLaMA 3 License
Llama-3-Chat-70B	April 2024	8k	Inference; Fine-tuning**	LLaMA 3 License
Mixtral-8x7B Instruct	January 2024	32k	Inference only	Apache-2.0 License
Nous-Hermes-Llama-2-13B	July 2023	4096	Inference; Fine-tuning	N/A

** Requires Enterprise License. If you don't already have one, please reach out to [email protected]

Model IDs for reference in the API and CLI

Language Model	Model ID	Slug ID
Bloom-560	99148c6d-c2a0-4fbe-a4a7-e7c05bdb8a09_base_ml_model	bloom-560m
Llama-2-Chat-7B	f0b97d96-51a8-4040-8b22-7940ee1fa24e_base_ml_model	llama2-7b-chat
Llama-2-Chat-70B	780dc245-41cc-4398-99ad-ea1a7d6ad011_base_ml_model	llama2-70b-chat
Llama-3-Chat-8B	62163345-b63f-4826-88f1-acd8e289f85e_base_ml_model	llama3-8b-chat
Llama-3-Chat-70B	07396fc3-64f1-43e7-b194-f10843e741f0_base_ml_model	llama3-70b-chat
Mixtral-8x7B Instruct	86898eca-d7eb-4ed5-aee4-6fe4f67c426b_base_ml_model	mixtral-8x7b-instruct
Nous-Hermes-Llama-2-13B	cc2dafce-9e6e-4a23-a918-cad6ba89e42e_base_ml_model	nous-hermes2

Language Model	Params	Release Date	Context Length	License
Starcoder	15.5B	May 2024	8192	OpenRAIL-M v1
Code Llama	7B, 13B, 34B	May 2024	4096	LLaMA 2 License; Cannot use LLaMA outputs to train other LLMs besides LLaMA and its derivatives

Note: Supported models are subject to change

The Gradient API also hosts embeddings models to convert your text data into generated embeddings.

Model	Release Date	License
bge-large	Aug 2023	MIT License

Note: Supported models are subject to change