Eos

Eos serves open models — Llama, Qwen, Gemma, Mistral, Whisper and more — on your own machines, with llama.cpp or vLLM, behind a gateway that checks API keys, counts tokens and scales idle deployments to zero. Try any deployment in the Playground, share it with another workspace, and call it with any OpenAI-compatible client.

Being written

The Eos documentation is being written to the same standard as Astraeus. Until it is published here, the guide inside the console (Docs in the console's menu) covers models, deployments, API keys, the Playground and sharing.

Eos runs on Astraeus: every model replica is work placed on your machines, so adding a machine and GPUs apply as they are.