Skip to content

macOS#

A Mac joins a cluster as an inference machine: it serves Eos models with llama.cpp, natively, on the Apple GPU through Metal, and runs the jobs that fetch their weights. It runs no Linux containers. Use this page to add a Mac, keep it serving, and know its limits before you plan on it.

Before you begin#

  • macOS 13.3 or newer (llama.cpp's own minimum).
  • Apple silicon for the GPU. An Intel Mac joins too, but serves on its CPU: llama.cpp's Intel macOS build has no Metal.
  • An administrator account: the command uses sudo.
  • Nothing else: no Homebrew, no Xcode, no Docker. The release brings the pinned llama.cpp server.
  • Outbound HTTPS to the Astraeus address and to the releases address, as for any machine (Network and firewalls).

What a Mac runs, and what it refuses#

On a Mac
Eos deployments of GGUF models on llama.cpp Yes, natively, with Metal on Apple silicon.
The run that fetches a model's weights into its drive Yes, in the Mac's own shell.
General runs (Linux container images) No. Refused with "Linux containers on a Mac arrive with the VM; this machine runs Eos models". The scheduler never places them there.
vLLM No. The fit says so; use a GGUF of the model.
Privileged runs, exec into a worker No.
A run's outbound network policy No: engines run on the host network, where it cannot be enforced. Such runs are refused.
The WireGuard mesh No. Workers use the Mac's own network.
Drives served to other machines, credentials, data sources No: only the agent's main part runs on a Mac (not its drives, credentials or data parts). Gated Hugging Face models, whose token is a credential, cannot be fetched on a Mac.
CPU and memory limits Not enforced (macOS has no cgroups). llama.cpp's context size and parallelism bound its memory.

Each model is served by llama-server on the host network, on port 8000: one replica of a deployment per Mac.

Add a Mac#

  1. In the console, open Compute → Machines → Add machine, choose the pool, and click Get the install command (Add a machine).
  2. Open Terminal on the Mac, paste the command and press Return. Enter your password when sudo asks.
  3. Answer where to keep model weights. Press Return to accept /Users/Shared/Astraeus.
$ echo '9c4e…' > ./astraeus-token && curl -fsSL https://console.astralyx.cloud/api/v1/install.sh | sudo sh -s -- --apiserver https://api.astralyx.cloud --token-file ./astraeus-token --releases https://console.astralyx.cloud/releases --org 'Acme Research' --org-id 0192f0c4-7d1e-7a51-9c33-5e8b2a4f6d10 && rm -f ./astraeus-token
Password:
Where should this Mac keep model weights? [/Users/Shared/Astraeus]
· downloading astraeus-agent-darwin-arm64.tar.gz (latest)
· installing
· starting
Installed on this Mac:
  the agent        /usr/local/bin/astraeus-agent (runs as cloud.astralyx.astraeus-agent, launchd)
  llama.cpp        /usr/local/lib/astraeus/engines (native, Metal on the Apple GPU)
  configuration    /etc/astraeus/agent.env (the token: /etc/astraeus/token, root only)
  its log          /Library/Logs/Astraeus/agent.log
  network          the host network (no mesh on a Mac yet): one replica of a deployment per Mac
  model weights    /Users/Shared/Astraeus
  macOS lists it under System Settings > General > Login Items (Allow in the Background): leave it on.
  sleep            kept awake while a model is served. To keep it reachable at all times: sudo pmset -a sleep 0
· mac-studio is connected to https://api.astralyx.cloud

The Mac's name is its local host name (scutil --get LocalHostName) in lower case, unless you pass --name.

Where to keep model weights#

The data location must be a folder a background service may read. The installer refuses:

  • folders macOS protects with privacy settings: Documents, Desktop, Downloads, iCloud Drive (Library/Mobile Documents) and cloud storage folders (Library/CloudStorage) of any user;
  • anything under /Volumes (external and network volumes): macOS would ask permission, and nobody is there to answer.

/Users/Shared/Astraeus is recommended. Without a terminal, pass --data-dir, or --no-data-dir to choose later in the console.

Options that do not apply#

--runtime, the NVIDIA and AMD options and --agents are ignored on a Mac, with a message. --network mesh is an error. See the Installer reference.

Keep it serving#

  • Allow in the Background. macOS lists the agent under System Settings → General → Login Items → Allow in the Background. Leave it on, or the agent does not run.

  • Sleep. A Mac asleep drops off the cluster. While a model is served, each engine holds a caffeinate assertion, so a desktop, or a laptop on power, stays awake. A laptop still sleeps when its lid closes. For a Mac that should answer at all times:

    $ sudo pmset -a sleep 0

  • Firewall. When the macOS application firewall is on, the installer allows llama-server to accept connections, so no prompt blocks it.

GPU memory#

The Apple GPU is reported as one GPU of vendor apple. Its memory is what Metal lets the GPU hold (the working set):

  • the GPU wired-memory limit, when an administrator set one (sysctl iogpu.wired_limit_mb, macOS 14 or newer);
  • otherwise two thirds of the RAM up to 36 GiB, and three quarters above.

The GPU and the CPU share one memory, so the scheduler also charges what a model holds on the GPU to the Mac's RAM: it is not promised twice. Memory the desktop uses on the GPU is not counted against it.

To give models more room on a Mac dedicated to serving:

$ sudo sysctl iogpu.wired_limit_mb=57344

The value does not survive a restart; set it again at boot if you rely on it.

Operate a Mac#

Task Command
Follow the log tail -f /Library/Logs/Astraeus/agent.log
Restart the agent sudo launchctl kickstart -k system/cloud.astralyx.astraeus-agent
Change the configuration Edit /etc/astraeus/agent.env, then restart the agent
See a model's output /var/lib/astraeus/worker/native/<id>/log

Each model runs as its own llama-server process, supervised in a session of its own: it keeps serving while the agent restarts, and the new agent adopts it. Update from the console downloads the macOS release, replaces the files under /usr/local and restarts the agent through launchd.

A Mac installed before October 2026

Its agent was /usr/local/bin/astraeus-worker, run as cloud.astralyx.astraeus-worker, configured by /etc/astraeus/worker.env and logging to /Library/Logs/Astraeus/worker.log. Update from the console keeps those names on a Mac. Run the install command again to move it to the names above: the configuration, the name and the data location are kept.

Remove a Mac#

$ curl -fsSL https://console.astralyx.cloud/api/v1/install.sh | sudo sh -s -- --uninstall
· uninstalling the Astraeus agent
· kept the data location /Users/Shared/Astraeus (drive copies): delete it by hand if nothing there is needed
· the Astraeus agent is uninstalled
  If this Mac is still listed in its cluster, remove it there (the console: the machine's page, Remove machine).

It stops the agent and the engines it started, and removes the launchd job, /usr/local/bin/astraeus-agent, /usr/local/bin/astraeus, /usr/local/lib/astraeus, /etc/astraeus, /var/lib/astraeus/worker and /Library/Logs/Astraeus. Add --purge to delete the model weights too. Remove the Mac from the cluster in the console as well (Update, drain and remove).