Chat with your documents in Eos

You want to ask a chat model of yours about your own documents and get answers that say where they come from. Today a deployment does not search a knowledge base by itself: there is no setting on a deployment, or field on a chat request, that names a base to search. What you do instead is give the model a tool that searches the base — a small function of the workspace — and the built-in save_to_knowledge tool, so it can also keep a note when you ask. The gateways run both for the model, in the Playground, in the app's chat and for your applications.

flowchart LR
  P[You ask] --> M[chat deployment]
  M -- search_handbook --> F[function search-handbook]
  F -- POST /v1/search --> K[knowledge base handbook]
  K -- passages with citations --> M
  M --> A[Answer citing<br/>document and page]
  M -. save_to_knowledge .-> D[Draft: you Save or Discard]

What you need:

• A knowledge base, here handbook from Your company handbook, Ready.
• A chat deployment of a model that calls tools well, here chat: Qwen3 8B or larger, as in Give your model web search. Its Capabilities list tools.
• The editor role, astra signed in, and the variables in How the recipes are written.

1. Write the search function

Inside the workspace, a knowledge base answers searches at http://metis-<name>:7700/v1/search — the same body and answer as the search API, without a token, and only for the workspace's own work. The function calls it and gives the model each passage with where it is:

search_handbook.py

import json
import os
import urllib.error
import urllib.request

BASE_URL = os.environ.get("METIS_URL", "http://metis-handbook:7700").rstrip("/")


def handler(question: str, max_results: int = 5) -> dict:
    """Search the company handbook (policies, onboarding, benefits, travel) and
    return the passages that answer the question, each with where it is, to cite.

    Args:
        question: What to look for, in words: a question or a topic.
        max_results: How many passages to return, 1 to 10.
    """
    k = max(1, min(int(max_results or 5), 10))
    body = json.dumps({"query": question, "top_k": k}).encode()
    req = urllib.request.Request(f"{BASE_URL}/v1/search", data=body, method="POST",
                                 headers={"Content-Type": "application/json"})
    try:
        with urllib.request.urlopen(req, timeout=25) as r:
            answer = json.loads(r.read())
    except urllib.error.HTTPError as e:
        return {"error": f"the handbook answered {e.code}: {e.read().decode(errors='replace')[:300]}"}
    except OSError as e:
        return {"error": f"could not reach the handbook at {BASE_URL}: {e}"}
    passages = []
    for h in answer.get("results", []):
        c = h["citation"]
        where = " › ".join([c["document"], *(c.get("headings") or [])])
        if c.get("page"):
            where += f", page {c['page']}"
        elif c.get("line"):
            where += f", line {c['line']}"
        passages.append({"cite": where, "text": h["text"]})
    return {"passages": passages, "warnings": answer.get("warnings", [])}

It needs no package. The docstring is what the model is told the tool does: say what the base holds, so the model knows when to call it.

2. Publish it next to the base

CLIConsoleAPI

$ astra astraeus functions deploy search-handbook -f search_handbook.py --timeout 30 --near metis-handbook --publish --alias prod
$ astra astraeus functions invoke search-handbook@prod -d '{"question": "How many vacation days do I have?", "max_results": 2}'

Astraeus → Functions → New function: name search-handbook, runtime Python. Paste the code, set its timeout to 30 s and Run next to metis-handbook, Save draft, Test it with {"question": "How many vacation days do I have?"}, then publish it with the alias prod.

$ curl -fsS -X POST "$ASTRALYX_API/functions" -H "Authorization: Bearer $ASTRALYX_TOKEN" -H 'content-type: application/json' \
    -d "$(jq -n --rawfile code search_handbook.py '{metadata: {name: "search-handbook"}, spec: {runtime: "python", code: $code, timeout_seconds: 30, near: "metis-handbook"}}')" > /dev/null

Then publish it with the alias prod (Publish and roll out).

The test answers the base's passages:

{"passages": [{"cite": "policies/leave-policy.pdf, page 2",
               "text": "Full-time employees accrue 25 days of paid vacation per calendar year. …"},
              {"cite": "policies/leave-policy.pdf, page 3",
               "text": "Unused vacation days carry over to the next year, up to 5 days. …"}],
 "warnings": []}

Run next to metis-handbook (the base's own replica group) puts the function's instances on the base's machine: machines in different networks may not reach each other, and a machine always reaches itself. To search another base with the same code, deploy it again under another name with -e METIS_URL=http://metis-<base>:7700 and its own docstring.

3. Give the model both tools

The search function is added like any tool; save_to_knowledge is a built-in that names the base its notes are proposed for, and is set in the deployment's spec.

CLIConsoleAPI

$ astra eos tools chat add search-handbook@prod
chat: tools saved

astra eos tools chat add does not add save_to_knowledge: use the API tab for it.

Eos → Deployments → chat → Tools → Add a tool: search-handbook, alias prod. Review schema shows what the model is told. save_to_knowledge is added through the API.

Both at once — the deployment's spec, with the two tools appended:

$ curl -fsS "$ASTRALYX_API/eos/deployments/chat" -H "Authorization: Bearer $ASTRALYX_TOKEN" \
    | jq '{spec: (.spec | .tools = ((.tools // []) + [
          {"function": "search-handbook", "alias": "prod"},
          {"builtin": "save_to_knowledge", "knowledge_base": "handbook"}]))}' > chat-spec.json
$ curl -fsS -X PUT "$ASTRALYX_API/eos/deployments/chat" -H "Authorization: Bearer $ASTRALYX_TOKEN" \
    -H 'content-type: application/json' -d @chat-spec.json > /dev/null

If you already added search-handbook with the CLI, append only the save_to_knowledge entry.

The base named by save_to_knowledge must be of the deployment's workspace. The model is told to use it only when the person asks to save, keep or remember something, and that nothing is saved until the person confirms.

4. Ask it in the Playground

Eos → Playground, the deployment chat. In System prompt:

You answer questions about the company from the handbook. Call search_handbook first,
answer only from the passages it returns, and cite the "cite" of each passage you use
in parentheses. If the passages do not answer, say so.

Ask How many vacation days do I have, and can I carry them over? The Playground shows the call while it runs — search-handbook@prod, its arguments, the passages it returned, how long it took — and then the answer, citing (policies/leave-policy.pdf, page 2) and (policies/leave-policy.pdf, page 3).

Then ask Save a short summary of this answer to the handbook. The model calls save_to_knowledge and the chat shows a card: The model proposes to save a note to handbook — not saved until you say so, with the title (which you may change) and the text. Save writes it as a note of the base (notes/saved/<date>-<title>.md); Discard drops it. A proposal nobody decides on is dropped after a day.

5. Use it in the app and from your applications

• The app's Assistant: choose chat as the chat's model; in the chat's menu, Instructions holds the same text as the system prompt. Answers search the handbook, and save_to_knowledge shows the same Save / Discard card.
• Your application, through the hosted gateway: an ordinary chat completion with an Eos API key. The gateway runs search_handbook itself; the answer lists what ran in astraeus.tools. A save_to_knowledge call's result is {"status": "awaiting_confirmation", "knowledge_base": "handbook", "draft": "…"}: your application shows the note and saves it with POST /v1/metis/knowledge-bases/handbook/drafts/<draft>/commit (Let a model save when asked).
• A gateway on your own machines runs search_handbook too, but does not propose notes: it answers save_to_knowledge as unavailable, and the model says so.

Instead of a tool

You want	Use
To ask about one document now, with no base	In the app's chat, Attach → A PDF or a text file: its text goes with your question (a long file's beginning). Nothing is indexed
Claude, ChatGPT or Cursor to answer from your bases	Their MCP connection to Astralyx: search_knowledge_base (Connect your AI assistant), or Claude Code's skill (Search your knowledge from Claude Code)
Your own code to retrieve, then ask a model	The search API, then a chat completion (below)

Retrieval in your own code — the search API with your API token, then the deployment through a gateway with an Eos API key (EOS_URL and ASTRAEUS_API_KEY as in the Eos recipes):

ask_handbook.py

import os
import sys

import requests
from openai import OpenAI

question = sys.argv[1]
r = requests.post(f"{os.environ['ASTRALYX_API']}/metis/knowledge-bases/handbook/search",
                  headers={"Authorization": f"Bearer {os.environ['ASTRALYX_TOKEN']}"},
                  json={"query": question, "top_k": 5}, timeout=60)
r.raise_for_status()
context = "\n\n".join(
    f"[{h['citation']['document']}{', page ' + str(h['citation']['page']) if h['citation'].get('page') else ''}] {h['text']}"
    for h in r.json()["results"])

chat = OpenAI(base_url=os.environ["EOS_URL"], api_key=os.environ["ASTRAEUS_API_KEY"])
reply = chat.chat.completions.create(model="chat", temperature=0.1, messages=[
    {"role": "system", "content": "Answer only from the context. Cite the bracketed source of each claim. If the context does not say, say so."},
    {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"},
])
print(reply.choices[0].message.content)

$ pip install openai requests
$ python ask_handbook.py "Can I carry vacation days over?"

This is the Embeddings for retrieval recipe with Metis doing the reading, chunking, indexing, hybrid search and reranking.

Footprint and cost

• The function: one instance while it is called, on the base's machine; idle instances stop after 300 s (scaling.idle_seconds). Its first call starts an instance.
• Each question costs the chat model's tokens, plus the passages the tool returns as prompt tokens (five passages of up to 1,200 characters each): lower max_results to send less.
• Searching costs no tokens: the base and its models run as the workspace's work, metered by cores and memory.

Clean up

Remove the tools by the names the model calls them, then delete the function search-handbook:

$ astra eos tools chat remove search_handbook
$ astra eos tools chat remove save_to_knowledge

Troubleshooting

Symptom	Cause	Fix
The model answers without calling search_handbook	A small model, or a question that did not look like one the handbook answers	Use Qwen3 8B or larger; keep the system prompt; say what the base holds in the docstring
The tool returns could not reach the handbook at http://metis-handbook:7700	The function runs on a machine that does not reach the base's	Publish it with Run next to metis-handbook
The tool returns the handbook answered 503	The base is not serving (Pending, Starting, Failed)	See its state in Metis → Knowledge bases (Troubleshoot)
spec.tools[…].knowledge_base "…": a knowledge base of the deployment's own workspace	save_to_knowledge names another workspace's base	Name a base of this workspace
The card says the proposal is gone	It was saved or discarded already, or a day passed	Ask the model again

Related

• Give a model tools
• Let a model save when asked
• Write and publish a function
• How search works
• Embeddings for retrieval (RAG)