Skip to content

Add files to a knowledge base#

Besides the drives it reads, every knowledge base has notes of its own: files you upload, notes saved from conversations and a memory's facts. You add a file once and it is searchable a few seconds later, with a citation like any other document. Use it for documents that live on your computer or phone, not on a drive.

Before you begin#

  • A knowledge base (Build a knowledge base). A base may also have no source drive at all and be filled only with what you add: create it with no sources.
  • Permission to add to it: knowledge-bases:write-content (the workspace roles editor and admin; the Developer and Analyst presets).
  • For the CLI, astra signed in with astra use <org>/<workspace>; for the API, a token in ASTRALYX_TOKEN and ASTRALYX_API=https://api.astralyx.cloud/v1.

Where added files live#

A file you add is written on the base's machine, in the base's own folder notes/, on the drive that keeps its index. It never passes through anything that stores it on the way, and it never leaves your machines.

Source drives are never written, whatever their kind:

  • a drive backed by a data source holds a revision of what the source has — a file written into it would belong to no revision, and the next listing would contradict it;
  • a plain drive may be other work's: the base reads it, read-only, so nothing the base does can change it.

One rule for every base: what you add lives with the base. Searches cite it with the base's drive as the source (metis-<base>) and notes/<path> as the document; the base's page shows where (notes in the API).

Warning

Deleting a knowledge base deletes what was added to it: uploads, saved notes and a memory's facts. Download what you want to keep first. The documents of its source drives stay.

Add files#

  1. Open Metis ▸ Knowledge bases and choose the base.
  2. On Documents, under Added to this base, drop files or folders on the page, or choose Choose files (with, optionally, a folder to put them in). On a phone, tap Choose files and pick them.
  3. Each file shows its progress, then pending (not read yet), indexed with its chunks, or failed with why.
  4. A file of the same path is already there: choose Replace it or give the new one another name.

Files of the base's source drives are listed too, read-only: change them on their drive.

$ astra metis add handbook ./travel-policy.pdf ./onboarding/
travel-policy.pdf: 100% of 1.2 MB
./travel-policy.pdf: added as notes/travel-policy.pdf
onboarding/first-week.md: 100% of 4 KB
./onboarding/first-week.md: added as notes/onboarding/first-week.md
indexing: 2 of 2 read
Flag Default Description
--to <folder> the notes' root Put them in this folder of the notes.
--replace off Replace a file of the same path.
--no-wait off Do not wait for them to be indexed.

A folder's readable files are added in their folders; hidden files and other kinds are skipped and named. astra metis files handbook lists what was added and how the index has each; astra metis rm handbook <path> removes one.

The body is the file itself; send Content-Length so a file that is too large is refused before it is sent. Each part of the path is percent-encoded.

$ curl -fsS -X PUT "$ASTRALYX_API/metis/knowledge-bases/handbook/files/policies/travel-policy.pdf" \
    -H "Authorization: Bearer $ASTRALYX_TOKEN" -H 'content-type: application/octet-stream' \
    --data-binary @travel-policy.pdf
{"path":"policies/travel-policy.pdf","document":"notes/policies/travel-policy.pdf","source":"metis-handbook","size":1210338,"replaced":false,"note":"indexed by the next pass, which starts now"}

?overwrite=true replaces a file of that path. GET …/files lists what was added with each file's state; GET and DELETE …/files/<path> read and remove one. See the Metis API.

Follow the indexing#

Adding a file starts a pass at once: only what changed is read. The file's state is pending until the pass reads it, then indexed (with its chunks) or failed (with why — a scanned PDF without text, a chunk longer than the embedding model's window). The base's Activity tab lists its passes: what each added, changed and removed.

Evaluation sets#

A JSON Lines file under .metis/ (.metis/onboarding.jsonl) is kept as an evaluation set: never indexed, listed on Evaluations, and asked by an evaluation as metis-<base>:notes/.metis/onboarding.jsonl. See Build a knowledge base.

Limits#

Limit Value
Kinds Markdown (.md, .markdown, .mdx), text (.txt, .rst, .csv, .tsv, .json, .jsonl, .log, …), HTML, PDF, Word (.docx) — what a base reads (Files read)
Size 64 MB per file
Path 400 characters; no . or .. parts; no hidden names but .metis/
Written Whole or not at all: an upload cut off leaves nothing behind

Troubleshooting#

Symptom Cause Fix
409 FILE_EXISTS A file of that path was added before Replace it (--replace, ?overwrite=true), or rename yours
400 UNSUPPORTED_TYPE Not a kind a base reads Export it as PDF, Word, Markdown or text
413 FILE_TOO_LARGE Larger than 64 MB Split it, or export a smaller version
503 KNOWLEDGE_BASE_NOT_READY The base's index is not serving yet Wait for Indexing or Ready; the message says why
The file stays failed It could not be read Its reason says why; fix the file and add it again with Replace