Skip to content

A camera at the door#

Frigate sees a person at the door and Home Assistant tells the workflow's address. A File box fetches the event's snapshot from Frigate — on your machine, on your network — a model that sees images looks at it, and when it is worth telling — someone at the door, a delivery — the photo arrives on your phone and in Telegram with a line of what happens. One notice per camera every two minutes, however many events Frigate sends.

This is the Camera at the door recipe of the console.

flowchart LR
  F([Frigate<br/>via Home Assistant]) --> T([camera<br/>HTTP call · one per camera per 2 min])
  T --> S[snap<br/>file · fetch]
  S --> L[look<br/>model · image · JSON]
  L -- worth telling --> N[tell<br/>notify · the photo]

Before you begin#

  • A machine of the workspace at home, on the network Frigate is on, with Iris files installed on it (Gallery → Iris files). Its function fetches the snapshot from there, and its drive keeps it.
  • A deployment of a model that sees images — Qwen2.5-VL 7B, Gemma 3 12B — that is up (or wakes: the box waits for it). Below, <model> stands for its name.
  • Frigate, and Home Assistant with Frigate's MQTT events (the Frigate integration sets this up).
  • To be told on your phone, the app's notifications; on Telegram, your Telegram connected to an assistant.

1. The workflow#

camera-at-the-door.yaml
apiVersion: anemoi.astralyx/v1
kind: Flow
metadata:
  name: camera-at-the-door
spec:
  description: A camera's event, looked at by a model; told with the photo when it matters.
  triggers:
    - {id: camera, kind: http, dedupe: "${{ event.camera }}", dedupe_seconds: 120}
  steps:
    - id: snap
      kind: function
      function: iris-files
      target: prod
      input:
        op: fetch
        url: "http://<frigate>:5000/api/events/${{ event.id }}/snapshot.jpg"
        folder: camera
        name: "${{ event.camera }}.jpg"
    - id: look
      kind: model
      needs: [snap]
      deployment: <model>
      format: json
      images: ["${{ steps.snap.outputs.file }}"]
      prompt: >-
        A snapshot from my camera ${{ event.camera }}, which saw: ${{ event.label }}.
        Answer JSON: who (person, delivery, car, animal or nobody), what (one
        short sentence of what happens), tell (true when it is worth telling me:
        someone at the door, a delivery, a stranger at night; false when nothing happens).
    - id: tell
      kind: notify
      needs: [look]
      when: "steps.look.outputs.tell == 'true'"
      title: "${{ event.camera }}: ${{ steps.look.outputs.who }}"
      message: "${{ steps.look.outputs.what }}"
      attach: "${{ steps.snap.outputs.file }}"
  outputs:
    who: "${{ steps.look.outputs.who }}"
    what: "${{ steps.look.outputs.what }}"
    photo: "${{ steps.snap.outputs.file }}"

Replace <frigate> with Frigate's address on your network (its port 5000 answers without a password from inside it) and <model>. In the console, New workflow → Start from a recipe → Camera at the door draws it; fill Who answers on look and the address on snap.

2. Its address#

Iris → Workflows → camera-at-the-door → Triggers → Make the address, and copy the URL (its last part is the secret, shown once). Try it with an event id Frigate's Explore page shows:

$ curl -sS -X POST '<address>' -H 'Content-Type: application/json' \
    -d '{"id": "1729339200.123456-abcd12", "camera": "porch", "label": "person"}'
{"execution":"camera-at-the-door-5be0a2","state":"Pending","deduplicated":false}

The execution's page shows snap fetching, look waiting for the model, then tell — and the photo arrives on your phone.

3. Home Assistant calls it#

In Home Assistant's configuration.yaml, a REST command to the address:

rest_command:
  iris_camera:
    url: !secret iris_camera_address
    method: POST
    content_type: application/json
    payload: '{"id": "{{ id }}", "camera": "{{ camera }}", "label": "{{ label }}"}'

with the address in secrets.yaml (iris_camera_address: <address>), and an automation on Frigate's events — each new object it starts tracking:

automation:
  - alias: Frigate to Iris
    trigger:
      - platform: mqtt
        topic: frigate/events
    condition:
      - "{{ trigger.payload_json.type == 'new' and trigger.payload_json.after.label in ['person', 'car'] }}"
    action:
      - service: rest_command.iris_camera
        data:
          id: "{{ trigger.payload_json.after.id }}"
          camera: "{{ trigger.payload_json.after.camera }}"
          label: "{{ trigger.payload_json.after.label }}"

Restart Home Assistant (or reload REST commands and automations). The next person Frigate sees starts the workflow.

Make it yours#

  • Another camera, no Frigate. A camera or NVR with a snapshot URL: use the File box's A camera's snapshot URL recipe (auth: true with the camera's user and password given to Iris files at install as a Credential), and call the address from whatever sees the motion.
  • Home Assistant's camera. The Home Assistant's camera recipe fetches /api/camera_proxy/<camera> with a long-lived access token given to Iris files at install.
  • Only at night. A condition before snap on the hour, or say so in look's prompt.
  • Keep a log. A Table box after look keeps who, what and the photo's file — and an assistant with the table as a tool answers "who came by today?".

Where the photo goes#

The snapshot is fetched by your machine and kept on its drive (30 days by default). To show it to the model, its bytes pass through Astralyx in memory, as a photo sent in a chat does, and are kept nowhere else. Sent with the notice, it goes to Telegram and is read again from your machine each time the inbox shows it.