# AI actions and pipelines

Transcribe, tag or convert a file with one call, cached by content; or run actions on every file that lands in a folder, with conditions, chains and ready-made templates.

An **action** does one thing to one version of a file. The server comes with three, and [plugins](/docs/plugins) add more.

| Action | Does | Writes | Needs on the server |
|---|---|---|---|
| `side.transcribe` | Speech to timed text, searchable | The `transcript` sidecar | ffmpeg, and `SIDESPACE_TRANSCRIBE_URL` pointing at any OpenAI-compatible `/audio/transcriptions` service (OpenAI, Groq, a local whisper.cpp or faster-whisper) with `SIDESPACE_TRANSCRIBE_KEY` |
| `side.tags` | A vision model looks at key frames: description, tags, shot type, setting, mood | The `ai.tags` sidecar | Claude by default (`SIDESPACE_ANTHROPIC_API_KEY`), or a local model: `SIDESPACE_TAGS_PROVIDER=openai`, `SIDESPACE_TAGS_URL`, `SIDESPACE_TAGS_MODEL` |
| `side.convert` | ffmpeg presets: `mp4`, `webm`, `prores-proxy`, `wav`, `mp3`, `jpg`, `png`, `webp`, `gif` | A new file next to the source, with a `derived` sidecar | ffmpeg |

`GET /v1/spaces/{sid}/actions` lists what this server offers. An action that isn't set up is listed with `available: false` and says which setting it needs, along with each action's `params_schema`.

## Run one

```sh tab=CLI
sidespace actions "Film A"
sidespace run "Film A" /Ep01/a.mov side.transcribe --param language=en
sidespace run "Film A" /Ep01/a.mov side.convert --param preset=mp4 --param max=720
sidespace runs "Film A" /Ep01/a.mov
```

```ts tab=TypeScript
// Waits for the result. Throws SideSpaceError (code run_failed) if it fails.
const run = await side.run("Film A", "/Ep01/a.mov", "side.transcribe", { language: "en" });
console.log(run.cached, run.result?.sidecars?.transcript);
```

```python tab=Python
run = side.run("Film A", "/Ep01/a.mov", "side.transcribe", {"language": "en"})
print(run["cached"], run["result"]["sidecars"]["transcript"]["text"])
```

```sh tab=curl
curl -X POST "https://side.example.com/v1/spaces/$SPACE/runs" \
  -H "Authorization: Bearer $SIDESPACE_TOKEN" -H "Content-Type: application/json" \
  -d '{"action": "side.tags", "path": "/Ep01/a.mov"}'
# then GET /v1/spaces/$SPACE/runs/$RUN until state is done or failed
```

A run moves through `queued`, `running`, then `done`, `failed` or `canceled`, with `progress` from 0 to 1 on the way. Editors can start runs; tokens need `runs:write`. `POST …/runs/{rid}/cancel` stops one. In the SDKs, `startRun`/`start_run` and `waitRun`/`wait_run` split starting from waiting.

### Cached by content

The same bytes with the same params are processed once. Uploading the file again, copying it or restoring an old version doesn't call the model again: the run comes back at once with `cached: true`. Results that only write sidecars are reused across spaces; results that make files or renditions are reused within the space. Pass `"fresh": true` (CLI `--fresh`) to run again anyway.

> [!TIP]
> Before running, read the file's sidecars. A transcript or tags may already be there from a pipeline.

### Failures and retries

Failures that may pass (a timeout, a 5xx from the model service) retry after 30 seconds and then 3 minutes, three tries in all. Failures that won't (a rejected key, a file with no audio) fail at once with the reason in `error`.

Changes a run makes are recorded for the person who started it, with `actor_kind: "plugin"` and `actor_via: <action>`, so history shows that AI did it.

## Pipelines

A pipeline is a standing rule for a space: every newly saved file that matches a path glob and a MIME prefix gets an action. Space managers make them; tokens need `pipelines:manage`.

```sh tab=CLI
sidespace pipeline create "Film A" side.tags --path "/dailies/**" --mime video/ --name "Tag dailies"
sidespace pipeline create "Film A" side.transcribe --mime audio/ --backfill    # files already there too
sidespace pipeline list "Film A"
sidespace pipeline enable "Film A" pip_… --off
```

```ts tab=TypeScript
await side.createPipeline("Film A", { name: "Tag dailies", action: "side.tags", path_glob: "/dailies/**", mime_prefix: "video/" });
```

```python tab=Python
side.create_pipeline("Film A", "Tag dailies", "side.tags", path_glob="/dailies/**", mime_prefix="video/")
```

Like webhooks, each pipeline follows the journal on its own, from the moment it's created; `--backfill` (or `POST …/pipelines/{pid}/backfill`) covers files already there. Versions made by runs never start pipelines, so a "convert to mp4" pipeline can't loop.

### Conditions

`conditions` narrow a pipeline to files that meet all of them: `min_size` and `max_size` (bytes), and from the media probe `min_duration_s`, `max_duration_s`, `min_width`, `min_height` and `has_audio`. A file that lacks the value fails the condition (an image has no duration), and unknown keys are refused, so a typo can't quietly match everything. A new version still being probed waits for the probe, for at most 15 minutes.

```sh tab=CLI
# Review copies of 4K masters only
sidespace pipeline create "Film A" side.convert --mime video/ --param preset=mp4 --param max=1920 --when min_width=2560
# Transcribe videos with sound that are longer than 10 seconds
sidespace pipeline create "Film A" side.transcribe --mime video/ --when has_audio=true --when min_duration_s=10
```

```sh tab=curl
curl -X POST "https://side.example.com/v1/spaces/$SPACE/pipelines" \
  -H "Authorization: Bearer $SIDESPACE_TOKEN" -H "Content-Type: application/json" \
  -d '{"name": "Transcribe interviews", "action": "side.transcribe", "mime_prefix": "video/", "conditions": {"has_audio": true, "min_duration_s": 10}}'
```

### Chains

`after: <action>` starts a pipeline when a run of that action finishes on a matching file, instead of when the file is saved. Use it for steps that need an earlier result, such as tagging a video only once it is transcribed. Runs are deduplicated by version, action and params, so a cycle stops after one round, and a chained pipeline's backfill takes only files the action it follows is done on.

```sh
sidespace pipeline create "Film A" side.tags --mime video/ --after side.transcribe
```

### Templates

Templates are ready-made sets of pipelines for common jobs. They live in the server's code, so an upgrade improves them.

| Template | Does |
|---|---|
| `video-insights` | Tags each new video with AI, and transcribes videos with sound |
| `review-copies` | A 1080p H.264 MP4 next to every new video wider than 2560 pixels |
| `interviews` | Transcribes new audio, and videos with sound |
| `tag-images` | Describes and tags new images |

```sh tab=CLI
sidespace pipeline templates "Film A"
sidespace pipeline template "Film A" video-insights --path "/Ep01/**" --backfill
```

```sh tab=curl
curl "https://side.example.com/v1/spaces/$SPACE/pipeline-templates" -H "Authorization: Bearer $SIDESPACE_TOKEN"
curl -X POST "https://side.example.com/v1/spaces/$SPACE/pipeline-templates/video-insights" \
  -H "Authorization: Bearer $SIDESPACE_TOKEN" -H "Content-Type: application/json" \
  -d '{"path_glob": "/Ep01/**", "backfill": true}'
# → {"items": [the pipelines it made], "runs": runs started on files already there}
```

Applying a template creates ordinary pipelines, which you then list, pause or delete like any other. A template whose action isn't set up on the server is still listed; its pipelines wait until the server is set up.

In the web app, space managers turn templates on from **Automations** in a folder's toolbar, for that folder or the whole space, and turn them off there.

## From an AI agent

Over [MCP](/docs/mcp), `list_actions` shows what can be done, `run_action` starts one and waits (60 seconds by default, up to 300), and `get_run` checks on it.