# Pre-baked seed corpus embeddings

This directory holds **pre-computed vector embeddings** for the seed corpus in `data/seed/`. Files here let every installation use the baseline RAG corpus without spending API tokens on first activation — they're read straight off disk and written into `{prefix}_amb_rag_chunks`.

## Naming

```
<provider>-<model>-<dim>.json    metadata + base64-encoded float32 vectors inline per chunk
```

Each `chunks[].vec` is a base64-encoded little-endian float32 blob (`dim * 4` bytes when decoded). Vectors are inlined into the JSON instead of being kept in a sibling `.bin` because plugin-check rejects bare binary payloads under `application_detected`.

Currently expected files (one per supported embedding provider):

```
openai-text-embedding-3-small-1536.json   (~250 KB for the starter corpus)
gemini-embedding-001-3072.json            (~1.4 MB for the starter corpus)
```

The slug format is defined in [`AMB_Rag_Indexer::shipped_vectors_slug()`](../../../includes/rag/class-indexer.php) and used by both the build script and the runtime loader.

## Generating these files

You need an API key for each provider you want to support. The build script does NOT load WordPress — it can run from any PHP 7.4+ shell.

```bash
# From the plugin root:

OPENAI_API_KEY=sk-... php bin/build-seed-embeddings.php --provider=openai
GEMINI_API_KEY=AIza... php bin/build-seed-embeddings.php --provider=gemini

# Or both at once:
OPENAI_API_KEY=sk-... GEMINI_API_KEY=AIza... php bin/build-seed-embeddings.php --provider=all
```

Useful flags:

| Flag        | Effect                                                                                                                 |
|-------------|------------------------------------------------------------------------------------------------------------------------|
| `--dry-run` | Walk the corpus, print what *would* be embedded, but make no API calls. Useful for chunk-count audits.                 |
| `--force`   | Re-embed even when the existing `.json` reports an unchanged body hash. Use after switching embedding model versions.  |
| `--help`    | Usage hint.                                                                                                            |

The build script is incremental by default: it reuses vectors for any source whose body hash hasn't changed since the previous build. Editing one seed file and rebuilding only re-embeds that file.

## After generating

Commit the generated `<slug>.json` file. It is part of the shipped plugin.

Sanity-check the build:

```bash
php -r 'print_r(json_decode(file_get_contents("data/seed/embeddings/openai-text-embedding-3-small-1536.json"), true)["chunk_count"]);'
# Should print the total chunk count.

ls -lh data/seed/embeddings/
```

After installing the plugin on a fresh WordPress site with the matching provider configured, the very first chat turn pulls these vectors directly — no API calls — and `GET /wp-json/amb/v1/preferences/rag-status` returns a non-zero `countsByType` immediately.

## What happens when these files are missing

The runtime path degrades gracefully:

1. `AMB_Rag_Indexer::seed_corpus()` first tries `load_shipped_vectors_for_active_provider()`. If the JSON file doesn't exist for the user's active provider, it returns `null` — no error.
2. The corpus then falls through to the **API-embed path** that was already in place before pre-baked vectors existed. The user pays for embedding their own corpus, identical to the original implementation.
3. A future plugin upgrade that ships the missing files starts using them automatically — `replace_source()` overwrites the API-embedded rows with the shipped ones on the next reseed.

So shipping these files is a strict optimization, not a hard requirement.

## When to rebuild

- **Seed corpus content changed** — any `.md` file under `data/seed/` was edited, added, or removed.
- **Embedding model version bumped** — e.g. OpenAI deprecates `text-embedding-3-small`. Update the `PROVIDERS` map in `bin/build-seed-embeddings.php` AND `AMB_Rag_Indexer::SHIPPED_PROVIDERS`, then rebuild.
- **You added a new provider to support** — extend the `PROVIDERS` map and the `SHIPPED_PROVIDERS` constant; rebuild only for the new provider.

See [`doc/RAG-SEED-CORPUS.md`](../../../doc/RAG-SEED-CORPUS.md) for the full author workflow.
