Best Practices

Vector databases are deceptively simple — store vectors, get nearest neighbors. The hard parts are choosing the right embedding model, sizing indexes, managing cost, and keeping the system fresh as you re-embed.

Choosing an Embedding Model

The single most-impactful choice. Criteria:

Models in 2026

Run a small eval: pick 100 representative queries with known relevant docs, compare top-K recall across 2-3 models. Quality differences are real and surprising.

Matryoshka Embeddings

Newer models (OpenAI v3, Nomic) produce Matryoshka embeddings — you can truncate to fewer dimensions with minimal quality loss:

// OpenAI: get a 256-dim version of a 1536-dim embedding
const emb = await openai.embeddings.create({
  model: 'text-embedding-3-small',
  input: text,
  dimensions: 256,    // truncate
});

256-dim is 6× cheaper to store and faster to query than 1536-dim with minor quality loss. Worth testing if cost matters.

Dimensionality Trade-offs

Storage matters at scale. 100M vectors × 1536 dims × 4 bytes ≈ 600 GB before indexes. Plan accordingly.

Index Types

HNSW is the default for most modern vector DBs — fastest queries, good recall. Build time and memory are the cost.

For >100M vectors, you'll need disk-based options (DiskANN, IVF+PQ on disk) — pure-in-memory HNSW gets expensive.

Tuning knobs:

• M (HNSW) — graph connectivity; higher = better recall, more memory. Default 16-32.
• ef_construction — quality of build; higher = slower build, better recall. Default 100-200.
• ef_search — quality of search; higher = slower query, better recall. Tune per query.

Most teams accept the defaults; revisit when you have measurable quality / latency / cost issues.

Filter Performance

Pre-filtering (apply filter, then ANN) vs post-filtering (ANN, then filter):

For very selective filters (per-user data), some DBs (Qdrant, pgvector with partial indexes) handle this well; others struggle. Test with realistic filter selectivity.

Multi-Tenancy

Same patterns as Search:

For SaaS with thousands of tenants: shared collection, tenant_id filter, index that field. For very large tenants (enterprise customers with millions of vectors each): one collection per tenant.

Re-Embedding

You'll need to re-embed when:

• Switching embedding models (vendor deprecation, quality upgrade).
• Adding new fields to the embedded text.
• The embedding model improves and you want the gains.

Plan for this from day one: store the embedding's model name + version with each vector. When re-embedding, do it in batches alongside the live index (Blue-green pattern):

1. Create new collection `products_v2`
2. Backfill: for each row, generate new embedding, upsert to v2
3. Validate quality on the new collection
4. Cut over reads (config flag)
5. Drop old collection

For embedding generation at scale, use the provider's batch API (50% cheaper at OpenAI, similar with others) — it's not real-time but huge for backfills.

Cost Management

Two cost axes:

Real example: 10M docs × 4 chunks each = 40M embeddings. At 1536 dim × 4 bytes = 240 GB. Plus HNSW index overhead (~1.5×) = ~400 GB working set.

• Pinecone Serverless: ~$500-1500/month at this size.
• Qdrant self-host on a 256 GB RAM machine: ~$200-400/month compute + ops time.
• pgvector on managed Postgres with pgvector plus a beefy instance: similar.

Costs scale roughly linearly with vector count × dimensions. Pick the model dim before you embed everything.

Observability

For RAG specifically, build an evaluation harness — a set of queries with expected answers. Run weekly. Score with RAGAS, LangSmith, or a custom rubric. Regressions are silent without it.

Stale Embeddings

If you embed user-generated content, you have a lifecycle issue: edits, deletions, re-classifications.

• On edit: re-embed and upsert.
• On delete: delete from the index (and your DB).
• On schema change (new fields in embedded text): re-embed.

The flow:

App write to DB ──► CDC / outbox ──► embedding worker ──► upsert to vector DB

Same pattern as keeping any derived index in sync with a source-of-truth DB. See Background Jobs for the queue layer.

Security

• API keys for embedding models in secret manager — see Secrets.
• Per-tenant filters at the query layer — never trust client-supplied tenant_id.
• Embeddings can leak content — though hard to reverse, they encode source information. Treat them as sensitive data.
• Don't embed PII you don't need in the search context. Embed identifiers; resolve to PII per-query in your app.

Common Pitfalls

Checklist