Skip to main content

Recipe: Add a Vector Store

The Problem​

"We want our agent to ground its answers in our own documentation instead of relying only on what the LLM was trained on. Before we can ingest any documents, we need a place to store their embeddings — one we can search later with semantic, keyword, or hybrid queries."

Ingredients​

  • client.create_vector_store()
  • An embedding model choice
  • A chunk size / chunk overlap choice

The Recipe​

1. Choose an embedding model​

ModelDimensionBest For
text-embedding-3-small ⭐1536General purpose — start here
text-embedding-3-large3072Higher accuracy needs
titan-text-embeddings-v21024AWS environments

2. Create the vector store​

from caip_agents_sdk import CAIPAgentsClient

client = CAIPAgentsClient()

vector_store = await client.create_vector_store({
    "name": "ProductDocs",
    "description": "Product documentation knowledge base",
    "embeddingModel": "text-embedding-3-small",
    "embeddingDimension": 1536,
    "chunkSize": 1000,      # sweet spot is 800-1200 characters
    "chunkOverlap": 200,    # ~15-20% overlap
})

vector_store_id = vector_store.vectorStoreId

Chunk size is a balance: too small (< 500) loses context, too large (> 2000) dilutes relevance — 800–1200 characters with 15–20% overlap is the recommended sweet spot.

3. Manage the vector store​

# Get
vector_store = await client.get_vector_store(vector_store_id)

# List
vector_stores = await client.list_vector_stores()

# Update
await client.update_vector_store(
    vector_store_id=vector_store_id,
    update_data={"name": "UpdatedName", "description": "Updated description"},
)

# Delete (embeddings must be removed first)
await client.delete_all_embeddings(vector_store_id)
await client.delete_vector_store(vector_store_id)
Bring your own index

If you already have a pre-built local FAISS index and don't want to upload data to a remote vector store, KnowledgeBaseSearchTool can run against it directly in FAISS mode instead — see the FAISS Search example.

Enterprise Hardening Checklist​

  • Keep embeddingModel and embeddingDimension consistent with your ingestion/search pipeline.
  • Define retention and deletion workflows before production ingestion.
  • Restrict who can mutate or delete vector stores in shared spaces.
  • Benchmark retrieval quality before and after chunking/model changes.
  • Document vector store ownership (team, service, on-call).