Skip to main content

Recipe: Search the Vector Store

The Problem​

"Once our documents are embedded, we need our agent to actually query the vector store when answering a question — falling back to keyword matching for exact terms (error codes, product names) and semantic search for conceptual questions, ideally through one tool the agent can call."

Ingredients​

  • A populated vector store (see Create & Ingest Embeddings)
  • KnowledgeBaseSearchTool (recommended for agents), or the direct client search methods (vector_search, hybrid_search, keyword_search) for scripts

The Recipe​

from caip_agents_sdk.tools import KnowledgeBaseSearchTool

kb_tool = KnowledgeBaseSearchTool(
    client=client,
    vector_store_id=vector_store_id,
)

@agent.tool_plain
async def search_docs(query: str) -> str:
    """Search product documentation."""
    return await kb_tool.search(query=query, search_type="hybrid", top_k=5)

response = await agent.run("How do I authenticate API requests?")

The tool automatically generates query embeddings, formats results for LLM consumption, and handles errors/retries — no manual embedding step needed.

Production recommendation: keep tool description explicit and scoped so the model learns when to call retrieval versus when to answer directly.

2. Choose a search strategy​

Your Needsearch_type
General questions (default)"hybrid" ⭐ — semantic + keyword
Conceptual questions ("What are authentication methods?")"vector" — semantic similarity
Exact terms, error codes, identifiers"keyword" — exact/fuzzy text match

3. Filter for precision​

@agent.tool_plain
async def search_v2_auth(query: str) -> str:
    """Search only v2 authentication docs."""
    return await kb_tool.search(
        query=query,
        search_type="hybrid",
        document_id="api/authentication/",       # prefix match on document ID
        metadata_filter={"version": "v2"},       # match on stored metadata
        top_k=5,
    )

4. Alternative: direct client methods (scripts, pipelines, no agent)​

query_embedding = await client.text_to_embedding(
    text="What are authentication methods?",
    vector_store_id=vector_store_id,
)

search_results = await client.vector_search(
    vector_store_id=vector_store_id,
    search_request={
        "queryEmbedding": query_embedding,
        "limit": 10,
        "scoreThreshold": 0.0,
        "includeEmbeddings": False,
    },
)
results = search_results.results if hasattr(search_results, "results") else []

Use this when you need direct API access without an agent — e.g. a data pipeline. hybrid_search() and keyword_search() follow the same shape (see the Search & Filtering reference).

5. Troubleshooting: no results found​

# 1. Check the vector store actually has embeddings
embeddings = await client.list_embeddings(vector_store_id)
print(f"Total embeddings: {len(embeddings)}")

# 2. Lower the score threshold
await kb_tool.search(query=query, score_threshold=0.5)

# 3. Remove filters temporarily
await kb_tool.search(query=query, metadata_filter=None, document_id=None)

# 4. Try hybrid search if you were using vector/keyword only
await kb_tool.search(query=query, search_type="hybrid")

Enterprise Response Pattern​

To reduce hallucinations, require answer grounding:

  • If retrieval returns no relevant context, instruct the agent to say it cannot verify from approved sources.
  • Include source snippets or document IDs in final responses for auditability.
  • Keep business-critical actions behind deterministic tools, not free-form generation.

Enterprise Hardening Checklist​

  • Use hybrid search as default; tune with evaluation data, not intuition.
  • Add metadata filters for tenant, product, region, and version isolation.
  • Track retrieval hit rate, empty-result rate, and citation coverage.
  • Add fallback behavior for retrieval failure (degraded mode + user-safe response).
  • Run regression checks when corpus, embedding model, or chunking strategy changes.