FAISS Local Index Q&A

A complete example demonstrating how to use your own FAISS index with CAIP's KnowledgeBaseSearchTool for semantic search and Q&A — without uploading data to a remote vector store.

Check Full Code Here

Overview

This example demonstrates:

✅ Loading a local FAISS index with companion document data
✅ Using KnowledgeBaseSearchTool in FAISS mode
✅ Query embedding generation via CAIP API
✅ Interactive Q&A loop with an agent

What You'll Build

A knowledge-base Q&A agent that can:

Search through your own pre-built FAISS index
Use CAIP-generated embeddings for query matching
Return semantically relevant results from local data
Work with any FAISS index type (flat, IVF, HNSW, etc.)

How It Works

┌─────────────────────────────────────────────────────────────────┐
│                        FAISS Search Flow                        │
│                                                                 │
│  1. User asks a question                                        │
│  2. CAIP API generates a query embedding (matching your model)  │
│  3. KnowledgeBaseSearchTool searches your local FAISS index     │
│  4. Top-k results returned to the agent                         │
│  5. Agent formulates an answer from retrieved context           │
└─────────────────────────────────────────────────────────────────┘

Prerequisites

Python 3.12 or Python 3.13
CAIP Agent ID (created via CAIP Portal)
CAIP API Key
A pre-built FAISS index file (index.faiss)
A companion pickle file (index.pkl) containing (docstore, index_to_docstore_id) from LangChain's FAISS wrapper

Quick Start

1. Install Dependencies

pip install caip-agents-sdk

2. Environment Setup

Create a .env file:

CAIP_API_KEY is a unified key for both Agents API and LLM API access. See CAIP API Key Authentication to obtain your key.

CAIP_API_KEY=your_api_key_here
CAIP_BASE_URL=your_caip_base_url_here
CAIP_AGENT_ID=your_agent_id_here
CAIP_REGION=your_region
CAIP_EMBEDDING_MODEL=text-embedding-3-small
CAIP_EMBEDDING_DIMENSION=384

3. Prepare Your FAISS Index

Your index directory should contain:

your-index-dir/
├── index.faiss    # FAISS index file (any index type)
└── index.pkl      # LangChain pickle: (docstore, index_to_docstore_id)

index.pkl format:

The pickle must contain a tuple of (docstore, index_to_docstore_id) — the standard output of LangChain's FAISS.save_local(). The docstore holds document objects with .page_content and .metadata attributes, and index_to_docstore_id maps each FAISS vector position to a document ID.

4. Complete Example

"""FAISS Local Index Q&A - caip-agents-sdk Example

Use your own FAISS index with CAIP's KnowledgeBaseSearchTool for Q&A.

How it works:
    - Your FAISS index holds the pre-built embeddings and document data
    - CAIP generates query embeddings using the specified embedding model
    - SDK's KnowledgeBaseSearchTool (in FAISS mode) searches your local index
    - All other operations (agents, threads) go through CAIP

Setup:
    pip install caip-agents-sdk 

Required env vars:
    CAIP_API_KEY    - Your CAIP API key
    CAIP_BASE_URL   - CAIP platform base URL
    CAIP_REGION     - Optional region selector (ROW default, CN supported)

Prerequisites:
    - A FAISS index file (index.faiss) built with your documents
    - A companion pickle file (index.pkl) in one of two formats:
      * LangChain format: (docstore, index_to_docstore_id) tuple
      * Generic format: (texts_list, metadata_list) tuple of plain lists
    - The embedding model name and dimension matching the one used to build
      the FAISS index (SDK uses CAIP API to generate query embeddings)

Usage:
    python faiss_search.py
"""

import asyncio
import os
import pickle
from pathlib import Path

# Workaround for Intel MKL library conflicts that can occur when both NumPy and
# FAISS link against different copies of MKL (common on macOS with conda/brew).
# Safe to remove if you don't encounter "OMP: Error #15" or similar MKL errors.
os.environ.setdefault("KMP_DUPLICATE_LIB_OK", "TRUE")

import faiss
from dotenv import load_dotenv

load_dotenv(Path(__file__).resolve().parents[2] / ".env")

from caip_agents_sdk import CAIPAgentsClient
from caip_agents_sdk.tools import KnowledgeBaseSearchTool

INDEX_DIR = Path(__file__).parent

AGENT_ID = os.getenv("CAIP_AGENT_ID", "YOUR_AGENT_ID")
EMBEDDING_MODEL = os.getenv("CAIP_EMBEDDING_MODEL", "text-embedding-3-small")
EMBEDDING_DIMENSION = int(os.getenv("CAIP_EMBEDDING_DIMENSION", "384"))
FRAMEWORK = "pydantic_ai"  # "langchain" or "pydantic_ai"


def load_index():
    """Load FAISS index and associated document texts/metadata."""
    index = faiss.read_index(str(INDEX_DIR / "index.faiss"))

    with open(INDEX_DIR / "index.pkl", "rb") as f:
        docstore, index_to_docstore_id = pickle.load(f)

    texts, metadata = [], []
    for i in range(index.ntotal):
        doc_id = index_to_docstore_id[i]
        doc = docstore.search(doc_id)
        texts.append(doc.page_content)
        metadata.append(doc.metadata)

    return index, texts, metadata


async def main():
    index, texts, metadata = load_index()
    print(f"Loaded FAISS index: {index.ntotal} vectors, {len(texts)} documents\n")

    client = CAIPAgentsClient()

    # Create agent with chosen framework
    agent = client.create_agent(FRAMEWORK, AGENT_ID)
    await agent.initialize()

    # Create a conversation thread
    thread = await client.create_thread(
        agent_id=AGENT_ID,
        thread_data={"title": "FAISS Search Q&A", "status": "open"},
    )
    agent.thread_id = thread.threadId

    # Set up the knowledge base search tool in FAISS mode.
    # The tool auto-detects FAISS mode when faiss_index is provided.
    # embedding_model + embedding_dimension are used to generate query embeddings
    # via the CAIP API (must match the model used to build the index).
    kb = KnowledgeBaseSearchTool(
        client=client,
        faiss_index=index,
        faiss_texts=texts,
        faiss_metadata=metadata,
        embedding_model=EMBEDDING_MODEL,
        embedding_dimension=EMBEDDING_DIMENSION,
    )

    # Register the search tool on the agent
    @agent.tool_plain
    async def search_knowledge_base(query: str, top_k: int = 5) -> str:
        """Search the FAISS knowledge base for relevant information."""
        return await kb.search(query=query, search_type="faiss", top_k=top_k)

    print("FAISS Q&A ready. Type 'quit' to exit.\n")
    while True:
        question = input("Question: ").strip()
        if question.lower() in ("quit", "exit", "q"):
            break
        if not question:
            continue
        response = await agent.run(question)
        print(f"\n{response}\n")


if __name__ == "__main__":
    asyncio.run(main())

Key Concepts Explained

1. FAISS Mode Auto-Detection

The KnowledgeBaseSearchTool automatically switches to FAISS mode when you pass faiss_index:

# Remote vector store mode (default)
kb_tool = KnowledgeBaseSearchTool(
    client=client,
    vector_store_id="vs_abc123"
)

# FAISS local mode (auto-detected)
kb_tool = KnowledgeBaseSearchTool(
    client=client,
    faiss_index=index,
    faiss_texts=texts,
    faiss_metadata=metadata,
    embedding_model="text-embedding-3-small",
    embedding_dimension=384,
)

2. Loading the Pickle

The load_index() function unpacks LangChain's pickle format and converts it to plain lists:

def load_index():
    index = faiss.read_index(str(INDEX_DIR / "index.faiss"))

    with open(INDEX_DIR / "index.pkl", "rb") as f:
        docstore, index_to_docstore_id = pickle.load(f)

    texts, metadata = [], []
    for i in range(index.ntotal):
        doc_id = index_to_docstore_id[i]
        doc = docstore.search(doc_id)
        texts.append(doc.page_content)
        metadata.append(doc.metadata)

    return index, texts, metadata

The search tool needs a simple positional mapping — vector at position i corresponds to texts[i] and metadata[i]. This function creates that alignment from LangChain's docstore.

3. Embedding Model Matching

The embedding model and dimension you specify must match what was used to build the FAISS index:

# If your index was built with text-embedding-3-small at dimension 384:
EMBEDDING_MODEL = "text-embedding-3-small"
EMBEDDING_DIMENSION = 384

# The SDK uses these to generate query embeddings via CAIP API
kb = KnowledgeBaseSearchTool(
    client=client,
    faiss_index=index,
    faiss_texts=texts,
    faiss_metadata=metadata,
    embedding_model=EMBEDDING_MODEL,
    embedding_dimension=EMBEDDING_DIMENSION,
)

4. Search Type

When using FAISS mode, use search_type="faiss":

result = await kb.search(
    query="your question",
    search_type="faiss",  # Required for FAISS mode
    top_k=5               # Number of results to return
)

Building Your FAISS Index

Use LangChain's FAISS wrapper to create the required files:

from langchain_community.vectorstores import FAISS
from langchain_community.embeddings import OpenAIEmbeddings

# Your documents and metadata
texts = [
    "Document 1 content...",
    "Document 2 content...",
    "Document 3 content...",
]

metadata = [
    {"source": "file1.pdf", "page": 1},
    {"source": "file2.pdf", "page": 1},
    {"source": "file3.pdf", "page": 2},
]

# Build the vector store
embedding_model = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = FAISS.from_texts(texts, embedding_model, metadatas=metadata)

# Save locally — creates index.faiss and index.pkl
vectorstore.save_local("my_index_dir")

This produces the two files the example expects:

index.faiss — the FAISS index vectors
index.pkl — the (docstore, index_to_docstore_id) tuple

Troubleshooting

Issue	Solution
`OMP: Error #15` or MKL errors	Set `os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"` before importing FAISS
`KeyError` in `index_to_docstore_id`	Your pickle is out of sync with the FAISS index — rebuild both together
Mismatched embedding dimensions	Ensure `embedding_dimension` matches the dimension used to build the index
Poor search results	Verify the `embedding_model` matches the one used during index creation

Next Steps

Tools — Learn about all tool types and registration patterns
RAG & Vector Stores — Use remote vector stores for managed RAG
Search & Filtering — Explore all search strategies
RAG Agent Example — See the remote vector store RAG example

Overview​

What You'll Build​

How It Works​

Prerequisites​

Quick Start​

1. Install Dependencies​

2. Environment Setup​

3. Prepare Your FAISS Index​

4. Complete Example​

Key Concepts Explained​

1. FAISS Mode Auto-Detection​

2. Loading the Pickle​

3. Embedding Model Matching​

4. Search Type​

Building Your FAISS Index​

Troubleshooting​

Next Steps​