Best Practices & Troubleshooting

Tips for optimizing your RAG implementation and solving common issues.

---

Best Practices

1. Choose the Right Embedding Model

Start with the recommended model:

Model	Dimension	When to Use
text-embedding-3-small ⭐	1536	Start here - general purpose
text-embedding-3-large	3072	Complex docs, need higher accuracy
titan-text-embeddings-v2	1024	AWS environments

# Recommended for most cases
vector_store = await client.create_vector_store({
    "name": "MyDocs",
    "embeddingModel": "text-embedding-3-small",
    "embeddingDimension": 1536
})

2. Optimize Chunk Size

The Goldilocks principle:

Size	Issue	Result
Too small (< 500)	Lost context	Poor results
Too large (> 2000)	Diluted relevance	Poor results
Just right (800-1200) ⭐	Good balance	Best results

# Recommended settings
vector_store = await client.create_vector_store({
    "name": "MyDocs",
    "chunkSize": 1000,      # ✅ Sweet spot
    "chunkOverlap": 200     # ✅ 20% overlap
})

3. Use Meaningful Metadata

# ✅ Good - Rich metadata
metadata = {
    "category": "api",
    "subcategory": "authentication",
    "version": "v2",
    "difficulty": "beginner",
    "last_updated": "2024-03-01"
}

# ❌ Bad - Minimal metadata
metadata = {"type": "doc"}

4. Batch Everything

Batch operations are 10-100x faster:

# ✅ Good - Batch processing
result = await client.build_and_push_embeddings(
    document_path="./docs/",  # Processes all files at once
    vector_store_id="vs_abc123"
)

# ❌ Bad - Individual processing
for file in files:
    # Process one at a time - very slow!
    embeddings = await client.generate_embeddings(...)
    await client.create_embeddings(...)

5. Use Hybrid Search by Default

# ✅ Recommended - Best for most queries
await kb_tool.search(query=query, search_type="hybrid")

# Only use specific types when needed
await kb_tool.search(query=query, search_type="vector")   # Semantic only
await kb_tool.search(query=query, search_type="keyword")  # Exact match only

6. Handle Errors Gracefully

@agent.tool_plain
async def safe_search(query: str) -> str:
    """Search with error handling."""
    try:
        return await kb_tool.search(query=query, search_type="hybrid")
    except Exception as e:
        return f"Search failed: {str(e)}. Please rephrase your query."

---

Performance Optimization

1. Use top_k Wisely

# ✅ Usually sufficient
await kb_tool.search(query=query, top_k=5)

# More results = slower (but more comprehensive)
await kb_tool.search(query=query, top_k=20)

2. Cache Vector Store Config

The KnowledgeBaseSearchTool automatically caches configuration - no action needed!

3. Filter When Possible

# ✅ Faster - Searches fewer documents
await kb_tool.search(
    query=query,
    metadata_filter={"category": "api"}
)

# Slower - Searches everything
await kb_tool.search(query=query)

---

Troubleshooting

Problem: No Results Found

Possible causes:

• Vector store is empty
• Score threshold too high
• Filters too restrictive
• Query doesn't match content

Solutions:

# 1. Check if vector store has embeddings
embeddings = await client.list_embeddings("vs_abc123")
print(f"Total embeddings: {len(embeddings)}")

# 2. Lower score threshold
await kb_tool.search(query=query, score_threshold=0.5)

# 3. Remove filters temporarily
await kb_tool.search(
    query=query,
    metadata_filter=None,  # Remove filter
    document_id=None       # Remove filter
)

# 4. Try hybrid search
await kb_tool.search(query=query, search_type="hybrid")

Problem: Embedding Generation Fails

Check these:

1. ✅ Vector store exists and is properly configured
2. ✅ Embedding model is available
3. ✅ Text is not empty
4. ✅ API credentials are valid

# Verify vector store
try:
    vs = await client.get_vector_store("vs_abc123")
    print(f"Vector store: {vs['name']}")
except Exception as e:
    print(f"Error: {e}")

Problem: Slow Performance

Common causes:

1. ❌ Processing files individually (use batch methods!)
2. ❌ Too many results (top_k too high)
3. ❌ No filters (searching everything)

Solutions:

# ✅ Use batch processing
await client.build_and_push_embeddings(
    document_path="./docs/",  # Processes all at once
    vector_store_id="vs_abc123"
)

# ✅ Limit results
await kb_tool.search(query=query, top_k=5)

# ✅ Add filters
await kb_tool.search(
    query=query,
    metadata_filter={"category": "api"}
)

Problem: Backend 500 Errors

Common issues:

1. Wrong data types in metadata filters
2. Missing required fields
3. Vector store not found

# ❌ Bad - Wrong type
metadata_filter = {"page": "1"}  # String instead of int

# ✅ Good - Correct type
metadata_filter = {"page": 1}  # Integer

# Enable debug logging
import logging
logging.basicConfig(level=logging.DEBUG)

Problem: Poor Search Results

Improve accuracy:

1. Use hybrid search (combines semantic + keyword)
2. Add better metadata (category, version, topic)
3. Organize document IDs (use hierarchical structure)
4. Adjust chunk size (try 1000 characters)

# ✅ Better configuration
vector_store = await client.create_vector_store({
    "name": "MyDocs",
    "embeddingModel": "text-embedding-3-small",
    "embeddingDimension": 1536,
    "chunkSize": 1000,
    "chunkOverlap": 200
})

# ✅ Use rich metadata
metadata = {
    "category": "api",
    "topic": "authentication",
    "version": "v2"
}

# ✅ Use hybrid search
await kb_tool.search(query=query, search_type="hybrid")

---

Quick Tips

✅ Do This

• Start with text-embedding-3-small (1536 dimensions)
• Use build_and_push_embeddings() for file ingestion
• Set chunkSize=1000, chunkOverlap=200
• Use hybrid search by default
• Add rich, structured metadata
• Batch process whenever possible
• Use meaningful document IDs

❌ Avoid This

• Processing files one at a time
• Generic document IDs (doc_1, doc_2)
• Minimal metadata ({"type": "doc"})
• Very large chunks (> 2000 characters)
• Processing without error handling

---

Common Patterns

Pattern 1: Multi-Source Knowledge Base

# Ingest from multiple sources
sources = [
    ("./api-docs/", {"source": "api", "version": "v2"}),
    ("./guides/", {"source": "guides", "type": "tutorial"}),
    ("./faq/", {"source": "support", "type": "faq"})
]

for path, metadata in sources:
    await client.build_and_push_embeddings(
        document_path=path,
        vector_store_id=vs_id,
        metadata=metadata
    )

# Search specific sources
@agent.tool_plain
async def search_api(query: str) -> str:
    return await kb_tool.search(
        query=query,
        metadata_filter={"source": "api"}
    )

Pattern 2: Version-Specific Search

# Add version metadata
await client.build_and_push_embeddings(
    document_path="./docs/v2/",
    vector_store_id=vs_id,
    metadata={"version": "v2"}
)

# Search specific version
@agent.tool_plain
async def search_v2(query: str) -> str:
    return await kb_tool.search(
        query=query,
        metadata_filter={"version": "v2"}
    )

Pattern 3: Progressive Enhancement

# Start simple
result = await client.build_and_push_embeddings(
    document_path="./docs/",
    vector_store_id=vs_id
)

# Add more documents later
result = await client.build_and_push_embeddings(
    document_path="./new-docs/",
    vector_store_id=vs_id,  # Same vector store
    metadata={"batch": "2024-03"}
)

---

Next Steps

• Overview - Back to main guide
• Embedding Operations - Working with embeddings
• Search & Filtering - Search strategies
• Examples - Complete implementations