Skip to main content

Agent with RAG and Web Search

Research agent that combines internal knowledge with external information.

Python Implementation​

import requests

# Create agent
create_response = requests.post(
    "http://localhost/api/v1/agent/configs",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "name": "research_agent",
        "display_name": "Research Assistant",
        "description": "Combines RAG for internal docs with web search for current info",
        "category": "research",
        "system_prompt": "You are a research assistant. Use web search for current information and real-time data. Use RAG to search internal documentation. Always cite your sources. If information is not available in your knowledge base, state that explicitly.",
        "model": "gpt-4",
        "temperature": 0.3,
        "builtin_tools": ["rag_search", "web_search"],
        "tool_choice": "auto",
        "max_iterations": 5
    }
)

agent_id = create_response.json()["id"]

# Chat with agent
chat_response = requests.post(
    "http://localhost/api/v1/agent/chat",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "agent_id": agent_id,
        "message": "What are the latest developments in AI and what do our internal documents say about it?"
    }
)

result = chat_response.json()
print(f"Response: {result['response']}")
print(f"\nTools used:")
for tool_call in result.get("tool_calls", []):
    print(f"  - {tool_call['tool_name']}")
    if "result" in tool_call:
        print(f"    {tool_call['result']}")

Response Analysis​

The agent will automatically:

  1. Search RAG: Query internal documentation for relevant information
  2. Search Web: Find current developments and news
  3. Synthesize: Combine findings into a comprehensive answer
  4. Cite Sources: Reference where information came from

Use Cases​

QueryExpected Behavior
"What is our API key policy?"Search internal docs, return policy
"Latest AI news"Search web, summarize recent articles
"How does X feature work?"RAG search for X docs, web search for tutorials
"Compare X and Y"RAG for both, web search for latest info

Customization​

Adjust agent for specific research needs:

More Conservative​

response = requests.put(
    f"http://localhost/api/v1/agent/configs/{agent_id}",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "temperature": 0.2,  # Lower temperature for more factual
        "max_iterations": 8,  # More iterations for comprehensive research
        "tool_choice": "required"  # Force tool usage
    }
)

More Creative​

response = requests.put(
    f"http://localhost/api/v1/agent/configs/{agent_id}",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "temperature": 0.8,  # Higher temperature
        "max_iterations": 3  # Fewer iterations
        "tool_choice": "auto"  # Let agent decide when to use tools
    }
)

Best Practices​

System Prompt​

Be specific about research methodology:

Good:

"When researching a topic, first search our internal documentation using RAG, then supplement with web search for current information. Always cite your sources: use [Document X, Source Y] format. If information is not available in your knowledge base, state that explicitly rather than guessing."

Avoid:

"Do comprehensive research on the topic by using various search tools and synthesizing the information..."  # Too verbose

Tool Selection​

  • Start with RAG: Internal docs are often more accurate
  • Add web search: For current events and developments
  • Consider costs: Web search and code execution may incur additional costs
  • Set appropriate limits: Control top_k and search results

Evaluation​

# Evaluate agent performance
metrics = {
    "rag_hit_rate": 0.85,  # 85% of queries used RAG
    "web_search_usage": 0.45,  # 45% used web search
    "avg_response_time": 3.2,  # seconds
    "user_satisfaction": 0.92
}

# Adjust configuration based on metrics
if metrics["rag_hit_rate"] < 0.5:
    # Improve RAG search or prompts
    print("Consider improving RAG configuration")

Advanced Features​

Source Attribution​

# Agent should cite sources clearly
citation_format = """
[1] Internal Document: "{doc_title}" (Section: {section})
[2] Web Source: "{source_title}" ({source_url})
"""

# When synthesizing, include citations

Fact-Checking​

# Compare information from multiple sources
def fact_check_claim(claim, rag_results, web_results):
    rag_facts = extract_facts(rag_results)
    web_facts = extract_facts(web_results)

    # Identify discrepancies
    discrepancies = compare_facts(rag_facts, web_facts)

    return {
        "claim": claim,
        "rag_facts": rag_facts,
        "web_facts": web_facts,
        "discrepancies": discrepancies
    }

Troubleshooting​

RAG Returns No Results​

Problem: RAG search finds no relevant documents

Solution:

  • Verify collection has documents
  • Lower similarity_threshold in agent config
  • Increase top_k to retrieve more results
  • Check query terms match document content

Web Search Fails​

Problem: Brave Search API returns errors

Solution:

  • Verify BRAVE_SEARCH_API_KEY is configured
  • Check API key has sufficient quota
  • Implement retry logic for failed requests
  • Check for service outages

Agent Doesn't Use Tools​

Problem: Agent generates response without using tools

Solution:

  • Check tool_choice is not set to none
  • Verify tools are in builtin_tools list
  • Review system prompt for tool usage encouragement
  • Set tool_choice to required if tools should be used

Next Steps​