tavily

# Tavily AI Search ## Overview Tavily is a search engine specifically optimized for Large Language Models and AI applications. Unlike traditional search APIs, Tavily provides AI-ready results with optional answer generation, clean content extraction, and domain filtering capabilities. **Key capabilities:** - AI-generated answer summaries from search results - Clean, structured results optimized for LLM processing - Fast (`basic`) and comprehensive (`advanced`) search modes - Domain filtering (include/exclude specific sources) - News-focused search for current events - Image search with relevant visual content - Raw content extraction for deeper analysis ## Architecture ```mermaid graph TB A[User Query] --> B{Search Mode} B -->|basic| C[Fast Search<br/>1-2s response] B -->|advanced| D[Comprehensive Search<br/>5-10s response] C --> E[Tavily API] D --> E E --> F{Topic Filter} F -->|general| G[Broad Web Search] F -->|news| H[News Sources<br/>Last 7 days] G --> I[Domain Filtering] H --> I I --> J{Include Domains?} J -->|yes| K[Filter to Specific Domains] J -->|no| L{Exclude Domains?} K --> M[Search Results] L -->|yes| N[Remove Unwanted Domains] L -->|no| M N --> M M --> O{Response Options} O --> P[AI Answer<br/>Summary] O --> Q[Structured Results<br/>Title, URL, Content, Score] O --> R[Images<br/>if requested] O --> S[Raw HTML Content<br/>if requested] P --> T[Return to Agent] Q --> T R --> T S --> T style E fill:#4A90E2 style P fill:#7ED321 style Q fill:#7ED321 style R fill:#F5A623 style S fill:#F5A623 ``` ## Quick Start ### Basic Search ```bash # Simple query with AI answer scripts/tavily_search.py "What is quantum computing?" # Multiple results scripts/tavily_search.py "Python best practices" --max-results 10 ``` ### Advanced Search ```bash # Comprehensive research mode scripts/tavily_search.py "Climate change solutions" --depth advanced # News-focused search scripts/tavily_search.py "AI developments 2026" --topic news ``` ### Domain Filtering ```bash # Search only trusted domains scripts/tavily_search.py "Python tutorials" \ --include-domains python.org docs.python.org realpython.com # Exclude low-quality sources scripts/tavily_search.py "How to code" \ --exclude-domains w3schools.com geeksforgeeks.org ``` ### With Images ```bash # Include relevant images scripts/tavily_search.py "Eiffel Tower architecture" --images ``` ## Search Modes ### Basic vs Advanced | Mode | Speed | Coverage | Use Case | |------|-------|----------|----------| | **basic** | 1-2s | Good | Quick facts, simple queries | | **advanced** | 5-10s | Excellent | Research, complex topics, comprehensive analysis | **Decision tree:** 1. Need a quick fact or definition? → Use `basic` 2. Researching a complex topic? → Use `advanced` 3. Need multiple perspectives? → Use `advanced` 4. Time-sensitive query? → Use `basic` ### General vs News | Topic | Time Range | Sources | Use Case | |-------|------------|---------|----------| | **general** | All time | Broad web | Evergreen content, tutorials, documentation | | **news** | Last 7 days | News sites | Current events, recent developments, breaking news | **Decision tree:** 1. Query contains "latest", "recent", "current", "today"? → Use `news` 2. Looking for historical or evergreen content? → Use `general` 3. Need up-to-date information? → Use `news` ## API Key Setup ### Option 1: Clawdbot Config (Recommended) Add to your Clawdbot config: ```json { "skills": { "entries": { "tavily": { "enabled": true, "apiKey": "tvly-YOUR_API_KEY_HERE" } } } } ``` Access in scripts via Clawdbot's config system. ### Option 2: Environment Variable ```bash export TAVILY_API_KEY="tvly-YOUR_API_KEY_HERE" ``` Add to `~/.clawdbot/.env` or your shell profile. ### Getting an API Key 1. Visit https://tavily.com 2. Sign up for an account 3. Navigate to your dashboard 4. Generate an API key (starts with `tvly-`) 5. Note your plan's rate limits and credit allocation ## Common Use Cases ### 1. Research & Fact-Finding ```bash # Comprehensive research with answer scripts/tavily_search.py "Explain quantum entanglement" --depth advanced # Multiple authoritative sources scripts/tavily_search.py "Best practices for REST API design" \ --max-results 10 \ --include-domains github.com microsoft.com google.com ``` ### 2. Current Events ```bash # Latest news scripts/tavily_search.py "AI policy updates" --topic news # Recent developments in a field scripts/tavily_search.py "quantum computing breakthroughs" \ --topic news \ --depth advanced ``` ### 3. Domain-Specific Research ```bash # Academic sources only scripts/tavily_search.py "machine learning algorithms" \ --include-domains arxiv.org scholar.google.com ieee.org # Technical documentation scripts/tavily_search.py "React hooks guide" \ --include-domains react.dev ``` ### 4. Visual Research ```bash # Gather visual references scripts/tavily_search.py "modern web design trends" \ --images \ --max-results 10 ``` ### 5. Content Extraction ```bash # Get raw HTML content for deeper analysis scripts/tavily_search.py "Python async/await" \ --raw-content \ --max-results 5 ``` ## Response Handling ### AI Answer The AI-generated answer provides a concise summary synthesized from search results: ```python { "answer": "Quantum computing is a type of computing that uses quantum-mechanical phenomena..." } ``` **Use when:** - Need a quick summary - Want synthesized information from multiple sources - Looking for a direct answer to a question **Skip when** (`--no-answer`): - Only need source URLs - Want to form your own synthesis - Conserving API credits ### Structured Results Each result includes: - `title`: Page title - `url`: Source URL - `content`: Extracted text snippet - `score`: Relevance score (0-1) - `raw_content`: Full HTML (if `--raw-content` enabled) ### Images When `--images` is enabled, returns URLs of relevant images found during search. ## Best Practices ### 1. Choose the Right Search Depth - Start with `basic` for most queries (faster, cheaper) - Escalate to `advanced` only when: - Initial results are insufficient - Topic is complex or nuanced - Need comprehensive coverage ### 2. Use Domain Filtering Strategically **Include domains for:** - Academic research (`.edu` domains) - Official documentation (official project sites) - Trusted news sources - Known authoritative sources **Exclude domains for:** - Known low-quality content farms - Irrelevant content types (Pinterest for non-visual queries) - Sites with paywalls or access restrictions ### 3. Optimize for Cost - Use `basic` depth as default - Limit `max_results` to what you'll actually use - Disable `include_raw_content` unless needed - Cache results locally for repeated queries ### 4. Handle Errors Gracefully The script provides helpful error messages: ```bash # Missing API key Error: Tavily API key required Setup: Set TAVILY_API_KEY environment variable or pass --api-key # Package not installed Error: tavily-python package not installed To install: pip install tavily-python ``` ## Integration Patterns ### Programmatic Usage ```python from tavily_search import search result = search( query="What is machine learning?", api_key="tvly-...", search_depth="advanced", max_results=10 ) if result.get("success"): print(result["answer"]) for item in result["results"]: print(f"{item['title']}: {item['url']}") ``` ### JSON Output for Parsing ```bash scripts/tavily_search.py "Python tutorials" --json > results.json ``` ### Chaining with Other Tools ```bash # Search and extract content scripts/tavily_search.py "React documentation" --json | \ jq -r '.results[].url' | \ xargs -I {} curl -s {} ``` ## Comparison with Other Search APIs **vs Brave Search:** - ✅ AI answer generation - ✅ Raw content extraction - ✅ Better domain filtering - ❌ Slower than Brave - ❌ Costs credits **vs Perplexity:** - ✅ More control over sources - ✅ Raw content available - ✅ Dedicated news mode - ≈ Similar answer quality - ≈ Similar speed **vs Google Custom Search:** - ✅ LLM-optimized results - ✅ Answer generation - ✅ Simpler API - ❌ Smaller index - ≈ Similar cost structure ## Troubleshooting ### Script Won't Run ```bash # Make executable chmod +x scripts/tavily_search.py # Check Python version (requires 3.6+) python3 --version # Install dependencies pip install tavily-python ``` ### API Key Issues ```bash # Verify API key format (should start with tvly-) echo $TAVILY_API_KEY # Test with explicit key scripts/tavily_search.py "test" --api-key "tvly-..." ``` ### Rate Limit Errors - Check your plan's credit allocation at https://tavily.com - Reduce `max_results` to conserve credits - Use `basic` depth instead of `advanced` - Implement local caching for repeated queries ## Resources See [api-reference.md](references/api-reference.md) for: - Complete API parameter documentation - Response format specifications - Error handling details - Cost and rate limit information - Advanced usage examples ## Dependencies - Python 3.6+ - `tavily-python` package (install: `pip install tavily-python`) - Valid Tavily API key ## Credits & Attribution - Tavily API: https://tavily.com - Python SDK: https://github.com/tavily-ai/tavily-python - Documentation: https://docs.tavily.com