Firecrawl

Introduction

🚀 Firecrawl is a powerful web crawler and scraper service for AI applications, knowledge bases, and data pipelines.

Features

• Advanced Web Crawling: Sophisticated web crawling capabilities
• AI-Powered Processing: Intelligent content processing with AI
• High Performance: Scalable and efficient architecture
• Easy Deployment: Simple setup and configuration
• Multiple Services: API, Worker, and Playwright microservices

Quick Start

1. Fill in the required API keys and configuration
2. After deployment, visit the API endpoint for the interactive crawler UI

Configuration

After deployment, you can customize Firecrawl by modifying environment variables in your Zeabur service settings.

Required Environment Variables

• OPENAI_API_KEY: API key for LLM features (OpenAI)
• PUBLIC_DOMAIN: The domain for your Firecrawl service

Optional Configuration

• MODEL_NAME: LLM model name (default: gpt-4)
• MODEL_EMBEDDING_NAME: Embedding model name (default: text-embedding-ada-002)
• OLLAMA_BASE_URL: Ollama API base URL
• REDIS_PASSWORD: Redis authentication password

Architecture

This template deploys multiple services:

• API Service: Main Firecrawl API server
• Worker Service: Background job processing
• Playwright Service: Browser automation for complex scraping
• Redis: In-memory data store for caching and job queues

Reference

• GitHub Repository
• Official Documentation
• Zeabur Environment Variable Guide

License

MIT