Crawl4AI

LLM-friendly web crawler that turns websites into clean Markdown for RAG and agentic workflows.

Guide pending

We haven't written this guide yet.

Guides are drafted from the project's own README and metadata. Until one exists, check the product's repository and documentation for authoritative information.

Nearby on the map

Related projects

Scrapling

D4Vinci/Scrapling

Adaptive web scraping and crawling framework for robust structured extraction from pages to large-scale pipelines.

unknown unknownactivity unknown

Lightpanda

lightpanda-io/browser

Machine-first headless browser in Zig; rendering-free and ultra-lightweight for AI agent browsing.

unknown unknownactivity unknown

Paperless-AI

clusterzx/paperless-ai

Automated document analyzer for Paperless-ngx with RAG-powered semantic search across your document archive.

unknown unknownactivity unknown

Firecrawl

firecrawl/firecrawl

Web Data API for AI - search, scrape, and interact with the web at scale. Clean markdown/JSON output with proxy rotation and JS-blocking handled automatically.

unknown unknownactivity unknown

wigolo

KnockOutEZ/wigolo

Local-first web intelligence and Model Context Protocol (MCP) server for web search, crawling, and structured data extraction. AGPL-3.0 licensed.

unknown unknownactivity unknown

Zoom Search

goofrey/zoom-search

MCP search and evidence tool for AI agents with query rewriting, source zoom-in, sourced answers, and runtime metrics. MIT licensed.

unknown unknownactivity unknown