TheCrawler for Claude Code

io.github.manchittlab/thecrawler

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

client:Claude Code transport:stdio runtime:npm

Install TheCrawler in Claude Code

run in your project directory

claude mcp add thecrawler -- npx -y thecrawler

Adds it for this project only. Append --scope user to make it available everywhere. Claude Code docs