Website Crawler — Markdown Pages, Sitemaps & RAG Provenance for Claude Code

io.github.kriton-dev/website-to-markdown

Crawl sites to Markdown with sitemaps, globs and depth limits, obeying robots and crawl-delay

client:Claude Code transport:streamable-http

Install Website Crawler — Markdown Pages, Sitemaps & RAG Provenance in Claude Code

run in your project directory

claude mcp add --transport http website-to-markdown https://mcp.apify.com/?tools=hermeskd/website-to-markdown

Adds it for this project only. Append --scope user to make it available everywhere. Claude Code docs