AI Crawler Checker — Robots.txt Policy for Six Bots for Claude Code

io.github.kriton-dev/ai-crawler-access-check

Check declared robots.txt permissions for six AI crawlers at URL paths

client:Claude Code transport:streamable-http

Install AI Crawler Checker — Robots.txt Policy for Six Bots in Claude Code

run in your project directory

claude mcp add --transport http ai-crawler-access-check https://mcp.apify.com/?tools=hermeskd/ai-crawler-access-check

Adds it for this project only. Append --scope user to make it available everywhere. Claude Code docs