AI Crawler Checker — Robots.txt Policy for Six Bots for VS Code
io.github.kriton-dev/ai-crawler-access-check
Check declared robots.txt permissions for six AI crawlers at URL paths
client:VS Code
transport:streamable-http
Install AI Crawler Checker — Robots.txt Policy for Six Bots in VS Code
.vscode/mcp.json
{
"servers": {
"ai-crawler-access-check": {
"type": "http",
"url": "https://mcp.apify.com/?tools=hermeskd/ai-crawler-access-check"
}
}
}Requires agent mode. MCP: Open User Configuration puts the same block in every workspace. VS Code docs