robots txt checker for Grok
io.github.ryuamarines/robots-txt-checker
Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
client:Grok
transport:streamable-http
tools:1
Install robots txt checker in Grok
.mcp.json (in your project root)
{
"mcpServers": {
"robots-txt-checker": {
"type": "http",
"url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
}
}
}Grok Build reads Claude Code's .mcp.json and Claude Desktop's config unchanged, so a server you have already set up there needs nothing new. The Grok web app and X integration do not take custom servers. Grok docs