robots txt checker for Windsurf
io.github.ryuamarines/robots-txt-checker
Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
client:Windsurf
transport:streamable-http
tools:1
Install robots txt checker in Windsurf
~/.codeium/windsurf/mcp_config.json · windows: %USERPROFILE%\.codeium\windsurf\mcp_config.json
{
"mcpServers": {
"robots-txt-checker": {
"serverUrl": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
}
}
}Global only — Windsurf has no per-project MCP config. Refresh Cascade after saving. Windsurf docs