crawl readiness for VS Code

io.github.grubbomatic/crawl-readiness

Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.

client:VS Code transport:stdio runtime:npm

Install crawl readiness in VS Code

.vscode/mcp.json

{
  "inputs": [
    {
      "type": "promptString",
      "id": "crawl-readiness-api-key",
      "description": "CRAWL_READINESS_API_KEY",
      "password": true
    },
    {
      "type": "promptString",
      "id": "crawl-readiness-api-base",
      "description": "CRAWL_READINESS_API_BASE",
      "password": true
    }
  ],
  "servers": {
    "crawl-readiness": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "crawl-readiness-mcp"
      ],
      "env": {
        "CRAWL_READINESS_API_KEY": "${input:crawl-readiness-api-key}",
        "CRAWL_READINESS_API_BASE": "${input:crawl-readiness-api-base}"
      }
    }
  }
}

VS Code prompts for each input the first time the server starts, so no secret is written to the file. VS Code docs

crawl readiness in other clients

crawl readiness MCP server