CompletionKit for Claude Desktop
com.completionkit/evals
Prompt evals over MCP: run a prompt on your dataset, score each output 1-5 with an LLM judge.
client:Claude Desktop
transport:streamable-http
tools:54
Install CompletionKit in Claude Desktop
~/Library/Application Support/Claude/claude_desktop_config.json · windows: %APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"evals": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://completionkit.com/mcp"
]
}
}
}Claude Desktop has no remote transport in its config file, so this bridges through mcp-remote. Restart the app after saving. Claude Desktop docs
What Claude Desktop can do once it is connected
- prompts_list
- prompts_get
- prompts_create
- prompts_update
- prompts_delete
- prompts_publish
- prompts_suggest_improvement
- runs_list
- runs_get
- runs_create
- runs_update
- runs_delete
- runs_generate
- runs_regrade
- runs_rerun
- runs_retry_failures
- responses_list
- responses_get
- datasets_list
- datasets_get
- datasets_create
- datasets_update
- datasets_delete
- datasets_create_from_url