retriEVAL for VS Code
io.github.hcarrillo001/retrieval-mcp
LLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination.
client:VS Code
transport:stdio
runtime:oci
Install retriEVAL in VS Code
.vscode/mcp.json
{
"inputs": [
{
"type": "promptString",
"id": "anthropic-api-key",
"description": "ANTHROPIC_API_KEY",
"password": true
},
{
"type": "promptString",
"id": "retrieval-judge-backend",
"description": "RETRIEVAL_JUDGE_BACKEND",
"password": true
},
{
"type": "promptString",
"id": "retrieval-judge-model",
"description": "RETRIEVAL_JUDGE_MODEL",
"password": true
},
{
"type": "promptString",
"id": "retrieval-budget-usd",
"description": "RETRIEVAL_BUDGET_USD",
"password": true
}
],
"servers": {
"retrieval-mcp": {
"type": "stdio",
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
],
"env": {
"ANTHROPIC_API_KEY": "${input:anthropic-api-key}",
"RETRIEVAL_JUDGE_BACKEND": "${input:retrieval-judge-backend}",
"RETRIEVAL_JUDGE_MODEL": "${input:retrieval-judge-model}",
"RETRIEVAL_BUDGET_USD": "${input:retrieval-budget-usd}"
}
}
}
}VS Code prompts for each input the first time the server starts, so no secret is written to the file. VS Code docs