evals mcp server changelog

https://raw.githubusercontent.com/cyanheads/evals-mcp-server/HEAD/AGENTS.md

− **Version:** 0.1.6− **Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.15`− **Engines:** Bun ≥1.4.0, Node ≥24.0.0− **MCP SDK:** `@modelcontextprotocol/server` ^2.3.1− **Zod:** ^4.6.5− **Surface (9 tools, 1 resource):**− **Load-bearing invariants:** records are a Zod `discriminatedUnion` on `task_type`; the grader DSL is a typed union serialized with the record (deterministic kinds run server-side, `llm_rubric` relies on recorded independent verification); the submit gate admits a record only if the gold passes its deterministic grader, ≥1 negative is rejected, and a recorded, decorrelated independent verification agrees with the gold. Storage is the `record-store` service, **not** `ctx.state` — records must be −   captureDir: z.string().optional().describe('Directory of framework-written tool-call captures.'),− Skills are modular instructions in `framework-skills/` at the project root. Read them directly when a task matches — e.g., `framework-skills/add-tool/SKILL.md` when adding a tool. `bun run list-skills` prints the full registry. The directory is deliberately not `skills/`: Claude Code and Codex auto-load a plugin's root `skills/`, so a server that ships `.claude-plugin/` or `.codex-plugin/` would hand these development skills to every agent that installs it. Keep `skills/` free for skills meant f− `npm run bundle` produces a `.mcpb` extension bundle for one-click install in Claude Desktop. The pack step is followed by `scripts/clean-mcpb.ts`, which prunes dev dependencies (`mcpb clean`) and strips two classes of `node_modules/**` content that root-anchored `.mcpbignore` patterns cannot reach: dependency-shipped agent docs (`framework-skills/`, `skills/`, `.claude/`, `.agents/`, `SKILL.md`) and platform-specific native bindings, which would otherwise lock the bundle to the platform it was + **Version:** 0.2.0+ **Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.22`+ **Engines:** Bun ≥1.4.4, Node ≥24.0.0+ **MCP SDK:** `@modelcontextprotocol/server` ^2.4.0+ **Zod:** ^4.6.7+ **Surface (9 tools, 1 record resource, 1 published skill):**+ | `skill://evals-guide/SKILL.md` | Published guide to independent verification, submission, and export. |+ **Load-bearing invariants:** records are a Zod `discriminatedUnion` on `task_type`; the grader DSL is a typed union serialized with the record. Deterministic submit requires gold and every positive to pass, at least one negative and all negatives rejected, and agreeing decorrelated verification. `llm_rubric` skips deterministic grading but requires a declared negative and verification. Draft readiness shares the record-local checks; capture resolution, confirmation, and deduplication run at subm+   captureDir: z.string().optional().describe('Directory of tool-call capture files read at submit.'),+ Development skills live in `framework-skills/`. Read the matching skill directly or run `bun run list-skills` for the registry. Published skills live in root `skills/` and reach plugin installs, the npm package, Docker, and the MCPB bundle. To serve them over MCP, pass `skills: 'skills'` to `createApp()`. The `add-skill` skill covers authoring and delivery, and `lint:packaging` validates the tree.+ | `add-skill` | Author and validate a published skill for agents that use this server |+ `npm run bundle` produces a `.mcpb` extension bundle for Claude Desktop over stdio. Use root-anchored exclusions in `.mcpbignore` for source, tests, docs, caches, and local configuration. Root `skills/` ships in the bundle. After packing, `scripts/clean-mcpb.ts` prunes dev dependencies and strips dependency agent documentation, platform-specific bindings, node-gyp build output, and framework build-only files. The SDK's runtime skills modules stay in the bundle.+ - [ ] Root `plugin.json` and `mcp.json` follow Agent Plugins 1.0.0. Keep fixed environment values in root `mcp.json` and forward user variables through the Codex overlay's `env_vars`. The `mcpServers` pointer in `.codex-plugin/plugin.json` lets Codex merge that overlay.

version

0.1.6 → 0.2.0

https://raw.githubusercontent.com/cyanheads/evals-mcp-server/HEAD/AGENTS.md

− **Version:** 0.1.5− **Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.13`− **MCP SDK:** `@modelcontextprotocol/server` ^2.2.0+ **Version:** 0.1.6+ **Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.15`+ **MCP SDK:** `@modelcontextprotocol/server` ^2.3.1

version

0.1.5 → 0.1.6

https://raw.githubusercontent.com/cyanheads/evals-mcp-server/HEAD/AGENTS.md

− **Version:** 0.1.4− **Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.6`− **MCP SDK:** `@modelcontextprotocol/server` ^2.0.0− **Load-bearing invariants:** records are a Zod `discriminatedUnion` on `task_type`; the grader DSL is a typed union serialized with the record (deterministic kinds run server-side, `llm_rubric` relies on recorded independent verification); the submit gate admits a record only if the gold passes its deterministic grader, ≥1 negative is rejected, and a recorded, decorrelated independent verification agrees with the gold. Storage is the `record-store` service, **not** `ctx.state` — records must be − - **Use `ctx.state`** for tenant-scoped storage. Never access persistence directly.−   sessionMode: { default: 'stateful', require: 'stateful' }, // the submit confirmation needs a live session− | `ctx.state` | Tenant-scoped KV — `.get(key)`, `.set(key, value, { ttl? })`, `.delete(key)`, `.getMany(keys)`, `.list(prefix, { cursor, limit })`. Accepts any serializable value. |− | `ctx.inputs` | Reader over a retried request's responses — `.accepted(key, schema)`, `.view(key)`, `.state()`, `.dropped`. Empty on the first round. |− | `ctx.requestId` | Unique request ID. |− **Recommended: typed error contract.** Declare `errors: [{ reason, code, when, recovery, retryable?, severity?, thrownBy? }]` on `tool()` / `resource()` to receive `ctx.fail(reason, …)` typed against the reason union. TypeScript catches typos at compile time, `data.reason` is auto-populated for observability, linter enforces conformance against the handler body. `recovery` is required (≥ 5 words, lint-validated) — the single source of truth for the agent's next move. Pass `ctx.recoveryFor('reaso−   if (!item) throw ctx.fail('no_match', `No item ${input.id}`, ctx.recoveryFor('no_match'));− **Every release goes through a release PR, straight-through** — `git-wrapup`'s "Release PR mode", mode `straight-through`. One run: `git-wrapup` lands the commit stack on `release/<version>`, pushes it, and opens the PR (title = the release commit subject, body = the changelog entry plus a gates section); `release-and-publish` then fast-forwards `main` locally with `git merge --ff-only`, creates the tag on `main`'s tip, pushes `main` and the tag, deletes the branch, and publishes. A caller's bri+ **Version:** 0.1.5+ **Framework:** [@cyanheads/mcp-ts-core](https://www.npmjs.com/package/@cyanheads/mcp-ts-core) `^0.13.13`+ **MCP SDK:** `@modelcontextprotocol/server` ^2.2.0+ **Load-bearing invariants:** records are a Zod `discriminatedUnion` on `task_type`; the grader DSL is a typed union serialized with the record (deterministic kinds run server-side, `llm_rubric` relies on recorded independent verification); the submit gate admits a record only if the gold passes its deterministic grader, ≥1 negative is rejected, and a recorded, decorrelated independent verification agrees with the gold. Storage is the `record-store` service, **not** `ctx.state` — records must be + - **Use `ctx.state`** for tenant-scoped consent records. Eval records persist through the record-store service as human-inspectable JSON.+   sessionMode: { require: 'stateful' }, // the submit confirmation needs a live session+ | `ctx.state` | Tenant-scoped KV — `.get(key)`, `.set(key, value, { ttl? })`, `.delete(key)`, `.getMany(keys)`, `.list(prefix, { cursor, limit })`. Accepts any JSON-serializable value; reads return its JSON form (a `Date` comes back as an ISO string). |+ | `ctx.inputs` | The request's responses — `.accepted(key, schema)`, `.view(key)`, `.state()`, `.dropped` — limited to what the client declared (`elicitation` and its form/url modes, `sampling`, `roots`). Client-supplied: a consent gate trusts only a `ctx.state` record it stored when it asked, bound to the operation, caller, and target (see the `api-context` skill). |+ | `ctx.clientCapabilities` | What the client declared for this request, `undefined` when no view exists. Decides whether to ask for optional context (e.g. roots); never a reason to skip a consent prompt. |+ | `ctx.requestId` | Request ID — the one every log record of the call carries and its error envelope returns as `data.requestId`. |+ **Recommended: typed error contract.** Declare `errors: [{ reason, code, when, recovery, retryable?, severity?, thrownBy? }]` on `tool()` / `resource()` to receive `ctx.fail(reason, …)` typed against the reason union. TypeScript catches typos at compile time, `data.reason` is auto-populated for observability, linter enforces conformance against the handler body. `recovery` is required (≥ 5 words, lint-validated) — the single source of truth for the agent's next move. The framework puts it on the+   if (!item) throw ctx.fail('no_match', `No item ${input.id}`);+ **Every release goes through a release PR, straight-through** — `git-wrapup`'s "Release PR mode", mode `straight-through`. One run: `git-wrapup` lands the commit stack on `release/<version>`, pushes it, and opens the PR (title = the release commit subject, body = the release digest: theme line, `## Changes`, `## Gates`, changelog link last); `release-and-publish` then fast-forwards `main` locally with `git merge --ff-only`, creates the tag on `main`'s tip, pushes `main` and the tag, deletes the 

version

0.1.4 → 0.1.5