UiPath/coder_eval
Stack:
Python
Claude Code
Docker
TypeScript
Open-source (Apache-2.0) framework from UiPath to evaluate and benchmark coding agents and their skills: declarative YAML tasks run in a sandbox against Claude Code, Codex or Antigravity, weighted 0.0–1.0 scoring, A/B experiments and a CI gate via GitHub Action.
Why it's interesting
I picked it because it answers the question nobody asks after writing a skill or an MCP server: "does it actually fire?". The `skill_triggered` criterion reads the agent's traces and hands you a confusion matrix with precision and recall — you stop eyeballing it and start measuring.