Solo.io Launches agentevals Open Source Project, Contributes agentregistry to CNCF

AMSTERDAM — Solo.io announced the launch of agentevals, a new open source project for evaluating and benchmarking agentic AI behavior, and the contribution of its agentregistry project to the Cloud Native Computing Foundation at KubeCon + CloudNativeCon Europe 2026. The company said the two initiatives address gaps in production reliability and governance for agentic AI workloads.

Solo.io said agentevals uses OpenTelemetry to capture and correlate individual invocations from distributed agentic interactions, then scores them against golden evaluation sets using an extensible evaluation engine. The project supports offline and online evaluation modes, ships with built-in evaluators for trajectory matching and LLM-as-judge scoring, and includes a CLI, web interface and Model Context Protocol server. The company said the tool works with any model and framework that emits OpenTelemetry spans, with no requirement for agent reruns.

“Evaluation is the biggest unsolved problem in agentic infrastructure today,” said Idit Levine, founder and CEO of Solo.io. “Organizations have frameworks for building agents, gateways for connecting them, and registries for governing them, but no consistent way to know whether an agent is actually reliable enough to trust in production.”

The agentregistry project, originally introduced by Solo.io in November 2025, provides a centralized registry where AI agents, MCP tools and agent skills are catalogued, discovered and governed. Solo.io said the contribution to CNCF governance will enable community growth alongside kagent, a CNCF sandbox project for running AI agents in Kubernetes, and agentgateway, which is housed in the Linux Foundation. The registry integrates with Kubernetes, AWS AgentCore and Google Vertex AI for deployment, and includes runtime discovery to detect agents deployed outside governed workflows.