FAQ

Does agentevals re-run my agent?

No. agentevals is built to score behavior from OpenTelemetry traces without re-running the agent.

What kind of telemetry does agentevals use?

agentevals works from OpenTelemetry trace data emitted by your agent system. See OTel Compatibility for more details.

Can I write my own evaluators?

Yes. See Custom Evaluators.

Can agentevals use external judging backends?

Yes. agentevals now includes an initial option to delegate evals to OpenAI’s Evals API. See OpenAI Evals API backend.

Can I deploy agentevals on Kubernetes?

Yes. The project now includes container deployment support and a Helm chart for Kubernetes. See Kubernetes & Helm.

Is agentevals only for batch processing?

No. There is also support for streaming-oriented workflows. See Streaming.