Advanced

Docs

Guide	Description
Eval Set Format	Schema, field reference, and examples for golden eval set JSON files
Custom Evaluators	Write your own scoring logic in Python, JavaScript, or any language
Live Streaming	Real-time trace streaming, dev server setup, and session management
OpenTelemetry Compatibility	Supported OTel conventions, message delivery mechanisms, and OTLP receiver

REST API Reference

While the server is running (agentevals serve), interactive API documentation is available at:

Endpoint	Description
`/docs`	Swagger UI with interactive request builder
`/redoc`	ReDoc reference documentation
`/openapi.json`	Raw OpenAPI 3.x schema (for code generation or CI)

The OTLP receiver (port 4318) serves its own docs at http://localhost:4318/docs.

Development

uv run pytest                      # run tests
uv run agentevals serve --dev      # backend
cd ui && npm run dev               # frontend (separate terminal)

See DEVELOPMENT.md for build tiers, Makefile targets, and Nix setup. To contribute, see CONTRIBUTING.md.