Observability & Evaluation

The instrumentation plane: traces, evals and the benchmarks that age.

Pantone 18-1438 Marsala · Color of the Year 2015 · 2 pieces

Traces and Trust: LangSmith, Langfuse, Arize, Braintrust and the Observability Layer

AI agent observability reaches 89% of teams while evaluation reaches 52%, and the summer's evaluation escapes showed what a trace is worth when it exists.

7 min · 14 sources

Benchmarks, Broken and Better: Evaluating Agents from SWE-bench to ARC-AGI-3

AI agent benchmarks now carry cost per task, confidence intervals and contamination warnings, and the careful buyer treats every leaderboard as an instrument with a stated error bar.

8 min · 14 sources