What’s It Doing That For? – Observability for AI in Production
As we all know, AI can be brilliant one minute and awful the next. Living in an unpredictable world of non-deterministic models can be… well, confusing. Especially as we move from prototypes to production, the question “what’s it doing now?” stops being a joke and starts becoming an operational concern.
This talk explores the emerging field of AI observability — the art (and occasional dark magic) of keeping LLM-powered systems reliable, explainable, and cost-effective once they’re out in the wild. We’ll look at practical approaches and tools for tracing behaviour across prompts, spotting drift, and catching those quiet regressions that sneak in after a model update.