OpenTelemetry for GenAI and the OpenLLMetry project -…

About this episode

Generative AI is everywhere, but how do we monitor and observe it? OpenTelemetry has been a prominent tool and standard for observability, and recently the OTel community has been aiming to expand its scope and cover GenAI workloads with semantic conventions and tools. In this episode, Horovits is joined by Nir Gazit, creator of the OpenLLMetry project, and member of the OpenTelemetry Generative AI SIG. We discuss new semantic conventions, tracing prompts and model behavior, the OpenLLMetry project’s journey, and what observability even means for modern AI systems. Nir Gazit is the CEO and co-founder of Traceloop, and brings a wealth of data and AI experience, with previous experience leading AI teams at Google and serving as the Chief Architect at Fiverr. You can read the recap post: https://medium.com/p/81b9cea6a771/ Show Notes: 00:00 - intro 04:09 - what is observability for AI 18:07 - AI observability differences from traditional observability 25:22 - OpenLLMetry intro 41:21 - OpenLLMetry latest updates and roadmap 47:00 - OpenTelemetry GenAI Semantic Conventions SIG 56:03 - KubeCon updates: CrossPlane, Knative, Dragonfly, in-toto reached CNCF graduation 1:00:08 - outro Resources: OpenTelemetry Generative AI Observability SIG: https://github.com/open-telemetry/community/blob/1c71595874e5d125ca92ec3b0e948c4325161c8a/projects/llm-semconv.md https://github.com/traceloop/openllmetry https://github.com/traceloop/hub https://github.com/traceloop/opentelemetry-mcp-server Socials: BlueSky: https://bsky.app/profile/openobservability.bsky.social Twitter: ⁠https://twitter.com/OpenObserv⁠ LinkedIn: https://www.linkedin.com/company/openobservability/ YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠ Dotan Horovits ============ Twitter: @horovits LinkedIn: www.linkedin.com/in/horovits Mastodon: @horovits@fosstodon BlueSky: @horovits.bsky.social Nir Gazit ======== Twitter: https://x.com/nir_ga LinkedIn: https://www.linkedin.com/in/nirga/ OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.