Description: RAG systems are often positioned as a remedy for hallucination by grounding model outputs in enterprise data. However, they introduce distinct quality risks across ingestion pipelines, embedding generation, vector indexing, retrieval logic, ranking mechanisms, and final response synthesis. Failures at any stage can produce answers that appear credible yet are incomplete, outdated, or contextually incorrect.
This session examines how to systematically test and validate Retrieval-Augmented Generation systems in production environments. It will cover common failure modes, evaluation frameworks, adversarial testing approaches, and measurable validation metrics. Attendees will gain practical strategies to bring observability, structure, and reliability to RAG architectures beyond surface-level prompt testing. About the Presenter, Harsha Sri, Engineering Manager-OSPO, TestMu AI:
Harsha is a highly accomplished technical leader with deep expertise in test automation and modern JavaScript tooling. He is recognized for his ability to anticipate future challenges and design practical, scalable solutions that enhance both reliability and performance. He is an active contributor to the open-source community, serving as a committer to Selenium and WebdriverIO, and participating in the Technical Leadership Committee of SeleniumHQ.
Currently, Harsha serves as Engineering Manager (OSPO) at TestMu AI, where he champions open-source collaboration and continues to advance innovative approaches to building, testing, and scaling software within the global developer ecosystem.
