AI-Enabled Delivery: Leveraging ChOP & LLMs in Delivering More Effective Learning Experiences at QCon

AI-Enabled Delivery: Leveraging ChOP & LLMs in Delivering More Effective Experiences at QCon dives into our journey and the lessons learned from using chat-oriented programming (ChOP), Retrieval-Augmented Generation (RAG), and Prompt Engineering to build an AI-driven certification program for the software conference QCon.

We wanted to create a certification program for conference attendees called the InfoQ Certified Architect in Emerging Technologies (ICAET) that centered around deeply understanding not just the content of QCon, but also capturing the experience of attending the event. To create this certification, we combined the intuition of a 16-time chair of the event, Wesley Reisz, with an authoritative knowledge base from the conference using Retrieval Augmented Generation.

This is the story of how we combined expertise found in human intuition with the power of an LLM to create experiences that far exceed what we could have done before. Some of the things we will discuss in the talk include:

• Insights from using chat-oriented programming to build a video pipeline that extracts text from videos, breaks the text into chunks, generates embeddings, and loads them into a vector database for an LLM.
• ChatOps, Vibe Coding, Agentic Engineering… 80% of the cost of a project is in the maintenance of the software we produce. So while AI tools enable us to move and get to market faster, what is the cost to the bigger picture? Agentic engineering leverages ChOP but keeps the developer (and their engineering expertise) in the driver's seat. We’ll talk about the difference and show actual working examples.
• Patterns we employed to build the training prompts, evaluate the solution’s capabilities and limitations, and extract actionable insights—such as leveraging traditional data stores for statistics and counts—to enhance the LLM's ability to count.
• Recommendations and pragmatic advice derived from using the intuition of an expert practitioner versus the LLM.

This talk is given by Wes Reisz, the 16-time chair of past QCons, who works at Equal Experts building solutions to reduce complexity in software. A key approach to reducing complexity is using AI Enabled Delivery.