Enterprises struggle with productionizing LLM-powered applications due to a lack of established practices, and developer constraints in productionizing LLM-powered applications. Moving beyond PoCs is especially difficult for conversational applications.
Inspired by how Netflix Championed microservices, we created the Intelligent Agents Platform (iA Platform) to reimagine AI development with agents as core building blocks, addressing these challenges and ensuring rapid adaptability.
In this talk we will share our learnings around:
- Practical patterns for productionizing LLM powered applications
- AI agents development lifecycle and toolkits we developed for cross functional teams (data scientists, engineers)
- Components and building blocks of the AI Agents computing platform we built on a JVM based stack (Kotlin first)
Interview:
What's the focus of your work these days?
We are building agents to run as services with the entire lifecycle of the agents to be managed by an underlying k8s first fabric . This allows us to scale both development and traffic with full lifecycle management optimized (e.g. multiple versions of agents, agents supporting different channels.)
We have gained insights on how to build and manage the lifecycle of agents with the core concerns abstracted in the platform enabling democratization of agent development and the time to market at scale.
How does your InfoQ Dev Summit Munich session address current challenges or trends in the industry?
There is a huge distance to cover from demos to productizing LLM based apps especially - customer facing. We have encountered these challenges head first and are baking these learnings into the platform we are building which allows democratization of building such. These include patterns, dealing with non-determinism with tactical and novel approaches, scaling the dev and lifecycle management approaches, etc.
Solving this at scale is another challenge the industry would face and we have made headways into building our agent platform to roll out use cases across multi countries (Telekom Europe), multilingual (European languages), and multi-channels (voice, chat, etc).
Several enterprises are figuring out what the right team structure skills (etc.) should required for applied AI applications. We have approached this not only from a platform perspective, but the lifecycle we have in the process of maturing, our Agent Development lifecycle, where product, engineers, data science, evals and testing teams have a well defined process while keeping the team very lean (platform team and use case development team.)
How do you see the concepts discussed in your InfoQ Dev Summit Munich session shaping the future of the industry?
We believe our learnings and platform could act as a blueprint for productionizing LLM applications with multi-agent paradigms, especially in larger enterprises (multi-country , multi-lingual, multi-channel, etc.,) and also provides the JVM-based dev community a solid and mature platform to build AI-apps.
We also envision our blueprints will serve as the foundation for inter-agent communication approaches and eventually an open ecosystem of interoperable agents.
Speaker
Arun Joseph
AI Engineering Lead @Deutsche Telekom, Previously Built IIoT Platforms for Merck, Thought Leader & Builder
Arun Joseph leads the Architecture and Engineering teams at Deutsche Telekom's AI Competence Center, driving the development and deployment of cutting-edge Generative AI (GenAI) solutions group wide across Europe. As a thought leader in applied AI, Arun has spearheaded the creation of an enterprise-grade agents computing platform, which now powers Deutsche Telekom’s LLM powered AI applications , including the award-winning service bot, Frag Magenta.
Arun is deeply passionate about the potential of GenAI solutions and the engineering design that heralds a new era of computing. His commitment to building world-class digital products and systems is matched by his dedication to nurturing top-tier teams. He is also an advocate for open source development; under his leadership, his teams have open source agents development frameworks.
Outside of work, Arun is a staunch advocate for the ethical deployment of technology, particularly AI. He champions the concept of the "Right to AI", striving for a more equitable world where access to AI is justly and ethically enabled for all.
Find Arun Joseph at:
Speaker
Patrick Whelan
Senior Software Engineer @Deutsche Telekom, 20+ Years in Engineering Enterprise Front- and Backends
- Over 20 years as Software Engineer, building enterprise Applications, both in the backend and frontend.
- Spent the last year as a lead Engineer on building an LLM-powered ChatBot for the Deutsche Telekom.
- Prior joining the AI Team, worked on applications around IoT, Cloud Service, and Content Management Systems for Online Portals.
- Interested in Koltin, Flutter and creating innovative applications around Generative AI.
- Creator and maintainer of the Kotlin AT framework
Arc: Agents Reactor (https://lmos-ai.github.io/arc)
