Enterprises struggle with productionizing LLM-powered applications due to a lack of established practices, and developer constraints in productionizing LLM-powered applications. Moving beyond PoCs is especially difficult for conversational applications.
Inspired by how Netflix Championed microservices, we created the Intelligent Agents Platform (iA Platform) to reimagine AI development with agents as core building blocks, addressing these challenges and ensuring rapid adaptability.
In this talk we will share our learnings around:
- Practical patterns for productionizing LLM powered applications
- AI agents development lifecycle and toolkits we developed for cross functional teams (data scientists, engineers)
- Components and building blocks of the AI Agents computing platform we built on a JVM based stack (Kotlin first)
Interview:
What's the focus of your work these days?
We are building agents to run as services with the entire lifecycle of the agents to be managed by an underlying k8s first fabric . This allows us to scale both development and traffic with full lifecycle management optimized (e.g. multiple versions of agents, agents supporting different channels.)
We have gained insights on how to build and manage the lifecycle of agents with the core concerns abstracted in the platform enabling democratization of agent development and the time to market at scale.
How does your InfoQ Dev Summit Munich session address current challenges or trends in the industry?
There is a huge distance to cover from demos to productizing LLM based apps especially - customer facing. We have encountered these challenges head first and are baking these learnings into the platform we are building which allows democratization of building such. These include patterns, dealing with non-determinism with tactical and novel approaches, scaling the dev and lifecycle management approaches, etc.
Solving this at scale is another challenge the industry would face and we have made headways into building our agent platform to roll out use cases across multi countries (Telekom Europe), multilingual (European languages), and multi-channels (voice, chat, etc).
Several enterprises are figuring out what the right team structure skills (etc.) should required for applied AI applications. We have approached this not only from a platform perspective, but the lifecycle we have in the process of maturing, our Agent Development lifecycle, where product, engineers, data science, evals and testing teams have a well defined process while keeping the team very lean (platform team and use case development team.)
How do you see the concepts discussed in your InfoQ Dev Summit Munich session shaping the future of the industry?
We believe our learnings and platform could act as a blueprint for productionizing LLM applications with multi-agent paradigms, especially in larger enterprises (multi-country , multi-lingual, multi-channel, etc.,) and also provides the JVM-based dev community a solid and mature platform to build AI-apps.
We also envision our blueprints will serve as the foundation for inter-agent communication approaches and eventually an open ecosystem of interoperable agents.
Speaker
Arun Joseph
Founder @Rhizome Foundry GmbH | Co-Founder @Masaic AI (AgC™, Agentic Compute) | Envisioned & Lead @Eclipse LMOS | Former Head of AI Engineering, Deutsche Telekom AG
Arun Joseph is the co-founder & CEO of Masaic, a stealth startup building AGC , the Agentic Compute: a new computing substrate for knowledge work. AGC is open by design and built for scale, providing the architectural foundation for decisioning and actioning systems that integrate with existing enterprise stacks.
Previously, he headed engineering at Deutsche Telekom AG’s Central AI program,AICC, where he envisioned & led one of Europe’s first agentic PaaS platforms, deployed at multi-country scale and now part of the Eclipse Foundation. Known as Eclipse LMOS, the platform demonstrated how empowering existing teams could produce best in class, cost-efficient systems that made work meaningful and accelerated results delivery.
He also founded Rhizome Foundry GmBH, a builder’s movement and foundry for agentic computing companies and systems, shaping the principles and infrastructure of this emerging paradigm.
Find Arun Joseph at:
Speaker
Patrick Whelan
Senior Software Engineer @Deutsche Telekom, 20+ Years in Engineering Enterprise Front- and Backends
- Over 20 years as Software Engineer, building enterprise Applications, both in the backend and frontend.
- Spent the last year as a lead Engineer on building an LLM-powered ChatBot for the Deutsche Telekom.
- Prior joining the AI Team, worked on applications around IoT, Cloud Service, and Content Management Systems for Online Portals.
- Interested in Koltin, Flutter and creating innovative applications around Generative AI.
- Creator and maintainer of the Kotlin AT framework
Arc: Agents Reactor (https://lmos-ai.github.io/arc)
