LLM Agents 360°: A Holistic View on Frameworks, Systems, and Simulations. @ ICLR 2025
About Us
Come join us at the ICLR 2025 Social:
LLM Agents 360°: A Holistic View on Frameworks, Systems, and Simulations
LLM agents are transforming automation, research, and real-world applications. With their increasing adoption, understanding the full landscape - from foundational frameworks to deployment trade-offs - is more critical than ever.
OpenAI has just released the Agents SDK, and MCP from Anthropic is also available.
This social event at ICLR 2025 will provide a comprehensive view of the evolution of LLM agents, exploring when and where they provide the most value, their strengths and limitations, and the critical factors in building reliable, scalable systems.
It will also cover the future of AI agents, including protocols, simulations, and emerging trends.
The event’s agenda now includes four short expert talks and a fireside chat with AI leaders working on AI agents.
Attendees will gain valuable insights into different frameworks, system architectures, and simulation approaches, helping them make informed decisions about using LLM agents in their own work.
They will also have the opportunity to exchange ideas with top researchers and practitioners, explore collaborative opportunities, and network with others interested in AI agents.
Agenda
Invited Talks
Bo An (15 min)
The Rise of LLM Agents: Building on Past Foundations:
Why LLM Agents? What limitations of older agent approaches led to the rise of LLM agents?
What Can They Do Now? What complex tasks demonstrate the current capabilities of LLM agents?
How Do They Work? What key techniques and architectures power today's LLM agents?
Weiwen Liu (15 min)
Making LLM Agents Work: Addressing Key Bottlenecks:
Why is it hard for AI agents to do things correctly (not just talk)?
What makes it difficult for agents to use tools well, and how can we fix that?
If agents get better at using tools, what more can they do, and what are the biggest problems left?
Xiao Liu (15 min)
AgentBench and Beyond: Quantifying Progress and Challenges in LLM Agency
Why is evaluating agent actions harder than evaluating language?
How does AgentBench measure and compare different LLMs as agents?
What key agent limits do benchmarks reveal, and what's next for evaluation?
Ang Li / Saaket Agashe (15 min)
From Lab Bench to Desktop: Engineering LLM Agents for Real-World Productivity
Agent S and Agent S2
Reality Check: What were the biggest unexpected engineering hurdles in taking LLM agents from research to a real-world product?
System Scaffolding: Beyond the LLM, what essential system components make agents reliable for complex tasks?
Human + Agent: How do you design agent interaction for user trust, control, and graceful failure handling in practice?
Irakli Butskhrikidze (10 min)
Agents in the Wild: Lessons from Real User Deployments
What behaviors emerge when agents interact with unpredictable, unscripted users
How users actually perceive and engage with agents (trust, frustration, delight)
Common failure modes in the wild — and how teams debug or prevent them
What product teams learned about user expectations and what "intelligence" means in practice
Panel Discussion
Future Directions in Multi-Agent AI systems (25 min)
We will discuss enhanced collaboration among agents, scalability and efficiency challenges, and approaches to improve robustness and adaptability in dynamic environments, along with ethical considerations and real-world applications that are shaping the future of multi-agent AI systems.
Our panel features a group of honored researchers, developers and founders who are at the forefront of agentic AI. They bring a rich background in creating and advancing AI frameworks and agent-based systems. Their work covers a wide range of topics including the development of open-source AI frameworks, fundamental research into human-like intelligence and agent cooperation, and efforts to ensure the safety and effectiveness of AI systems through innovative methodologies. This diverse expertise contributes to a deeper understanding of how agentic AI can evolve and integrate into various technological and societal domains.
Panelists include: