Agentic Memory: Stateful AI & RAG Extensions (Chapter 16)

MEMRIQ INFERENCE DIGEST - LEADERSHIP EDITION Episode: Agentic Memory: Chapter 16 Deep Dive into Stateful AI & RAG Extensions

MORGAN: 00:00

Welcome to Memriq Inference Digest - Leadership Edition, your go-to podcast for cutting-edge AI insights tailored for decision makers. This show is brought to you by Memriq AI, a content studio building tools and resources for AI practitioners — check them out at Memriq.ai.

CASEY: 00:20

Today, we’re diving into a fascinating topic from Chapter 16 of 'Unlocking Data with Generative AI and RAG' by Keith Bourne: Agentic Memory — Extending Retrieval-Augmented Generation, or RAG, with Stateful Intelligence. We’ll explore how AI agents can go beyond simple one-time responses and actually remember, learn, and adapt over time to become more like trusted colleagues.

MORGAN: 00:42

If you want to go deeper beyond our highlights — with detailed diagrams, thorough explanations, and hands-on code labs — just search for Keith Bourne on Amazon and grab the second edition of his book.

CASEY: 00:55

And we’re thrilled to have Keith himself joining us throughout the episode to share insider insights, behind-the-scenes thinking, and real-world experience that didn’t all make it into the book.

MORGAN: 01:07

Over the next 20 minutes, we’ll unpack the strategic implications of agentic memory, compare some leading tools like Mem0, LangMem, Zep and Graphiti, discuss real business use cases, and even debate which approach suits which scenario best. So, let’s get started.

JORDAN: 01:25

Here’s something that really flips the script on how most people think about AI chatbots. What if I told you that agentic memory transforms AI from a forgetful, one-off responder into a smart, adaptive partner that actually learns and remembers your preferences over time?

MORGAN: 01:42

Wait, so these aren’t just chatbots that start fresh each time?

JORDAN: 01:46

Exactly. Unlike traditional bots that forget everything once the conversation ends, agents with memory build ongoing relationships by recalling past interactions and evolving their knowledge. It’s like moving from a customer service rep who answers a question once, to a colleague who knows your history, context, and even anticipates your needs.

CASEY: 02:05

That’s a powerful shift. It means AI can act more like a knowledgeable teammate rather than a transactional assistant.

MORGAN: 02:12

And I imagine that kind of personalization could massively improve customer engagement and operational efficiency.

JORDAN: 02:17

Spot on. It opens up strategic advantages for companies willing to invest in AI that remembers — driving loyalty, reducing friction, and enabling smarter decision-making over time.

CASEY: 02:29

To boil it down in one sentence: Agentic memory extends Retrieval-Augmented Generation by equipping AI agents with memory systems that let them remember, learn, and adapt through structured, evolving knowledge stores.

MORGAN: 02:43

And the tools leading the pack here are Mem0, LangMem, Zep, and Graphiti — each taking a different approach to how that memory is stored and used.

CASEY: 02:52

So if you remember nothing else, think of agentic memory as the technology turning AI from a flash-in-the-pan answer machine into a strategic, long-term partner in your business.

JORDAN: 03:04

So, why is now the moment for agentic memory? The problem before was that most AI models had a limited “context window”—basically, they could only “remember” a small chunk of recent conversation or data. That meant if you wanted AI to recall anything beyond that, it simply forgot.

CASEY: 03:22

Right, and that short context window was a real bottleneck for applications needing ongoing, personalized relationships, like in finance, healthcare, or education.

JORDAN: 03:30

Exactly. Now, advances in memory architectures and the ability to dramatically expand these context windows have changed the game. AI can maintain state—what we call “stateful AI”—which means it keeps track of history over long periods, not just minutes.

MORGAN: 03:45

Who’s actually adopting these systems now?

JORDAN: 03:48

Leading financial advisors are using agentic memory to remember client portfolios and risk profiles over months. Healthcare assistants track patient histories and preferences longitudinally. Educational platforms adapt to individual students’ progress. Basically, companies focused on customer experience and operational continuity see massive value here.

CASEY: 04:05

So the convergence of business demand and technical readiness creates a unique window for investment in stateful AI.

JORDAN: 04:11

Spot on — this isn’t just a cool tech trick. It’s a strategic lever to unlock competitive advantage today.

TAYLOR: 04:19

Let’s get to the heart of it. Agentic memory equips AI agents with multiple types of memory — think of it like how humans remember different things in different ways. We have working memory for short-term context, episodic memory for experiences, semantic memory for facts, and procedural memory for skills and processes.

MORGAN: 04:37

So it’s not just one big “memory bank” but a more nuanced system?

TAYLOR: 04:41

Exactly. This is a core idea in the CoALA framework that Keith introduces in the book — it models AI memory architecture after human cognition to create richer, more adaptive interactions. Traditional RAG systems focus on retrieving documents or facts statically — a bit like referencing a library on demand. Agentic memory extends this by creating dynamic, personalized knowledge stores that evolve. So the AI doesn’t just pull information — it learns from prior interactions to generate smarter responses.

MORGAN: 05:08

Keith, as the author, what made you decide to cover agentic memory so prominently in your book?

KEITH: 05:13

Great question, Morgan. When I was consulting with companies deploying AI at scale, I saw a clear gap between the promise of RAG and the reality: AI that forgets too quickly can’t build trust or provide lasting value. Agentic memory is the bridge — it transforms AI into a partner that truly understands context over time. Covering it early helps leaders grasp the strategic shift from static retrieval to dynamic intelligence.

TAYLOR: 05:37

That really frames the architectural decisions companies need to make upfront.

TAYLOR: 05:41

Now, let’s compare how some leading tools approach agentic memory, each with their own trade-offs.

CASEY: 05:47

I’ll push back when needed — these sound great, but what’s the catch in each?

TAYLOR: 05:51

First up, Mem0. It offers a unified long-term memory store that prioritizes simplicity and performance, without explicitly separating memory types. That’s great if you want fast, easy deployment with solid accuracy.

CASEY: 06:03

But does that mean it might miss nuances by treating all memories the same?

TAYLOR: 06:07

Possibly. Then there’s LangMem, which explicitly separates memory into working, episodic, semantic, and procedural types — aligning with the CoALA framework. This allows for more nuanced memory management and supports procedural memory, which means the AI can learn skills or routines over time.

MORGAN: 06:22

Sounds more sophisticated but probably more complex to build.

TAYLOR: 06:25

Exactly. Next, Zep paired with Graphiti uses temporal knowledge graphs to track how memories evolve over time, which aids historical reasoning and auditability. However, it doesn’t natively support procedural memory.

CASEY: 06:38

So great for tracking and explaining decisions over time but limited in adaptive behavior?

TAYLOR: 06:42

Right. Each has strengths and trade-offs around complexity, capability, and operational needs.

CASEY: 06:48

So from a leadership perspective, you’d choose Mem0 when you want speed and simplicity, LangMem when you need detailed cognitive memory modeling, and Zep/Graphiti when audit trails and temporal reasoning are critical.

TAYLOR: 06:59

Perfect summary.

ALEX: 07:02

Let’s peel back the curtain and walk through how agentic memory works under the hood — the step-by-step flow. First, the AI captures raw interactions — conversations, user inputs, or sensor data. This goes into a memory extraction pipeline, which distills the information into structured memories. These memories are categorized into four types: Working memory: Think of this as the AI’s short-term scratchpad to maintain immediate context. Episodic memory: Stores specific experiences like “the customer asked about product X last month.” Semantic memory: Facts and general knowledge—like “product X costs $100.” Procedural memory: Skills and routines, such as “how to troubleshoot a common issue.”

ALEX: 07:45

These memories are stored using different technologies. Vector databases enable efficient semantic search by converting memories into numerical “embeddings” — basically, a way for the AI to find relevant info based on meaning rather than keywords.

MORGAN: 07:58

So it’s like the AI’s brain translating memories into a language it can search quickly?

ALEX: 08:02

Exactly. Other systems use knowledge graphs to map relationships and temporal metadata to track how memories change over time. Mem0, for instance, focuses on a unified vector store optimized for performance — it uses clever indexing and memory curation strategies like deduplication and summarization to keep things lean. LangMem separates memory types and implements procedural memory to allow agents to learn new behaviors, which is a big step towards more autonomous AI. Zep with Graphiti adds a temporal dimension — think of it as the AI’s diary, noting not just facts but when and how they changed.

ALEX: 08:33

Keith, your book has extensive code labs on these mechanics — what’s the one thing you want readers to really internalize here?

KEITH: 08:38

It’s that agentic memory isn’t just about storing data; it’s about designing how AI thinks over time. Understanding these different memory types and how they interact is key to building agents that adapt meaningfully rather than just regurgitating static info. The labs walk through how to implement these concepts practically, which really helps internalize the architecture.

ALEX: 08:58

That perspective is vital. It’s about building AI that learns like we do — not just recalling but evolving.

ALEX: 09:04

Let’s talk numbers, because what really matters to leadership is the impact. Mem0 boasts a 26% accuracy improvement over standard OpenAI memory approaches, with a whopping 90% reduction in latency and token consumption. That’s huge — faster responses, lower costs, and better answers.

MORGAN: 09:22

Wow, 90% reduction in latency? That’s a game changer for real-time applications.

ALEX: 09:26

Absolutely. Zep with Graphiti hits 94.8% accuracy on deep memory retrieval tests and improves long-memory evaluation by 18.5%, while also cutting latency by 90%.

CASEY: 09:37

So these aren’t just incremental gains — they’re leaps forward in both performance and efficiency.

ALEX: 09:42

Exactly. The practical effect? More coherent conversations, better personalization, and AI that actually learns from history instead of starting fresh each time.

MORGAN: 09:51

That translates directly into better customer experiences and operational efficiency — a clear ROI.

ALEX: 09:56

Right, but it’s also worth noting these gains come with complexity and the need for ongoing maintenance, which we’ll touch on next.

CASEY: 10:03

Speaking of which, let’s get real. What can go wrong with agentic memory?

MORGAN: 10:07

I’m bracing for some tough truths here.

CASEY: 10:10

Implementing these systems is complex — you need high-quality data capture pipelines and constant maintenance to prevent “memory bloat,” where irrelevant or outdated memories clutter the system, degrading performance.

JORDAN: 10:21

Memory sclerosis is another issue — where memories become stale or biased over time, leading to poor decisions.

CASEY: 10:28

Procedural memory, which is key for adaptive behavior, is still less commonly supported and can be tricky to get right.

MORGAN: 10:35

What about privacy? Storing personal memories sounds like a regulatory minefield.

CASEY: 10:39

Exactly. Data isolation, GDPR compliance, and user consent are critical challenges that can’t be overlooked.

MORGAN: 10:46

Keith, from your consulting experience, what’s the biggest mistake organizations make with agentic memory?

KEITH: 10:50

The biggest pitfall is treating it as a “set and forget” system. These memory architectures require ongoing curation — pruning irrelevant memories, updating models, and balancing privacy with personalization. Neglect this, and you risk degraded performance and compliance issues. Also, rushing implementation without clear business goals leads to wasted effort.

CASEY: 11:10

So thoughtful strategy and continuous investment are non-negotiable.

KEITH: 11:14

Absolutely. The RAG book is honest about these limitations because we want leaders to be fully prepared.

SAM: 11:21

Let’s bring this home with some real-world examples. In finance, some advisory firms use agentic memory to track client portfolios and risk tolerance over years. This enables advisors to deliver personalized recommendations without making clients repeat themselves every time.

MORGAN: 11:37

That’s a real customer loyalty win.

SAM: 11:39

In healthcare, virtual assistants remember patient symptoms, treatments, and preferences longitudinally, helping clinicians get a fuller picture without sifting through piles of records.

CASEY: 11:48

That must improve care continuity and reduce errors.

SAM: 11:51

Exactly. Tech support bots build memory from past debugging conversations, so they get smarter at troubleshooting and reduce repeat escalations.

JORDAN: 11:59

And in education, AI tutors adapt to students’ learning styles and progress over time, personalizing lesson plans dynamically.

SAM: 12:05

Across industries, agentic memory is a key differentiator for companies that want AI to truly become a strategic partner rather than just a tool.

SAM: 12:12

Now, let’s stir up some debate. Imagine you’re building an AI customer support agent for a regulated financial services firm that needs audit trails and evolving compliance rules.

MORGAN: 12:22

I’d lean into Zep with Graphiti. The temporal knowledge graph’s ability to track changes over time and provide auditability is a must-have in finance.

CASEY: 12:31

But what about Mem0? Its simplicity and performance could mean faster deployments and lower operational overhead.

TAYLOR: 12:38

True, but Mem0 doesn’t natively support procedural memory or temporal reasoning, which could limit adaptability in a complex domain like finance.

ALEX: 12:46

LangMem’s explicit memory types and procedural capabilities might offer a middle ground — enabling the agent to learn compliance routines while keeping memory structured.

SAM: 12:55

So it boils down to a trade-off between auditability and complexity. Zep/Graphiti for compliance-heavy needs; Mem0 for rapid, unified memory; LangMem when procedural learning is critical.

MORGAN: 13:07

Could a hybrid approach work?

TAYLOR: 13:09

Absolutely. Combining Zep’s temporal modeling with LangMem’s procedural memory could cover all bases — but that comes with integration complexity.

SAM: 13:17

Leaders need to weigh these trade-offs carefully based on product goals, regulatory needs, and team capabilities.

SAM: 13:23

Before we wrap, some quick tips for leaders considering agentic memory. Start with a clear memory strategy—define what types of memory your AI really needs based on use case.

MORGAN: 13:34

Avoid the temptation to store everything—implement memory curation like deduplication and decay to keep the system efficient.

CASEY: 13:41

Use semantic similarity search with vector databases for fast, meaningful retrieval — it’s way smarter than keyword matching.

JORDAN: 13:47

Balance personal and community memory carefully: personal memories drive customization, while community memories capture shared knowledge.

ALEX: 13:53

And build robust memory extraction pipelines that translate raw data into structured memories—this often gets overlooked but is foundational.

SAM: 14:00

These patterns help ensure your agentic memory system scales and delivers value without becoming a maintenance nightmare.

MORGAN: 14:07

Just a quick reminder: we’re giving you the highlights here, but Keith’s book goes much deeper — with detailed diagrams, thorough explanations, and hands-on code labs that let you build these systems yourself. Search for Keith Bourne on Amazon and grab the second edition of Unlocking Data with Generative AI and RAG.

MORGAN: 14:27

This podcast is produced by Memriq AI — an AI consultancy and content studio building tools and resources for AI practitioners.

CASEY: 14:33

Memriq helps engineers and leaders stay current with the rapidly evolving AI landscape. Head to Memriq.ai for more deep-dives, practical guides, and cutting-edge research breakdowns.

SAM: 14:42

Looking ahead, there are still some big challenges. Scaling memory systems without slowing down as data grows exponentially is a major engineering hurdle.

TAYLOR: 14:51

Developing adaptive procedural memory that reliably evolves agent behavior remains an open frontier.

CASEY: 14:57

And the tension between personalization and privacy compliance is only going to get tougher, especially with new regulations emerging.

ALEX: 15:03

Plus, we need better evaluation metrics to truly measure long-term learning and behavioral adaptation — current tests often miss the bigger picture.

SAM: 15:10

Leaders should watch these areas closely and plan investments accordingly to stay ahead.

MORGAN: 15:17

My key takeaway? Agentic memory is less a feature and more a strategic transformation — AI that remembers is AI that builds trust.

CASEY: 15:25

For me, the biggest lesson is that complexity is real here — but with clear focus and ongoing curation, the payoff can be enormous.

JORDAN: 15:31

I’m excited by how agentic memory brings AI closer to human-like relationships, unlocking richer, more personal experiences.

TAYLOR: 15:37

The CoALA framework is a game-changer — it gives leaders a clear mental model to evaluate memory architectures.

ALEX: 15:43

Remember, it’s not just about storage — it’s about designing how AI thinks and learns over time.

SAM: 15:49

Real-world deployments show this isn’t theoretical — agentic memory is driving differentiation right now.

KEITH: 15:54

As the author, the one thing I hope you take away is that integrating agentic memory thoughtfully isn’t just a technical challenge — it’s a leadership imperative to unlock AI’s true potential for your business.

MORGAN: 16:07

Keith, thanks for giving us the inside scoop today.

KEITH: 16:10

My pleasure — I hope this inspires you to dig into the book and build something amazing with agentic memory.

CASEY: 16:17

And thanks to everyone listening. Remember, we covered key concepts today, but Keith’s book goes much deeper — detailed diagrams, thorough explanations, and hands-on code labs that let you build this stuff yourself. Search for Keith Bourne on Amazon and grab the second edition of Unlocking Data with Generative AI and RAG.

MORGAN: 16:34

Thanks for tuning in to Memriq Inference Digest - Leadership Edition. Until next time!