Procedural Memory for RAG (Chapter 18)

MEMRIQ INFERENCE DIGEST - LEADERSHIP EDITION Episode: Procedural Memory for RAG: Chapter 18 Deep Dive with Keith Bourne

MORGAN: 00:00

Welcome to Memriq Inference Digest – Leadership Edition. I’m Morgan, and this podcast is brought to you by Memriq AI, a content studio creating tools and resources for AI practitioners. Check them out at Memriq.ai.

CASEY: 00:15

Today, we’re diving into an exciting topic from Chapter 18 of *Unlocking Data with Generative AI and RAG* by Keith Bourne. We’ll explore how procedural memory empowers Retrieval-Augmented Generation, or RAG, through a tool called LangMem. It’s about turning AI agents from static responders into autonomous learners that improve themselves over time.

MORGAN: 00:35

And Keith Bourne himself is here with us! Keith, welcome. We’re really looking forward to hearing your insider insights and some behind-the-scenes thinking on this groundbreaking concept.

KEITH: 00:45

Thanks, Morgan and Casey. Really pleased to join you all and share how procedural memory is reshaping AI interactions.

CASEY: 00:52

We’ll unpack what procedural memory means in practical terms, why it matters now, and how LangMem and related frameworks bring this to life in business settings. Plus, we’ll cover real-world applications, challenges, and what leaders need to know before investing.

MORGAN: 01:05

If you want to go deeper than today’s overview—with detailed diagrams, thorough explanations, and hands-on code labs—search for Keith Bourne on Amazon and grab the second edition of *Unlocking Data with Generative AI and RAG.* The book goes much deeper than we can today.

MORGAN: 01:20

Okay, let’s get started.

JORDAN: 01:23

I want to kick off with something that really surprised me about procedural memory and LangMem. It’s that this approach actually lets AI agents *self-heal.* Imagine an AI that detects when it messes up and then corrects its own behavior—without a human stepping in.

MORGAN: 01:38

Hold on—that’s huge. So the AI learns not just from what you feed it upfront but from every interaction it has afterwards?

JORDAN: 01:45

Exactly. Every conversation, every user question becomes training data. The agent compounds its own performance over time, improving success rates while cutting down the manual effort needed to maintain it.

CASEY: 01:58

That sounds like a dream for scaling customer service or advisory bots. But I’m curious—how reliable is this self-correction? Does it really save operational costs?

JORDAN: 02:05

According to the data in the book, success rates jump significantly, and operational costs drop as agents become better at anticipating problems and adjusting strategies on the fly.

MORGAN: 02:15

Wow. That’s not just incremental improvement; it’s a strategic game-changer. Instead of one-and-done deployment, the AI gets smarter and more autonomous continuously.

KEITH: 02:23

That’s right. When I wrote that chapter, my goal was to showcase how procedural memory unlocks that autonomous learning. It’s a shift from static AI models to ones that evolve intelligently, spotting failures and refining their playbook themselves.

CASEY: 02:36

I like the idea, but I’m already thinking about the risks of letting AI “self-heal.” What if it learns the wrong lesson?

JORDAN: 02:42

Great point. We’ll dive into those risks later. But for now, just imagine the competitive advantage of AI that uncovers customer insights automatically, saving companies from endless manual tuning.

MORGAN: 02:53

That image sticks with me — AI as an evolving team player, not a fixed tool.

CASEY: 02:58

Here’s the one-sentence essence for busy leaders: Procedural memory with LangMem enables AI agents to learn from interactions, adapt behavior, and continuously improve retrieval-augmented systems.

MORGAN: 03:07

And the key players are LangMem itself, which manages the procedural memory, LangChain for orchestrating AI workflows, the CoALA framework for modular architecture, and OpenAI’s GPT models powering the natural language understanding.

CASEY: 03:20

If you remember nothing else: procedural memory turns AI from static responders into adaptive learners that get better with every conversation, making RAG systems smarter and more autonomous.

JORDAN: 03:30

Let’s set the stage. Traditionally, AI agents have been pretty static—they rely on fixed knowledge bases or pre-trained models updated manually. That’s a bottleneck for scalability and personalization.

MORGAN: 03:39

Right, static AI means every time you want to improve or adapt, your team has to step in with updates. That’s slow and costly, especially with complex or rapidly changing domains.

JORDAN: 03:48

Exactly. But business environments now demand AI that personalizes experiences and self-improves without constant human oversight. Customers expect smarter, faster, more relevant interactions every time.

CASEY: 03:58

So what changed recently that makes procedural memory viable?

JORDAN: 04:02

A few things. Advances in frameworks like LangChain and LangMem provide modular, scalable architectures. Plus, the maturity of GPT models means AI understands language better, enabling it to detect subtle failures and learn from them. Also, companies are recognizing that continuous learning isn’t a nice-to-have; it’s essential for competitive differentiation.

MORGAN: 04:20

And who’s actually adopting this tech? Is it just the big tech players?

JORDAN: 04:24

Not at all. Early adopters include investment advisory firms personalizing portfolio strategies, healthcare providers optimizing patient care, and customer service teams reducing escalations. They’re tapping into procedural memory to cut operational costs and improve customer satisfaction simultaneously.

CASEY: 04:38

That’s a powerful combination. Faster time-to-resolution and lower maintenance overhead—two metrics every leader wants to see improving.

TAYLOR: 04:45

Let’s unpack the fundamental idea behind procedural memory. Unlike traditional AI memory, which might just store facts or past conversations, procedural memory stores *behavioral patterns* and strategies. That means it remembers *how* to act, not just *what*.

MORGAN: 04:58

So it’s like the difference between knowing the facts about a customer and knowing the best way to handle their particular problem or personality.

TAYLOR: 05:04

Exactly. Procedural memory enables AI to apply the right strategy at the right level—whether that’s for an individual user, a community segment, a specific task, or even globally across all interactions. It’s hierarchical.

KEITH: 05:15

That’s a key point. I wanted readers to see that AI behavior isn’t one-size-fits-all. LangMem, for example, organizes its learning into these scopes—user level, community level, task level, and global. This modularity ensures the AI tailors responses appropriately and learns across different dimensions.

TAYLOR: 05:30

And the CoALA framework supports this by enabling domain-agnostic procedural memories to plug into specific agents. So whether it’s healthcare or finance, the architecture remains consistent, but the learned strategies adapt to domain specifics.

CASEY: 05:43

That modularity sounds like a smart design to avoid reinventing the wheel for every application.

TAYLOR: 05:47

It is. And it creates a feedback loop where agents learn from what works and what doesn’t, continuously optimizing future interactions—making AI smarter in a way that aligns with business goals.

MORGAN: 05:57

Keith, as the author, why was it important to cover procedural memory so early in the book?

KEITH: 06:02

Because it represents a paradigm shift. Early chapters build the foundation with RAG basics, but procedural memory is where things leap forward—from static retrieval to dynamic learning. I wanted leaders to grasp that this is where future-proof AI systems are headed.

TAYLOR: 06:15

Now, let’s compare procedural memory with other approaches. Traditional RAG systems—think of them as search engines—pull documents or data based on keyword matches or semantic similarity, which means “meaning-based” matching.

CASEY: 06:26

So traditional RAG helps AI find relevant info but doesn’t learn from whether the answer worked or not.

TAYLOR: 06:31

Exactly. Then there are memory-enhanced RAG systems that add episodic memory—remembering specific past events—and semantic memory—understanding concepts over time. But these still don’t adapt behavior based on outcomes.

CASEY: 06:41

That’s where procedural memory comes in?

TAYLOR: 06:43

Yes. Procedural memory adds a meta-learning layer that *refines the agent’s behavior* based on real-world success and failures. It learns which strategies yield the best results and adjusts accordingly.

MORGAN: 06:53

Can you give an example of when to use one over the other?

TAYLOR: 06:56

Sure. Use traditional RAG when you need straightforward document lookup without behavior adaptation—like static FAQs. Episodic or semantic memory helps when you want the AI to remember past conversations or concepts but don’t need behavior changes—for example, a customer chatbot recalling prior orders. Procedural memory, like LangMem, shines when you want *autonomy, continuous improvement,* and *personalized strategies* across users and tasks—think complex advisory agents or healthcare assistants.

CASEY: 07:19

What about downsides?

TAYLOR: 07:21

Procedural memory systems require more upfront investment in design and metrics, and careful governance to avoid learning the wrong lessons. But the payoff can be big.

ALEX: 07:29

Let’s take a closer look at how procedural memory actually works in LangMem. No code, just the big picture.

MORGAN: 07:34

This is where I get excited.

ALEX: 07:36

The system starts by interfacing with a domain-specific agent—say, an investment advisor bot. LangMem doesn’t hardcode domain logic; instead, it captures patterns from conversations as *readable procedures*—think step-by-step strategies. These strategies are organized into hierarchical scopes: individual user, community segments, task types, and global strategies. This lets the AI apply the right approach depending on context.

CASEY: 07:58

Why is that hierarchy important?

ALEX: 08:00

Because a one-size-fits-all strategy often fails. For example, a user-level strategy might adapt to a single customer’s preferences, while a community-level strategy captures trends across a market segment. The AI can choose or combine these depending on the situation.

MORGAN: 08:12

How does the system know if a strategy worked?

ALEX: 08:14

That’s where *success metrics* come in. The AI tracks outcomes—like whether a customer’s question was resolved, or if an investment recommendation met a target. Based on these metrics, the system updates confidence scores for each strategy. Critically, LangMem maintains a *feedback loop* that adjusts these scores, favoring strategies with higher success and rolling back ones that degrade performance. This momentum-based update ensures stable evolution.

KEITH: 08:38

Alex, you nailed the core. When I designed the code labs for the book, my focus was to let readers internalize how to define meaningful success metrics and implement feedback loops safely. That’s the heart of adaptive AI.

ALEX: 08:50

Keith, what’s the one thing you want readers to really take away from those code labs?

KEITH: 08:53

It’s that designing success metrics aligned with business goals isn’t trivial—but it’s essential. Without the right metrics, the AI can optimize the wrong objectives, which is risky. The labs walk through how to structure metrics and implement safe rollbacks—practices that are often overlooked but critical for trust and reliability.

ALEX: 09:10

The numbers from LangMem deployments are impressive. From just a couple of interactions, the system learns multiple strategies across user, community, task, and global scopes. For example, it might learn eight distinct strategies from two initial sessions.

MORGAN: 09:22

That’s efficiency I can get behind. Learning that much from so little input?

ALEX: 09:25

Yes, and hierarchical retrieval ensures the system personalizes responses effectively. Success rates improve from an initial 75% to over 85%, sometimes even hitting 90%, all while reducing errors and customer frustration.

CASEY: 09:36

And that translates to real business impact?

ALEX: 09:38

Absolutely. Reduced operational costs due to fewer manual updates and better-first-contact resolution lead to faster time-to-resolution and higher customer satisfaction scores. The confidence scores help prioritize strategies, so the AI focuses on what’s proven effective.

MORGAN: 09:50

That’s a big win for any customer-facing operation or advisory service.

ALEX: 09:53

The downside? Initial setup requires thoughtful design and domain expertise, but once running, these self-improving agents deliver measurable ROI.

CASEY: 10:01

Now, let’s bring a dose of reality. Procedural memory systems sound promising, but what can go wrong?

MORGAN: 10:05

I’m curious what you found.

CASEY: 10:07

First, defining success metrics is tricky. If the metrics don’t align perfectly with business goals, the AI might optimize for the wrong outcomes—leading to degraded performance or unintended behaviors.

JORDAN: 10:16

And continuous learning systems can overfit—meaning they might get too narrowly focused on specific cases or noisy data, losing generality.

CASEY: 10:23

Right. That’s why rollback mechanisms are critical. If the AI’s strategy declines, you need a safe way to revert to prior versions.

KEITH: 10:30

Casey, you’re raising exactly the concerns I wanted to highlight in the book. In my consulting, I often see teams rushing into adaptive AI without robust metric design or rollback plans. That’s the biggest mistake.

CASEY: 10:39

What’s your advice for leaders to avoid these pitfalls?

KEITH: 10:42

Invest time upfront to define clear, multi-dimensional success metrics. Collaborate closely between domain experts and data scientists. And implement rigorous testing—like “needle in a haystack” tests, where you hide specific facts in large documents to ensure the AI can still find them and not hallucinate false info. Transparency and auditability are essential for trust.

CASEY: 11:00

That honesty is refreshing. Procedural memory isn’t a magic bullet but a powerful tool that requires careful governance.

SAM: 11:06

Let’s look at real-world applications. In investment advisory, procedural memory agents learn personalized portfolio strategies based on client preferences and market behavior, improving recommendation success and client retention.

MORGAN: 11:17

That personalization is huge in finance.

SAM: 11:19

Healthcare is another domain. AI assistants optimize patient care pathways and appointment scheduling by adapting to individual needs and systemic patterns, leading to better outcomes and reduced no-shows.

CASEY: 11:29

What about customer service?

SAM: 11:31

Procedural memory helps bots refine ticket resolution flows, reducing escalations and improving first-contact resolution rates. Similarly, in education, tutors adapt teaching styles dynamically to learner comprehension levels, enhancing engagement.

MORGAN: 11:43

So across industries, the common thread is adaptive, context-aware AI that continuously refines its approach.

SAM: 11:48

Exactly. It’s competitive advantage through smarter, more efficient AI that's less dependent on ongoing manual tuning.

SAM: 11:53

Picture this scenario: A company must choose between traditional RAG, memory-enhanced RAG, or procedural memory-enabled agents for their customer support.

MORGAN: 12:00

Traditional RAG’s strength is simplicity—basic retrieval of documents or FAQs with minimal complexity.

CASEY: 12:06

But it won’t learn or adapt behavior, so the company will need constant manual updates.

TAYLOR: 12:11

Memory-enhanced RAG adds context—remembering past conversations or concepts—which improves continuity but still lacks behavioral refinement.

ALEX: 12:18

Procedural memory agents deliver autonomy, continuously improving and personalizing strategies, but with higher upfront design and monitoring costs.

CASEY: 12:26

So if budget is tight and the domain is straightforward, traditional RAG might suffice. But for complex, dynamic environments, procedural memory wins.

MORGAN: 12:34

The trade-off is between short-term ease and long-term scalability and ROI.

SAM: 12:38

Exactly. Procedural memory justifies its higher initial investment through reduced maintenance costs and improved customer satisfaction down the line.

SAM: 12:45

For leaders ready to embrace procedural memory, here are practical tips. First, use modular domain interfaces to separate your domain logic from the core procedural memory system. This keeps things flexible.

TAYLOR: 12:56

That modularity allows you to swap in new domain models without redesigning your entire memory setup.

SAM: 13:01

Second, implement hierarchical scopes—user, community, task, global—to target learning and retrieval effectively.

ALEX: 13:07

And define clear success metrics that align with your business goals. Without these, the system can’t learn properly.

SAM: 13:12

Third, leverage feedback loops with momentum-based updates. This means your AI remembers not just the last interaction but the trend over time.

CASEY: 13:20

Don’t forget to build in safe rollback mechanisms to mitigate risk from degraded strategies.

MORGAN: 13:25

Those are practical guardrails for deploying adaptive AI with confidence.

MORGAN: 13:29

If you want to go deeper on all this, *Unlocking Data with Generative AI and RAG* by Keith Bourne is a must-read. It’s full of detailed illustrations, thorough explanations, and hands-on code labs that walk you through building procedural memory systems step by step. Search for Keith Bourne on Amazon and grab the second edition.

MORGAN: 13:48

Quick reminder – this podcast is produced by Memriq AI, an AI consultancy and content studio building tools and resources for AI practitioners. For more AI deep-dives, practical guides, and cutting-edge research breakdowns, head to Memriq.ai.

CASEY: 14:00

Memriq helps engineers and leaders stay current with the rapidly evolving AI landscape.

SAM: 14:05

Procedural memory is powerful, but challenges remain.

MORGAN: 14:08

Like what?

SAM: 14:09

Designing domain metrics that balance competing business goals is complex. You might want to optimize for speed but also accuracy, and those can conflict.

CASEY: 14:16

Scaling procedural memory systems to handle millions of users or multiple domains requires robust infrastructure.

SAM: 14:21

Transparency and auditability are crucial to maintain trust, especially in regulated industries. Autonomous learning must be explainable.

KEITH: 14:28

And adapting to rapidly changing environments without losing valuable learned knowledge—what I call “catastrophic forgetting”—is a key research frontier.

MORGAN: 14:37

So for leaders, the field is ripe with opportunity but demands strategic vision and resource investment.

MORGAN: 14:44

My takeaway: Procedural memory transforms AI agents into dynamic learners, unlocking continuous improvement and strategic advantage.

CASEY: 14:50

I’d emphasize the need for strong governance—metrics, monitoring, and rollback—to avoid costly mistakes.

JORDAN: 14:56

For me, it’s the idea that AI can self-heal and uncover customer insights automatically—huge potential to save time and money.

TAYLOR: 15:02

The hierarchical design is brilliant—scaling personalization from individuals to communities.

ALEX: 15:07

The feedback loops and momentum updates are the clever levers that make procedural memory reliable and effective.

SAM: 15:13

Real-world applications show this isn’t theoretical—it delivers measurable ROI across finance, healthcare, customer service, and education.

KEITH: 15:20

As the author, the one thing I hope you take away is this: investing in procedural memory-enabled AI is investing in future-proof systems that grow smarter with use—building competitive advantage that compounds over time.

MORGAN: 15:33

Keith, thanks so much for giving us the inside scoop today.

KEITH: 15:36

My pleasure. I hope this inspires you to dig into the book and build something amazing.

CASEY: 15:41

And remember, adaptive AI is powerful—but only if you design, monitor, and govern it carefully.

MORGAN: 15:47

We covered the key concepts today, but the book goes much deeper—with detailed diagrams, thorough explanations, and hands-on code labs that let you build this yourself. Search Keith Bourne on Amazon and get the second edition of *Unlocking Data with Generative AI and RAG.*

MORGAN: 16:05

Thanks for listening. See you next time on Memriq Inference Digest – Leadership Edition.