Enhancing LLM Conversations with Persistent Knowledge

LLMs are amazing tools. As soon as I started “just talking to them” like I was having a real conversation, their capabilities really opened up. But, one thing was always a pain. Every conversation was starting over at the beginning. You have a great conversation about your project one day, and the next day - nothing. It’s gone. You have to explain everything all over again.

This limitation was driving me crazy. AI conversations are productive in the moment, but the knowledge vanishes when the chat ends. It’s like working with someone who has perfect general knowledge but forgets everything about your specific situation each time you talk.

The Problem with Forgetting

I’ve seen this play out over and over:

You have a detailed conversation with an AI about your research project
You get valuable insights and make progress together
You come back the next day to continue
The AI has no memory of what you discussed

This pattern makes it nearly impossible to build on previous context. You end up repeating yourself, manually copying information from old conversations, or trying to remember where that one key insight is buried in your chat history.

In my own work, I found myself copying snippets from previous conversations just to maintain continuity. I knew there had to be a better way.

Building a Memory Layer

Basic Memory started as a simple idea around Christmas 2024: what if conversations with AI could build lasting knowledge structures instead of disappearing? We designed a system where:

Key information from conversations gets captured in a knowledge graph
This graph maintains connections between related topics
When you start a new conversation, the AI can navigate this graph to find relevant context

The result? AIs that remember what you’ve discussed before and build on that knowledge over time.

How It Actually Works

Let me walk through a concrete example. During a conversation about coffee brewing, I mentioned Ethiopian beans. The system created a structured note containing:

# Ethiopian Coffee Beans

## Observations
- [origin] Grown primarily in Yirgacheffe, Sidamo, and Harrar regions
- [flavor] Known for bright acidity and floral, fruity notes
- [processing] Natural processing common, contributing to berry flavors

## Relations
- pairs_well_with [[Pour Over Brewing]]
- contrasts_with [[Dark Roast Methods]]

A week later, when discussing brewing methods, the AI was able to suggest pour over specifically for my Ethiopian beans, referencing their floral characteristics from our previous conversation. It wasn’t starting from scratch - it had access to context from across our conversation history.

The Real-World Difference

Using this system has changed how I work with AI in several ways:

Conversations build on each other: Each interaction contributes to a growing knowledge base
Context flows naturally: The AI can find relevant information without me explicitly providing it
Ownership of insights: Everything is captured in a format I control, not lost in cloud services
More productive interactions: Less time spent re-explaining, more time making progress

The shift from ephemeral to persistent conversations makes AI feel more like a continuous collaborator than a stateless tool.

What’s Next

This is still early work. There are challenges in determining what information should be captured, how to navigate large knowledge graphs efficiently, and how to handle conflicting or outdated information.

But the core concept - that AI conversations should contribute to durable knowledge rather than disappear - has proven incredibly valuable. At Basic Machines, we’re continuing to refine how information flows between humans and AI systems, always with the principle that your knowledge should remain in your control.