Memory Architecture for a Conversational AI: Profiles, Observation Logs, and Derived Beliefs

The hard part of giving a conversational AI a memory is not storage. It is stopping the model from amplifying its own guesses until the customer’s profile is a distorted picture nobody agreed to. The design below, from an AI concierge project, splits memory into three tiers that are deliberately not allowed to contaminate each other.

Tier 1: the durable profile

A small, mutable record of facts that are stated, not inferred. First name, delivery area, member since, contact preferences, and hard boundaries the customer has explicitly set. It is written directly, only on a clear signal, and it changes rarely. Because nothing here is interpreted, the drift risk is close to zero.

Tier 2: the observation log

An append-only, event-sourced list of everything the system has noticed. It is never edited and never deleted. Each entry records a type such as preference, outcome, boundary, context, or style, plus the subject, the value, a verbatim quote as evidence, the source of the observation, a confidence level, and a timestamp. If the customer says a previous purchase was too strong, that becomes one immutable record with their exact words attached. The log is the ground truth.

Tier 3: the derived belief summary

A compact set of fixed belief slots: things like preferred format, potency ceiling, flavours, what to avoid, and conversational style. Each slot has a value, a confidence, and a list of the observation record IDs that support it. This summary is regenerated from the log by a language-model pass. It is never edited in place, and it is never derived from the previous summary, only ever from the raw log. That single rule is what stops the model from compounding its own earlier interpretation.

Deterministic routing between tiers

A distiller step tags each new observation with its type. A plain rule, not a model, decides what happens next: explicitly stated facts and hard boundaries get promoted into the durable profile; everything else stays in the log and only ever influences the profile indirectly, through the regenerated summary. The model never writes directly to the durable profile.

No citation, no assertion

A belief slot with no supporting record IDs does not get stated to the user or used in a recommendation. If the system cannot point at the evidence, it does not act on the belief. That makes the whole memory auditable: every claim the assistant makes about a customer can be traced to something they actually said or did.

Why the separation is worth the effort

• The log is operationally clean, because it is verbatim and immutable.
• The profile is safe to trust, because only stated facts reach it.
• The summary can be thrown away and rebuilt at any time, because the log still holds everything.
• Drift is contained, because interpretation never feeds on interpretation.

Related