Adding Memory to an AI Agent

Distinguish four kinds of continuity

Working context contains the messages and tool observations needed for the immediate turn. Workflow state records durable facts such as which approval is pending. Episodic history records past interactions or outcomes. Semantic memory stores selected facts or preferences, such as a user's chosen reporting format. Calling all four chat history hides important differences.

Workflow state should be explicit and authoritative. A payment status belongs in a transaction record, not in a model-written summary of the conversation. Preferences can be softer and user-editable. Past interaction text is evidence, not automatically a current fact. Design each store around its purpose instead of putting every message into a vector index and retrieving it indefinitely.

Decide what may be remembered

Create a memory policy before creating a save-memory tool. Define allowed categories, prohibited sensitive material, who can create a memory, and whether consent is required. A user saying 'use bullet points for my weekly summaries' is a plausible preference. A temporary medical detail mentioned during an unrelated task should not become a durable profile by default.

Every record needs provenance and scope. Save the source interaction identifier, exact supporting text or event, owner, timestamp, confidence or verification state, intended uses, and expiration. Keep tenant and user boundaries in storage queries. Do not ask the model to enforce privacy through instructions when the database can enforce access structurally.

Retrieve memory selectively

Memory should enter context only when it helps the current task. Filter by user, permitted purpose, active status, and relevant category before semantic ranking. Limit the number of items and label them as prior information that may be stale. Current explicit instructions should override an old preference unless policy says otherwise.

Resolve conflicts visibly. If two records contain different preferred names or destinations, do not select one based only on similarity. Prefer verified recency where appropriate or ask the user. A memory that affects a consequential action should be confirmed against the source system. Personalization is not permission to infer unstated traits or make decisions about a person.

Provide control and maintenance

Users should be able to see meaningful durable memories, correct them, remove them, and disable optional memory. Deletion must remove derived index entries and caches, not just hide a row in the interface. Expiration jobs should be observable, and backups need a documented retention path consistent with the product's commitments.

Evaluate memory with paired tasks: one where the saved fact should improve the result and one where it should not be used. Test stale, conflicting, maliciously planted, cross-user, and deleted records. Measure inappropriate recall as well as helpful recall. Check current provider storage behavior and APIs quarterly, but keep ownership and lifecycle rules in your own application contract.

Remember a report preference safely

A manager asks an assistant to use a five-row risk table for future weekly project summaries.

1. Classify the request as an optional presentation preference, not authoritative project state, and show what will be saved.
2. Store the table preference with the manager's identity, source request, intended weekly-summary scope, and review date.
3. On a later summary task, retrieve only active preferences for that manager and purpose, then apply current task instructions first.
4. Expose a control to edit or delete the preference and remove its searchable representation when deletion occurs.
5. Test that the preference appears in weekly summaries but does not alter an unrelated incident report with its own required format.

Memory record contract

Require these fields before any durable fact or preference can influence later work.

• Content and category: exact remembered item plus whether it is state, preference, history, or verified fact.
• Provenance: source event, creator, verification status, and time recorded.
• Scope: owner, tenant, permitted purposes, sensitivity, and systems allowed to read it.
• Lifecycle: expiration, review, correction, deletion, index cleanup, and backup handling.
• Use rule: retrieval conditions, conflict behavior, override priority, and confirmation requirement.

Common mistakes

• Treating the entire conversation archive as trustworthy long-term memory without purpose or expiration.
• Saving a model inference about the user as a fact without consent, evidence, or correction controls.
• Deleting the visible memory record while leaving vectors, caches, or derived summaries available for retrieval.

Try one

Sources

• OpenAI agents guideOfficial guidance for building agent workflows with current OpenAI platform capabilities.
• OpenAI safety best practicesOfficial recommendations for testing, constraints, and human oversight.