---
title: "Two Writers, One Index: How Static Files Corrupt Agent Memory"
date: "August 12, 2026"
description: "Two subagents wrote one record and the second write erased the first 80 ms later. I replaced my markdown summary index with Firestore transactions on Cloud Run."
category: "Agent Architecture"
canonical: "https://ulukaya.dev/posts/ai-agent-split-brain-trap"
---

# Two Writers, One Index: How Static Files Corrupt Agent Memory

I sent [two subagents](#lab-split-brain) to update one shared Firestore user profile at the same time. Both subagents read version N of the document in the same millisecond. Subagent A saved its change. 80 ms later, Subagent B saved its own copy on top. B never saw A's change, so A's work was gone, and no error showed. This is a classic distributed split-brain race condition.

		

Agent memory in flat markdown files breaks the same way, and so does any memory that agents write with no coordination. In production, it is sure to corrupt state. In long sessions with many turns, file-based memory also drifts away from the real state of each record. Then my model reasons over stale summaries and hallucinates. I call that a split-brain condition, too.

		

As I explored in [Part 1: The 10 Cognitive Biases of Autonomous Systems](/posts/ten-cognitive-biases-ai-agents), memory drift in long-running agents is rarely an LLM prompt failure. I found it is a dual-write cache invalidation breakdown. Its cause is treating language models as database engines. To end the drift, I need ACID transaction boundaries and an atomic lock per session in a transactional store. I use Firestore transactions for mine.

		

> The distributed systems paradox: I often see engineers treat agent memory drift as a prompting defect. They try to fix the hallucinations with longer system prompts. In my production systems, memory drift in multi-turn agents is a dual-write cache invalidation failure. Its cause is that the language model got the database's indexing job.

	

	
	

## The static index anti-pattern and markdown storage

	

To keep my context windows small and save input tokens, I first tried a two-tier storage pattern:

	
		- **Tier 1 (granular entity files):** One file per record, with its full detail, such as `contacts/alice.md` or `tasks/task-402.json`.

		- **Tier 2 (the summary index):** One high-level markdown file (such as `INDEX.md` or `SUMMARY.md`) with a table that sums up the active records, their priorities, and their statuses.

	

	
		

### 01. The decoupling threshold (turn 20+)

		

Every time my agent takes an action that changes state, it must do a dual-write. First it updates the entity file. Then it parses the summary table in the index file and updates that, too.

		

A language model edits a file by probability. Nothing makes the edit a transaction. So under load, dual-writes fail. My model updates the entity file but skips the index table. Or it rewrites the table with slightly changed column headers.

		

On turn 25, my agent checks its overall status to choose its next step. To save tokens, it reads the shorter summary index. That index holds stale, uncommitted state. My agent treats it as ground truth and enters a hallucination loop that it cannot recover from.

		

I enforce strict relational consistency for **deterministic operational state**: tasks, status queues, assignments, and tool locks. Associative episodic memory, such as user preferences and the nuances of a conversation, benefits from vector search embeddings. My operational coordination layer still requires ACID transactions.

	

	
	

## Failure anatomy: The three concurrency and state traps

	
		

### 02. Trap 1: Concurrency collisions and lost updates

		

Flat files on a local filesystem have no built-in locking. Sometimes my agent [spawns parallel subagents](#lab-split-brain), or runs a background heartbeat while an interactive session is active. Then two processes try to write to `tasks.md` at the same time.

		

Without atomic row-level locks, the operating system runs those writes in an unpredictable order. Foundational distributed systems work by [Leslie Lamport (1978)](https://amturing.acm.org/p558-lamport.pdf) and [Bernstein & Goodman (1981)](https://people.eecs.berkeley.edu/~brewer/cs262/concurrency-distributed-databases.pdf) established the result. Concurrent writes with no agreed order are sure to lose updates. The last process to finish silently overwrites the earlier changes, and no error is raised. That is a lost update. In my architecture, [Jim Gray's transaction concept (1981)](https://jimgray.azurewebsites.net/papers/thetransactionconcept.pdf) is mandatory, because it guarantees ACID isolation.

	

	

I built the sandbox below to show how writes with no coordination silently drop state. It steps through the same read, think, write cycle as my terminal demo, one agent at a time. Choose how many agents share the markdown task index, then step through their writes with no transaction and with one:

	

*[Interactive Architecture Simulator: split-brain - Explore live at https://ulukaya.dev/posts/ai-agent-split-brain-trap#lab-split-brain]*

	

	
		

### 03. Trap 2: Memory tearing and container recycles

		

I run my agents on serverless platforms such as Cloud Run or Cloud Functions. They give me elastic scale, but each container has a lifecycle. Say my agent process stops, or scales to zero, while it writes a 50 KB markdown index. Then the file is left half-written.

		

When the container starts again on the next turn, my agent finds truncated JSON or broken markdown syntax. Its tool calls crash at once. For deeper client transport failure patterns, read my guide on [Client-Side Runtime Agent Resilience](/posts/client-runtime-agent-resilience).

	

	
		

### 04. Trap 3: Phantom state loops and stale cache trust

		

Sometimes an index file says a task is `OPEN` while the underlying database marks it `RESOLVED`. My agent then holds two versions of the truth, and it falls into split-brain confusion.

		

The model does not query the ground truth. It trusts the summary file, decides that the resolution must have failed, and re-runs its API calls against the external systems. In my early tests, this produced duplicate GitHub issues, repeated Slack pings, and wasted compute. For cost protection patterns against runaway execution loops, see [The Production Reality of Firebase Spend Caps](/posts/cloud-spend-caps-firebase).

	

	
	

## The solution: The three-tier zero-drift stack

	

To remove the split-brain trap for good, I changed my architecture at its base: **I offload state indexing from the LLM to a transactional database.**

	
		
			
				
					Architectural dimension
					Fragile file-based storage
					Transactional cloud memory
				
			
			
				
					One fact, how many copies **State coherence**
					Two copies to keep in step. Dual-writes required across entity files and markdown index tables.
					One copy, read fresh. Single source of truth in Cloud Firestore with dynamic indexed queries.
				
				
					Two writers at once **Concurrency control**
					The last save wins. No atomic file locks; parallel tool calls overwrite and clobber state.
					A stale save is refused. An optimistic `version` check inside `runTransaction`; the server transaction itself locks what it reads (pessimistic by default in Standard edition).
				
				
					A container restarts **Serverless lifecycle**
					The state on disk is gone. Local disk state lost on container recycle or scale-to-zero.
					Nothing lives on disk. Stateless Cloud Run workers with zero persistent local disk state.
				
				
					What the agent reads **Reasoning stability**
					An old summary. Stale markdown summary tables poison multi-turn agent reasoning.
					The current record. Every query returns ground truth directly from the database engine.
				
			
		
	

	
		

### 05. Production architecture overview

		

My transactional memory architecture has three tiers, and each tier has one job: ingress identity, stateless execution, and atomic state storage.

		
		
			- **Service isolation and client attestation.** [Firestore Security Rules](https://firebase.google.com/docs/firestore/security/get-started) set `allow read, write: if false;` on `agent_tasks`, so no client SDK can touch it. Only my backend's Admin SDK writes. Server libraries bypass rules and authorize through the worker's IAM service account, so every write goes through the transactional path. App Check on the client-facing endpoints is a separate abuse control. It plays no part in the drift fix.

			- **Stateless tool execution and OCC coordination.** The worker runs agent tool logic statelessly. No conversation state or scratch files stay on a container's local disk from one turn to the next.

			- **Atomic document transactions and dynamic indexes.** Firestore provides ACID document transactions, atomic counters, and query indexes that reflect 100% fresh state on every read.

		
	

	
	

## Production implementation: Transactional memory in TypeScript

	

The TypeScript module below is my atomic agent memory engine. It runs on Cloud Run with the Firebase Admin SDK. It uses Firestore transactions to prevent lost updates. On a version conflict, it hands back the current state, so the agent can re-plan and retry. Its dynamic query helpers remove static index files entirely:

	
	

```
// Transactional Agent Memory Engine for Cloud Run and Cloud Firestore
import { initializeApp, getApps } from 'firebase-admin/app';
import { getFirestore, FieldPath, FieldValue, Timestamp } from 'firebase-admin/firestore';

if (getApps().length === 0) {
  initializeApp(); // Uses Application Default Credentials on Cloud Run
}

const db = getFirestore();

export type TaskStatus = 'PENDING' | 'IN_PROGRESS' | 'COMPLETED' | 'FAILED';

export interface AgentTaskDocument {
  title: string;
  status: TaskStatus;
  version: number;
  assignedAgent: string;
  lastUpdated: Timestamp;
  payload: Record;
}

export interface AgentTaskResponse {
  id: string;
  title: string;
  status: TaskStatus;
  version: number;
  assignedAgent: string;
  lastUpdated: string;
  payload: Record;
}

export type TaskUpdates = Partial>;

export type MutationResult =
  | { success: true; newVersion: number }
  | { success: false; reason: 'NOT_FOUND' }
  | { success: false; reason: 'VERSION_CONFLICT'; currentState: AgentTaskResponse };

export class TransactionalMemoryEngine {
  private tasksCol = db.collection('agent_tasks');

  private formatTask(id: string, data: AgentTaskDocument): AgentTaskResponse {
    return {
      id,
      title: data.title,
      status: data.status,
      version: data.version,
      assignedAgent: data.assignedAgent,
      lastUpdated: data.lastUpdated
        ? data.lastUpdated.toDate().toISOString()
        : new Date().toISOString(),
      payload: data.payload || {},
    };
  }

  /**
   * Writes only if nobody changed the task since the caller read expectedVersion.
   * The version field is the optimistic check; the transaction makes read, check, and write atomic.
   */
  async updateTaskAtomic(
    taskId: string,
    expectedVersion: number,
    updates: TaskUpdates
  ): Promise {
    const taskRef = this.tasksCol.doc(taskId);
    const { payload = {}, ...fields } = updates;

    // Expected outcomes are return values; anything thrown (permissions, deadlines, contention) propagates
    return db.runTransaction(async (transaction): Promise => {
      const snapshot = await transaction.get(taskRef);
      if (!snapshot.exists) {
        return { success: false, reason: 'NOT_FOUND' };
      }

      const currentData = snapshot.data() as AgentTaskDocument;
      if (currentData.version !== expectedVersion) {
        return { success: false, reason: 'VERSION_CONFLICT', currentState: this.formatTask(taskId, currentData) };
      }

      const newVersion = currentData.version + 1;
      // Field/value pairs: a FieldPath merges each key into the payload map as-is, even with dots or slashes
      transaction.update(taskRef, 'version', newVersion, 'lastUpdated', FieldValue.serverTimestamp(),
        ...Object.entries(fields).flat(),
        ...Object.entries(payload).flatMap(([key, value]) => [new FieldPath('payload', key), value]));
      return { success: true, newVersion };
    });
  }

  /**
   * On a version conflict, re-plans against the fresh state and retries, as agent B does in Figure 1.
   */
  async updateWithRetry(
    task: AgentTaskResponse,
    plan: (current: AgentTaskResponse) => TaskUpdates,
    maxAttempts = 3
  ): Promise {
    let current = task;
    for (let attempt = 1; ; attempt++) {
      const result = await this.updateTaskAtomic(current.id, current.version, plan(current));
      if (result.success || result.reason !== 'VERSION_CONFLICT' || attempt >= maxAttempts) return result;
      current = result.currentState; // re-read: plan again on top of the write that won
    }
  }

  /**
   * Fetches fresh, query-indexed state directly from Firestore.
   */
  async getActiveTasksForAgent(
    agentId: string,
    limitCount = 10
  ): Promise {
    const querySnapshot = await this.tasksCol
      .where('assignedAgent', '==', agentId)
      .where('status', 'in', ['PENDING', 'IN_PROGRESS'])
      .orderBy('lastUpdated', 'desc')
      .limit(limitCount)
      .get();

    return querySnapshot.docs.map((doc) =>
      this.formatTask(doc.id, doc.data() as AgentTaskDocument)
    );
  }
}
```

	

	

> Firestore composite index configuration: A query that combines equality filters, in operators, and custom ordering needs a composite index. Deploy the following configuration in your firestore.indexes.json:

	

```
{
  "indexes": [
    {
      "collectionGroup": "agent_tasks",
      "queryScope": "COLLECTION",
      "fields": [
        { "fieldPath": "assignedAgent", "order": "ASCENDING" },
        { "fieldPath": "status", "order": "ASCENDING" },
        { "fieldPath": "lastUpdated", "order": "DESCENDING" }
      ]
    }
  ]
}
```

	

I develop and test offline for $0.00. The Firebase Local Emulator Suite runs the entire transactional stack on my machine, so I provision no cloud resources. I start it with `firebase emulators:start --only firestore`.

	

> Architectural takeaway: In stateful autonomous systems, I never use language models to maintain storage indexes. I offload state to atomic database transactions. The database engine does the indexing and the concurrency control.

	

> Architecture blueprint and spec: Inspect my complete Transactional Memory Blueprint → or scaffold a repository-native specification tree with my noVibes Agent Spec Generator →

	
		

## Industry validation and benchmarks

		
			- [Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation (Sep 2026, arXiv:2609.03619v1)](https://arxiv.org/abs/2609.03619v1): Confirms that concurrent multi-agent state updates require explicit confidence-weighted state reconciliation to prevent conflicting memory overwrites.

			- [Trace2Tower: Transition-Aware EigenTrace Induction of Multi-Level Skills for LLM Agents (Sep 2026, arXiv:2609.05261v1)](https://arxiv.org/abs/2609.05261v1): Confirms that multi-agent transition traces must enforce strict state-transition preconditions to prevent divergent execution graphs.

			- [Bernstein and Goodman (1981): Concurrency Control in Distributed Database Systems (ACM Computing Surveys)](https://people.eecs.berkeley.edu/~brewer/cs262/concurrency-distributed-databases.pdf)

			- [Leslie Lamport (1978): Time, Clocks, and the Ordering of Events in a Distributed System (CACM)](https://amturing.acm.org/p558-lamport.pdf)

			- [Jim Gray (1981): The Transaction Concept: Virtues and Limitations (VLDB)](https://jimgray.azurewebsites.net/papers/thetransactionconcept.pdf)

			- [Cloud Firestore Transactions and Batched Writes](https://firebase.google.com/docs/firestore/manage-data/transactions)

			- [Firebase App Check Overview](https://firebase.google.com/docs/app-check)

			- [Cloud Run Serverless Compute Architecture](https://cloud.google.com/run/docs/overview/what-is-cloud-run)

			- [Firebase Local Emulator Suite](https://firebase.google.com/docs/emulator-suite)

			- [Google Genkit Open Source Framework](https://genkit.dev)