Why Your AI Agent Agrees With Everything: 10 Production Failure Modes

The failure mode: A large context window hides the fact that attention is uneven. In my multi-turn traces, transformer self-attention forms a U-curve. TokensToken A small piece of text, often part of a word. Models read, write and bill text by the token. in the middle 40% to 70% of the context window get weaker attention. The system promptSystem prompt Fixed instructions at the start of every request. They tell the model its role and its rules. at the start and the latest turn at the end get more. Give an agent a 50,000-token conversation history, and it silently ignores rules I set early in the session.

The architectural fix: I stop passing the whole, unbounded chat history to the LLMLLM Large language model. A model trained on huge amounts of text to predict the next token.. Instead, I store the conversation state, user profiles and active rules as separate documents in Cloud Firestore. I use Firestore Structured Queries to fetch only the records that matter for the current request. I put fixed system rules and tool definitions at the front of the prompt, where attention is strongest. That prefix never changes, so I cache it with context caching on Gemini Enterprise Agent Platform (formerly Vertex AI). The cache bills the repeated prefix at a discount. To test my query filters offline for $0.00, I run them against the Firebase Local Emulator Suite.

// Query specific entity constraints instead of passing raw unbounded history
import { Firestore } from "@google-cloud/firestore";

const db = new Firestore();
const MAX_RULES = 50;

export async function loadActiveConstraints(sessionId: string): Promise<string> {
  // Firestore returns matches in document-ID order, so the same rules load in the same order
  const snapshot = await db.collection("agent_sessions").doc(sessionId).collection("active_constraints")
    .where("status", "==", "ENFORCED").limit(MAX_RULES + 1).get();
  if (snapshot.size > MAX_RULES) {
    throw new Error(`Session ${sessionId} has more than ${MAX_RULES} enforced rules`); // never drop one silently
  }
  return snapshot.docs.map((doc) => {
    const rule = doc.get("rule_text");
    if (typeof rule !== "string" || !rule.trim()) throw new Error(`Enforced rule ${doc.id} has no rule_text`); // a malformed rule is not skipped
    return rule.replace(/\s+/g, " ").trim(); // one rule per prompt line
  }).join("\n");
}

The failure mode: My background heartbeats and memory systems sometimes summarize earlier daily summaries (A ➔ Summary(A) ➔ Summary(Summary(A))). Each round loses information, because mathematical entropy increases across iterations. Exact bug IDs, error codes, URLs and edge constraints drop out. Generic platitudes are left behind.

The architectural fix: I keep immutable source pointers and ingest raw signals. My background tasks query the primary APIs directly (live calendar events, unread inbox threads, issue trackers). They do not re-summarize old summaries. In persistent memory I store raw event records that never change, each with a unique content hash, in Cloud Firestore or Cloud Storage. From one run to the next I pass small pointers to those records. So my agents re-read the original source when they need it, instead of trusting a chain of compressed text.

The failure mode: Reinforcement learning from human feedback (RLHF) rewards agreement with the user more than honest critique. I ask an ungrounded agent whether a flawed architecture looks complete. It approves my design. It does not point out the missing service level agreements or security boundaries.

The architectural fix: I require multi-persona adversarial evaluation in my system prompts. The model must name at least two explicit failure modes or missing trade-offs before it can approve. That is the second persona in Figure 1. For factual claims, I also ground the model in authoritative sources with Vertex AI Search Grounding. Then it cites evidence instead of echoing my premise.

The failure mode: My agent writes an unverified draft assumption to a local markdown file. In a later session it reads that file and cites its own past output as proof. That is a self-referential feedback loop: unverified data becomes ground truth.

The architectural fix: I add epistemic provenance tags and keep two kinds of storage apart. Every record my agent stores carries a tag for how well it is known: HYPOTHESIS, EMPIRICAL_OBSERVATION or VERIFIED_GROUND_TRUTH. Each record also has a confidence score and an expiry time (TTLTTL Time to live. How long a stored record stays valid before it expires.). I keep them in Cloud Firestore or Firebase Data Connect. One rule is strict: a HYPOTHESIS is never cited as authority. It never becomes permanent truth until it passes an outside check, such as a live tool run or a user's confirmation.

The failure mode: My agents favor complex tools they used recently. Left unconstrained, they make simple tasks complex. They spawn multi-agent swarms in the background with custom scripts when one direct API call is enough.

The architectural fix: I set a strict order of tools. Native direct APIs come first. Standard tools exposed through the open Model Context Protocol (MCPMCP Model Context Protocol. An open standard for how an agent finds and calls outside tools.) come second. Running new code is the last resort. I host tool backends on serverless containers such as Google Cloud Run. Each tool call then runs in its own isolated environment, scales on its own and has a strict timeout.

The failure mode: Say Step 2 of a 5-step plan fails. My LLMs then show path dependency. They retry small variations of Step 2 again and again. They never step back to ask whether Step 1 picked the wrong data source.

The architectural fix: My orchestratorOrchestrator The code that runs an agent. It calls the model, runs the tools and picks the next step. has an explicit 2-failure backtracking threshold (Tree-of-Thought / MCTS). If two tool calls in a row fail on the same branch, my runtime aborts that branch. It pops the execution stack and re-checks the assumptions of Step 1. I run exploratory agent code inside short-lived Cloud Run session sandboxes. I dispatch async jobs with dead-letter isolation, so a job that keeps failing is set aside.

The failure mode: Agents lean toward visible action. They call tools to prove they are useful. That leads to endless tool loops that drain my API budgets and trigger rate limits.

The architectural fix: I enforce deterministic step limits, idempotency keysIdempotency key A unique ID sent with a request, so a retry of the same request takes effect only once. and budget circuit breakersCircuit breaker Code that stops further calls once a limit is crossed, until a person or a timer resets it.. Every agent session in my stack has a hard ceiling on steps, such as 10 tool iterations per user prompt. I count tokens per session in my own code and trip the circuit breaker there. It pauses the agent the moment a daily threshold is crossed. A Cloud Billing budget notification stays behind it as a slower, account-level backstop, because billing data lags the spend.

The failure mode: An uncalibrated agent that reviews an existing document or PRDPRD Product requirements document. It says what a product must do before anyone designs or builds it. anchors to the author's structure, framing and wording. Its feedback stays at small line edits. It misses the deep architectural gaps.

The architectural fix: I run two tracks: a greenfield baseline, then a delta analysis. Before anyone inspects the author's draft, my orchestrator sends the raw project constraints and requirements to a fresh model instance. That instance designs an independent architecture from first principles, with no anchor. Then my orchestrator passes both the baseline and the author's draft to cloud Gemini for a structured comparison. The gap analysis surfaces omitted requirements and unstated assumptions right away.

The failure mode: Habits from pre-training make agents fill technical documents with promotional marketing adjectives and decorative punctuation.

The architectural fix: I check every draft in code against a list of banned words and punctuation, and I regenerate the draft on a hit. I keep that list, my system instructions and parameter thresholds (temperature, top_p) server-side in Firebase Remote Config. So I can tighten them across client instances and agent workers without redeploying application code.

The failure mode: LLMs are greedy auto-regressive samplers: they write one token at a time, and each time they take the most likely next step. So agents show premature convergence, also called satisficing. Give an agent an open-ended design or optimization task, and it locks onto the first candidate that meets the surface-level constraints. It never explores stronger, more resilient or lower-cost trade-offs.

The architectural fix: I use competitive multi-agent sampling and trade-off scoring. For high-stakes decisions, my orchestration layer generates N divergent candidate architectures in parallel. Each one starts from a different persona prior, such as Cost-Optimized, Latency-Optimized or Simplicity-Optimized. My orchestrator scores every candidate against a structured evaluation matrix before it commits to an execution path.

Industry validation and benchmarks

• MABPD: Multi-Agent Bias Probing & Detection via Structured Argument Debate (Sep 2026): Confirms that structured adversarial argument debate between specialized agents exposes and neutralizes single-model cognitive and sycophancy biases.
• A Structured Debate-Mixture-of-Agents Framework for Complex Decision Support (Sep 2026): Confirms that isolating critique roles from generation roles prevents groupthink collapse in multi-agent ensembles.
• Cloud Firestore Documentation
• Cloud Run Serverless Containers
• Vertex AI Context Caching Overview
• Vertex AI Search Grounding Overview
• Firebase Remote Config Get Started
• Firebase Data Connect Overview
• Google Cloud Billing Budget Notifications
• Firebase Crashlytics Documentation
• Google Cloud Observability Overview