Code Over Context: Why Written Agent Skills Fail in Production (and How to Distill Them into Deterministic Tools)

As agent capabilities expand, developers accumulate markdown instruction files describing CLI flags, formatting rules, and error recovery procedures. In a typical agent harness, these files are injected into the system prompt on every turn.

This practice introduces three severe production bottlenecks:

• Compounding Token Tax: Loading 15,000 to 20,000 tokens of markdown instructions across a 20-turn session consumes 300,000 to 400,000 input tokens, inflating API billing linearly with conversation length.
• Attention Degradation: As context length grows, models suffer from needle-in-a-haystack attention decay, missing critical constraints buried in middle paragraphs.
• Non-Deterministic Drift: Natural language instructions are probabilistic suggestions. Models occasionally skip validation steps, hallucinate non-existent CLI parameters, or format outputs inconsistently.

This failure mode is well-documented in tool-use literature. As demonstrated in ToolLLM (Qin et al., 2023), stuffing raw documentation into prompts causes severe parameter hallucination, whereas distilling capabilities into structured tool contracts restores execution fidelity.

Written skills create an invisible dependency on massive frontier models. Frontier reasoning models possess sufficient cognitive capacity to follow complex multi-step instructions despite prompt ambiguity.

However, when you attempt to deploy the same agent across lighter runtimes, the system breaks down in distinct ways:

• On-Device Runtimes: Local mobile and browser models operate under constrained context windows (often 2K to 8K tokens) and limited compute budgets. They cannot reliably ingest ten pages of markdown rules while maintaining conversational state.
• Fast Serverless Endpoints: While lightweight cloud models offer expansive context windows, stuffing them with markdown skills triggers attention degradation, inflates per-turn latency, and multiplies token costs across multi-turn sessions.

If your agent architecture requires a frontier model just to parse a date or validate a JSON payload, your system is economically and architecturally fragile.

To build resilient agents that operate across model tiers, decouple the cognitive reasoning layer from the deterministic execution layer. We formalize this separation as the Elastic Cognitive Envelope:

• The Brain (Probabilistic Reasoning): Intent classification, ambiguous user goal decomposition, creative synthesis, and high-level strategy. This layer belongs in the model.
• The Exoskeleton (Deterministic Code): State machines, schema validation, arithmetic calculations, API authentication, and file system mutations. This layer belongs in compiled or interpreted code.

Deterministic code eliminates prompt token bloat, executes in milliseconds, and enables standard unit test coverage. By moving operational invariants into code, the prompt shrinks from thousands of lines to a concise tool schema.

Depending on security, compute, and platform constraints, teams deploy this deterministic exoskeleton across several architectural topologies:

• In-Process Execution: In local developer tooling and CLI agents, the exoskeleton runs directly in the host process (as demonstrated in the git engine below), executing system commands with zero network overhead.
• Client-Side Runtimes: In mobile or web applications, one option is calling Gemini through client SDKs like Firebase AI Logic, allowing the client application to execute local on-device tools directly in-process.
• Stateless Serverless Services: When tools require private credentials, heavy dependencies, or privileged infrastructure, teams often package the exoskeleton as stateless containerized microservices on platforms like Google Cloud Run. When exposed to client applications, this boundary can be secured with cryptographic attestation like Firebase App Check to prevent unauthorized invocations.

How do you transition from written skills to deterministic code without losing developer velocity? We employ a three-stage distillation flywheel:

This pattern builds on the code-as-action model pioneered by Voyager (Wang et al., 2023), which demonstrated that autonomous agents scale by synthesizing executable programs into a permanent skill library rather than accumulating prompt instructions:

1. Stage 1 (Frontier Exploration): Use a frontier model with extended thinking to explore an ambiguous problem space, interact with APIs, and discover edge cases.
2. Stage 2 (Code Distillation): Once the workflow stabilizes, instruct the frontier model to synthesize the multi-turn interaction into a typed script with strict input and output schemas.
3. Stage 3 (Light Runtime Deployment): Expose the distilled script as a single tool call. Fast serverless or on-device models invoke the tool with minimal token overhead and zero execution drift.

The following implementation demonstrates a distilled tool harness. Instead of injecting a 500-line markdown guide on git branching and commit hygiene, the agent invokes a typed function that enforces invariants deterministically:

This pattern mirrors the architectural direction of the Model Context Protocol (MCP), which standardizes how agents discover and execute typed, deterministic tool contracts rather than parsing natural language instructions in system prompts.

import { z } from 'zod';
import { execFileSync } from 'node:child_process';

// 1. Strict input schema replaces 40 lines of prompt formatting rules
export const CommitActionSchema = z.object({
  branch: z.string().regex(/^[A-Za-z0-9_/-]+$/, 'Invalid branch name format').refine(b => !b.startsWith('-'), 'Branch name cannot start with a dash'),
  message: z.string().min(10).max(72),
  files: z.array(z.string().refine(f => !f.startsWith('/') && !f.includes('..'), 'Must be relative path inside repository')).nonempty(),
  signoff: z.boolean().default(true),
});

export type CommitAction = z.infer<typeof CommitActionSchema>;

// 2. Deterministic execution engine replaces multi-turn prompt retries
export class DistilledGitEngine {
  public static execute(action: CommitAction): { success: boolean; hash?: string; error?: string } {
    try {
      // Validate schema contracts before touching disk
      const validated = CommitActionSchema.parse(action);

      // Safe branch checkout without destructive overwrite
      try {
        execFileSync('git', ['checkout', validated.branch]);
      } catch {
        execFileSync('git', ['checkout', '-b', validated.branch]);
      }

      // Prevent CLI argument injection via '--' delimiter
      execFileSync('git', ['add', '--', ...validated.files]);

      const args = ['commit', '-m', validated.message];
      if (validated.signoff) args.push('--signoff');
      execFileSync('git', args, { encoding: 'utf8' });

      // Retrieve commit hash deterministically rather than parsing stdout
      const hash = execFileSync('git', ['rev-parse', 'HEAD'], { encoding: 'utf8' }).trim();

      return {
        success: true,
        hash,
      };
    } catch (err: any) {
      // Return structured, actionable error instead of raw stack trace
      return {
        success: false,
        error: err.message || 'Execution failed',
      };
    }
  }
}

When evaluating whether a capability belongs in a written skill or a distilled tool, evaluate the trade-offs across these dimensions:

Primary research and documentation

• Voyager: An Open-Ended Embodied Agent with Large Language Models (Wang et al., 2023): Foundational research establishing the code-as-action model and iterative skill library synthesis referenced in Section 04.
• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs (Qin et al., 2023): Empirical demonstration of tool distillation and execution efficiency over raw prompt instructions referenced in Section 01.
• Model Context Protocol (MCP) Specification: Open architectural standard for exposing typed, deterministic tool contracts referenced in Section 05.
• Google Cloud Run Documentation: Reference architecture for stateless serverless container execution hosting deterministic tool microservices.
• Firebase App Check Documentation: Reference implementation for cryptographic client attestation securing serverless tool endpoints against unauthorized agent invocations.