Interview · Agent · OpenAI

Inside a Self-Built Coding Agent: Prompt Assembly, Context Compression, and Multi-Agent Orchestration

By 沉默王二 · Jul 29, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Most developers use coding agents as black boxes and hit a ceiling when the tool misbehaves — wrong tool selection, lost context, style drift. Understanding the prompt structure, compression pipeline, and role isolation lets an engineer diagnose failures at the architecture level instead of re-prompting blindly, and the open-source PaiCLI implementation makes these internals concrete and reproducible.

Summary

A developer who built an open-source terminal coding agent (PaiCLI) walks through its internal architecture in an interview-style format. The prompt sent to the model is assembled from nine layers: four static layers (identity, persona, mode, approval) placed first to maximize Prompt Caching hit rates, followed by five dynamic layers that change each round. Context compression uses three tiers — tool-result truncation, conversation-history summarization triggered at 80% of the context window, and emergency auxiliary-context discard — each operating on different content at different granularities.

Skills and Tools are treated as distinct primitives: Tools are executable capabilities called via tool_call, while Skills are decision-level knowledge loaded on demand through a progressive-disclosure system with an LRU buffer capped at three active Skills. For complex tasks, the agent switches from ReAct mode to a Plan-and-Execute mode with topological sorting of dependencies, and a Team mode introduces Planner, Worker, and Reviewer roles with isolated conversation histories and prompt-level constraints that prevent the Reviewer from executing tools — keeping the roles from becoming both player and referee.

Incremental modifications trigger a full re-plan rather than appending to the original plan, because new requirements can change the dependency graph of already-completed steps. The whole system is open-source and serves as a reference implementation for developers who want source-level understanding of how coding agents work under the hood.

Takeaways

— Prompt assembly uses nine layers stitched in fixed order; the four static layers (identity, mode, persona, approval) go first to maximize Prompt Caching hit rates, while five dynamic layers change per round.

— Context compression runs in three tiers: tool-result truncation (immediate, no LLM cost), conversation-history summarization (triggers at ~80% of context window), and emergency auxiliary-context discard as a last resort.

— Each compression tier operates on different content — tool output, dialogue history, and auxiliary context respectively — so a single blanket compression would either waste space or hit the limit.

— Over-compression is detected by behavioral anomalies (model repeats completed work or forgets prior discussion) and task-success-rate drops; mitigation includes expanding retained rounds, marking key info as non-compressible, and backing up history before compression.

— Incremental feature additions trigger a full re-plan rather than appending to the existing plan, because new requirements can invalidate the dependency assumptions of already-completed steps.

— Tools and Skills are separate primitives: Tools are executable capabilities called via tool_call, Skills are decision-level knowledge loaded on demand; they complement each other and are not interchangeable.

— Skills use progressive disclosure — a 4KB index of names and descriptions stays in context, full bodies load via load_skill into an LRU buffer capped at three active Skills, and reference docs load only when explicitly required.

— Tool-call frequency runs 10:1 to 20:1 versus Skill loads, which is why Skills are not kept permanently resident in context.

— Complex tasks switch from ReAct mode to Plan-and-Execute mode with JSON plans carrying dependency lists; topological sorting catches circular dependencies before execution.

— Team mode assigns Planner (decomposition only, no tools), Worker (execution with full toolset, max 2 concurrent), and Reviewer (quality gate, max 2 rejections) with fully isolated conversation histories.

— Role isolation is enforced by prompt constraints, not technical blocks — the Reviewer could technically access tools but is instructed not to, preventing it from fixing and then reviewing its own code.

Conclusions

Placing static prompt layers first for Prompt Caching is a cost-optimization detail that most agent tutorials skip, yet it can multiply token costs if done wrong.

The three-tier compression design is notable because each tier targets a different content type — tool output, dialogue history, auxiliary context — rather than applying one generic summarizer, which avoids both premature compression and context-window overflows.

Triggering history summarization at 80% of the context window (not 100%) leaves headroom so the LLM call for summarization itself doesn't push the context over the limit.

The decision to re-plan on incremental changes rather than append is a real architectural tradeoff: appending is cheaper but silently breaks when new requirements shift the dependency graph of already-finished steps.

Keeping Planner and Reviewer tool-free via prompt constraints (not technical locks) is a pragmatic design choice that makes responsibility boundaries debuggable — wrong code means Worker, bad plan means Planner, missed defect means Reviewer.

The 10:1 to 20:1 tool-to-skill call ratio justifies progressive Skill loading economically; loading all Skills into context would waste tokens on capabilities most conversations never use.

Building an agent from scratch to understand commercial tools is a learning strategy that mirrors how developers once built their own frameworks to understand React or Kubernetes — the source-level understanding transfers directly to better prompting and debugging of production agents.

Concepts & terms

Prompt Caching

An LLM API feature that reuses computation for repeated prefix portions of prompts. By placing static content at the front, every request shares the same prefix, maximizing cache hits and reducing token costs. Dynamic content inserted before static content breaks the prefix and defeats caching.

ReAct Mode

A reasoning-and-acting loop where the model interleaves thought steps with tool calls. Suitable for simple, single-step tasks that don't require upfront planning or dependency management.

Plan-and-Execute Mode

A two-phase agent pattern: a planner first generates a structured plan with dependency-ordered sub-tasks, then executors carry out the plan. Used for complex tasks where steps have prerequisites that must be resolved before execution.

LRU Buffer

Least-Recently-Used eviction policy. In the Skills system, a fixed-capacity buffer holds up to three loaded Skills; when a fourth is needed, the one accessed least recently is dropped to make room.

MCP (Model Context Protocol)

A protocol for dynamically connecting agents to external tools and data sources. In this architecture, MCP-provided tools join the shared tool registry alongside the 11 core built-in tools.

Topological Sorting

An algorithm that orders nodes in a directed acyclic graph such that every node appears before all nodes that depend on it. Used here to validate the planner's output for circular dependencies and to determine parallel execution batches.

From the discussion

The discussion is thin. Most remarks dismiss the article as shallow, repetitive, or clickbait. One comment mistakes the technical walkthrough for corporate scandal coverage. No substantive engagement with the agent architecture emerges.

See top comments, translated →

Source: juejin.cn ↗ Google Translate ↗ Backup ↗