跪拜 Guibai
← All articles
Frontend · AI Programming · Agent

AI Agents Dominate Coding but Fail Everywhere Else — Here Are the Five Constraints Blocking Them

By 姆斯李 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

The agent boom in coding has created unrealistic expectations for other domains. The bottleneck is not model intelligence — it is the absence of fast, objective feedback and the legal impossibility of shifting liability onto software. Teams betting on agents for healthcare, legal, or enterprise automation are hitting a ceiling that better models alone will not break.

Summary

Programming gives AI agents a perfect training ground: a formal language with unambiguous semantics, a compiler that delivers binary feedback in milliseconds, and a sandbox where mistakes cost nothing. GitHub repositories go further, packaging commit histories as implicit action-outcome-reward chains that teach models what good code looks like. These conditions make reinforcement learning with verifiable rewards (RLVR) straightforward.

Outside of coding, every one of these advantages collapses. Medical knowledge resists formalization because "probably" is the most precise statement a nonlinear system allows. Legal and corporate data are passive logs, not causal trajectories — they record what happened, never why, and never what would have happened otherwise. The feedback loops that make agents improve simply do not exist.

A deeper constraint is sociological, not technical. In programming, the toolchain absorbs responsibility: a failing test blocks a merge, and nobody goes to jail. In medicine, law, or management, the human who signs bears the full legal and career cost of an AI mistake. That risk-reward asymmetry makes adoption irrational for any individual practitioner, no matter how accurate the model becomes.

Takeaways
Programming languages are humanity's most thorough knowledge-compression project; every token relationship is a hard constraint, which makes code an ideal training corpus for transformers.
GitHub repositories supply implicit reinforcement-learning signals: unit tests define correctness, PR reviews encode expert judgment, and stars provide distributed quality voting.
A merged PR is a behavioral trajectory of action → outcome → reward, not a static text sample, giving models millions of causal chains to learn reasoning from.
Medical records, court judgments, and ERP logs are passive event logs — they lack counterfactuals, decision rationales, and outcome labels, so they cannot support reinforcement learning.
Compilers and test suites deliver binary, zero-noise feedback in milliseconds; human evaluation is orders of magnitude slower and injects bias, fatigue, and cultural presuppositions.
In medicine and law, "correctness" is inherently non-binarizable — it is multi-interpretive, context-coupled, and continuous, so a discrete reward function cannot approximate it.
Coding decouples exploration from delivery: git branches and Docker containers make trial-and-error costless, while the physical world and organizational decisions have no undo.
When an AI decision succeeds, benefits are distributed; when it fails, the cost concentrates entirely on the human who signed off — a negative-convexity risk structure that makes adoption irrational.
Responsibility absorption by the toolchain (compiler, CI, git) is a unique property of programming; in other fields, every AI action still requires a human to judge, verify, and bear the consequences.
Insurance cannot solve the liability problem at scale because AI error distributions are not yet quantifiable for actuarial pricing, and most organizations cannot afford the premiums.
Agent penetration in other fields will follow a pattern of gradual erosion — AI handles low-stakes, well-bounded subtasks while a human always signs off at the end of the chain.
Conclusions

The argument reframes the agent gap not as an intelligence deficit but as an environmental mismatch: coding provides the three ingredients reinforcement learning requires — formal knowledge, fast objective feedback, and costless sandboxes — and almost no other domain supplies all three.

The distinction between passive event logs and causal trajectories is under-discussed in enterprise AI. Most industry data tells a model what happened, but reinforcement learning needs data that says what happened because of what action, which is rarely recorded.

The responsibility problem is a genuine hard ceiling, not a temporary friction. Legal personhood for algorithms is philosophically incoherent under current legal systems, and no amount of model scaling changes that.

The observation that AI adoption stops at "generate a draft for a human to revise" in most fields is not a failure of ambition — it is the rational equilibrium point given the absence of a referee and the concentration of liability.

Quantitative finance looks like a near-fit for agents but fails on two hidden dimensions: signal impurity (profit ≠ correct strategy) and the regulatory requirement that a licensed human remains accountable.

The insurance workaround merely transforms "who bears the cost" into "who can afford the premium," which excludes most organizations and does not resolve the underlying legal-subject problem.

Concepts & terms
RLVR (Reinforcement Learning with Verifiable Rewards)
A reinforcement learning paradigm where an agent improves through trial and error guided by a reward signal that can be objectively and automatically verified, such as a compiler's pass/fail output or a unit test result. It requires fast, clean, large-scale feedback to work.
Negative convexity in risk-reward structures
A situation where the downside of a decision is concentrated on a single actor while the upside is diffused across many. For AI-assisted decisions, the human who signs off bears the full legal and career cost of a failure, while the benefits of a correct decision are shared broadly, making the expected personal value negative even if the system is accurate.
Goodhart's Law
The adage that when a measure becomes a target, it ceases to be a good measure. In quantitative finance, a trading strategy that appears profitable in backtesting may stop working once widely adopted because market participants adapt, making the historical signal unreliable.
Knowledge formalizability
The degree to which a domain's knowledge can be expressed as explicit, context-independent rules. Programming is highly formalizable (x+1 always equals x+1); medicine and law are not, because their core concepts are inherently context-dependent and resist compression into deterministic rules.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗