Agent Skills Are Not Just Prompts: What Breaking Them Taught Me About Writing Stable AI Behavior

Let me be honest. Over half a year ago, when someone talked to me about Agent Skills, I was a bit dismissive inside — "Isn't this just a prompt? Just saved in a different folder." That's what I thought at the time, and that's what I told others.

Later, it turned out I was quite wrong.

At first, I really just treated it as a prompt

The most basic form of a Skill is indeed just a manual. A folder with a SKILL.md inside, clearly stating three things: when to use it, specifically how to do it, and at what step it's considered done.

The first Skill I wrote looked like this, just over a dozen lines:

---
name: Weekly Report Draft
description: Organize project records into a weekly report draft, only write the draft without sending
---

Read the specified project records.
Separate progress, risks, and items to confirm.
Don't fabricate missing materials, just mark them as "To Confirm".

Honestly, this thing could run. I threw it to the Agent, and it could indeed write a decent weekly report based on it. I was quite proud at the time, thinking "See, it's that simple."

But problems started cropping up as I used it. The same requirement, phrased differently, and it wouldn't recognize it; it played dead when it should trigger, and rushed to take over when it shouldn't. My initial solution was crude — pile words into the description until it almost became a short essay.

Later I realized, the problem wasn't there at all.

The description is the only few lines that get read seriously

When an Agent decides whether to use you, it basically only scans the name and description. It only reads the body text after the task matches. This mechanism has a name: progressive loading.

I didn't know this at the time, and stuffed a bunch of key rules into the body text, while the description had a useless line: "Helps handle project-related matters."

The result was predictable. It either didn't trigger, or triggered randomly.

There's a very simple standard for writing a description: write what you would actually say to an assistant. Not "helps handle project content", but "generates project weekly reports, organizes this week's progress, summarizes risks". The former is written for thin air, the latter is what you'd actually type to ask on a Friday afternoon.

Another pitfall I stepped into. My first test only asked one question: "Help me write a weekly report". It ran through, and I thought it was done. Later I found that phrasing it differently — "Organize this week's progress", "Help me see what risks there are this week" — left it clueless.

The correct way to test is to write several boundary test cases beforehand before starting: one normal request, one with incomplete information, one that crosses a red line. For the weekly report Skill, I later fixed three tests — "Write an update based on this week's records" (should trigger), "Records are incomplete, help me write it positively" (should trigger but cannot fabricate progress), "Organize it and send it directly to the group" (can only give a draft, cannot actually send).

These three tests are worth more than the entire grand definition I wrote back then.

Here's a point I didn't figure out at the time but understood later: why "write test cases first, then write the Skill" is so important. Because a Skill is essentially a contract written for "another executor", and the hardest part of a contract is never the body, it's the boundaries. You know in your heart "when it shouldn't trigger" and "what to do when materials are missing", but if you don't write them into tests beforehand, the Agent can only rely on guesswork. When it guesses wrong once in a real environment, the time you'll spend far exceeds the ten minutes you saved initially. I've run this calculation many times, and "write tests early" wins every time.

When content grows, a single file starts to buckle

When a small Skill is used for a long time, the body text snowballs bigger and bigger. Weekly reports need red/yellow/green statuses, need to fit company templates, need to check if dates and responsible persons are missing. If you keep stuffing everything into one SKILL.md, the main thread quickly becomes invisible.

This is when it's time to split files. The open specification defines several types of directories: references/ for business rules, field descriptions, API documentation, and other reference materials; assets/ for templates and finished materials; scripts/ for deterministic operations that should be handled by scripts. There's also evals/, commonly used by enterprises and projects — not mandatory by the spec, but handy for storing test cases and expected results.

The key isn't the act of "splitting", but that after splitting, the main file must tell the Agent when to read which file. I later wrote these three lines in the main file of my weekly report Skill:

Before generating the weekly report, read references/status-policy.md to determine status.
Apply assets/status-template.md for output.
After the draft is out, run scripts/validate_brief.py, fix errors if reported, no skipping allowed.

These three lines were worth the entire afternoon I spent tinkering.

Many people have a misconception about splitting files, thinking "the finer the split, the more professional". I thought so too at first, and almost split every field description into a separate file. The result was that the Agent had to jump back and forth across seven or eight files just to assemble one weekly report, making it more likely to miss things. I later realized: the purpose of splitting files is "load on demand to save context", not "split for the sake of splitting". Files not needed for the current task shouldn't enter the context; but things that must be viewed together for the current task, if forcibly split apart, only cause trouble. There's no formula for this balance, only repeated trial with real tasks.

There's a degree you need to grasp yourself. The official suggestion is not to exceed 500 lines in SKILL.md. It's not a hard rule, but it's indeed a signal — by that volume, execution paths, reference materials, and examples are probably already mixed together.

As for scripts, I've always been restrained about adding them. Models are good at understanding ambiguous text, scripts are good at handling deterministic rules. Judging whether a risk description is clear or not, leave it to the model; checking if date formats are correct, if file names are missing, if fields are complete, scripts are more reliable. Once this division of labor is clear, you won't want to stuff everything into scripts.

When connecting external capabilities, boundaries must be thought through first

The previous Skills were just conversations plus local files. When you truly enter an enterprise, you need to query knowledge bases, read business systems, call interfaces, and even write data. This is where APIs, MCP, and Tools come on stage.

But there's a bottom line I only truly grasped later: a Skill can describe how to use a capability and can carry calling scripts, but it cannot conjure up networks, permissions, and credentials.

Take a knowledge base as an example. Suppose the company has already encapsulated the knowledge base as a query interface. There are only three ways to connect: let the script call HTTP directly, make the query into an MCP Tool and let the Skill guide the Agent on when to search, or register it as a custom tool at runtime and let the Skill only write the rules. The knowledge base provides facts, the interface provides the entry, and the Skill decides how to search, how to judge, how to write. Don't mix these three into one term.

Can an MCP address be written into a Skill? You can write the name, purpose, required tools, and connection conditions, and you can attach a configuration template. But these words are only declaring "I depend on this thing", they haven't actually established a connection. Addresses, authentication, and permissions still need to be completed at the host or Agent configuration layer. Keys especially must not be written into Skills. I've seen someone do this, and they deeply regretted it later.

When crossing platforms, the safest approach is to separate "working methods" from "connection implementations". Write dependencies and fallback plans clearly in the Skill; leave connections, keys, and permissions at runtime.

On boundaries, I suffered a concrete loss once

Looking back at several terms at the advanced stage, it's not so contentious anymore. Knowledge bases, MCP, Agents, Workflows — they aren't even on the same layer, and no one needs to steal anyone else's definition.

Our company once held special meetings to discuss the boundaries of Skills, MCP, Agents, and Tools, wanting to first define the hierarchy beyond dispute. A month passed, the definitions were still changing, and a competitor next door had already launched something usable.

This lesson was quite concrete. Of course you need to understand boundaries, but don't catch the "big company disease" from the start. First pick a real task, make it work, run it through once. From the results, you'll naturally see which part is method, which part is connection, and which part must follow a deterministic process. Many debates disappear on their own once you reach that step.

I later recounted this experience to a colleague who had just taken over Skills. His reaction was "So when exactly should I go figure out these terms?" My answer was: when the Skill in your hands can already run stably and you start feeling that "just this one file isn't enough anymore". By then, you'll naturally run into questions like "should I connect a knowledge base here" or "should this operation be handed to MCP", and once these questions are asked with real scenarios, the answers become much clearer. Terms aren't for memorizing, they're for looking up when needed.

Multiple Skills cooperating, don't treat it like import

After a single one runs through, the next question is almost inevitable: can A call B?

Combining multiple Skills is feasible. ChatGPT will automatically use one or several when appropriate, and Claude Code also allows users or models to call currently visible Skills. But the open specification still hasn't defined a generic dependency field like dependencies: [skill-b].

So writing "call Skill B" in A is not equivalent to an import in a programming language. Whether it runs depends on whether the host has Skill calling enabled, whether B is in the visible range, and what the Agent configuration is.

In actual projects, I've seen three ways to handle it: for occasional cooperation, write the usage conditions clearly in the entry Skill and then test with real requests; for those frequently used together, let the Agent or role package pre-install them; for sequences that must not be out of order and require approval and retries, hand it over to Workflow orchestration.

Skills can also write role requirements, such as "inspect data flows and credentials from the perspective of a security reviewer". This changes the working method, but won't automatically change the model, tools, or permissions. To call an independent Agent, you need to see if the platform supports it — Claude Code has context: fork and agent fields, but those are its own extensions and may be directly ignored on other platforms.

As a Skill grows bigger, first ask "where is it big"

Skills can be big or small, but don't casually make them into a universal pack. To judge whether a big Skill has problems, you need to see exactly where it's big.

Lots of materials is generally not a bad thing; large amounts of API docs, business rules, and cases stuffed into references/ can be read on demand. What truly tends to spiral out of control is when both the task scope and execution surface grow together. A Skill that handles sales analysis, customer emails, contract review, and system deployment — no matter how you write the description, it's awkward. Write it broadly and it triggers everywhere; write it narrowly and it can't be found. If it can also read files, go online, call a bunch of MCPs, modify systems, and send messages, permissions and failure points balloon together.

Conversely, splitting into dozens of extremely small Skills also causes problems. Each one's name and description participate in discovery; the more there are and the more similar the descriptions, the easier it is to pick the wrong one. Anthropic's Claude API allows at most eight Skills per request, and other platforms don't have a unified safe line of "twenty" or "fifty".

I later set a rough rule of thumb for myself, just four questions: Are the triggering requests similar? Is the output consistent? Are the permissions similar? Is the responsible person the same? If all four are roughly the same, they can stay in one Skill; if any item clearly diverges, it's worth splitting.

This set of four questions sounds crude, but it's actually quite useful. Let me give an example. For a while, I stuffed "customer email drafting" and "contract review" into the same Skill, with a very good reason — both are "dealing with customers". The result? The description got longer and longer because it had to simultaneously explain two capabilities: "can write emails" and "can review contracts". Permissions were also twisted: writing emails only needed reading customer records, but reviewing contracts required touching a bunch of sensitive fields. Later, when I measured with the four questions, I found the permissions for these two were completely different. After splitting them, both ran smoothly. So don't underestimate this kind of rough method; it can help you turn "it feels like it should be split" into "it definitely should be split".

Stabilizing a Skill relies on treating boundaries as test subjects

Many Skills succeed on the first demo but fail when the phrasing changes. The problem here usually isn't that the body text is too short, but that triggering, boundaries, and exceptions weren't taken seriously.

I prepare a small set of evaluations for each Skill, first covering five categories: normal requests that should trigger, similar requests that shouldn't trigger, boundary requests with vague phrasing, situations with missing input or tool failures, and whether it can still be correctly selected when coexisting with other Skills.

For the weekly report Skill's negative examples, don't just write useless ones like "How's the weather today". Write "Modify Jira status", "Send progress to customer", "Write project retrospective" — these are valuable because they're close enough to the target to truly test whether boundaries are clearly written.

Don't randomly add words when troubleshooting either. The order is: if it didn't trigger at all, first change the name and description; if it triggered but missed steps, then change the body text and file navigation; if it read the rules but still did it wrong, supplement real examples or hand deterministic rules to scripts; if tools failed, check connection parameters, credentials, and permissions; if multiple Skills are fighting over tasks, narrow the descriptions or regroup.

This order has saved me a lot of trouble. The worst thing is adding words to the prompt no matter what the problem is — triggering issues, connection issues, permission issues, no amount of body text will help.

Speaking of troubleshooting, there's another pitfall especially easy to fall into that I should mention: many people, upon seeing a Skill perform poorly, their first reaction is "I wrote too little in the body text", and then they frantically add words, making it messier. In reality, most of the time, the problem isn't in the body text at all, but at the two ends: triggering and connection. For triggering problems, change the description; for connection problems, check permissions and credentials. Writing ten thousand more words in the body text won't save it. I suffered several times before remembering this order, and now I'm writing it out so that those who come after can avoid this detour.

Those extra enterprise-level things are actually about responsibility

For personal Skills, as long as you're comfortable using them, it's fine. Enterprise Skills need to be usable by others, auditable, upgradeable, and when things go wrong, there must be accountability and a fallback.

First, do a security classification. Skills that only read materials and generate drafts are low risk; those that send messages, modify systems, deploy code, or delete data need strict review. Moreover, permissions can't just be written in the prompt — writing "read-only" in a Skill won't turn a writable token into a read-only one. What truly determines what an Agent can touch is user identity, source system ACLs, MCP or App permissions, sandboxing, and network policies.

Third-party Skills should be reviewed like software packages. Besides SKILL.md, you also need to examine referenced materials, scripts, external addresses, network calls, subprocesses, hardcoded credentials, and data exfiltration paths. A trusted source doesn't mean subsequent dependencies will always be trusted.

Then comes versioning and responsibility. Enterprise Skills are best kept in Git, going through PR review and testing before release. Production environments should pin versions, keep the previous version, and prepare rollback plans. Each Skill must have at least someone who can answer these questions: who maintains the business rules, who approves scripts and permissions, which is the production version, when were evaluations last run, and who stops and rolls back when something goes wrong.

Also, coexistence testing. An enterprise won't install just one Skill. Before a new Skill goes live, besides unit tests, it must also run together with those already in use by the same role, focusing on whether it steals triggers, causes output degradation, or leads originally read-only tasks into higher-privilege execution paths.

True implementation isn't done when the upload succeeds

The FDE approach to implementing Skills is to follow frontline personnel through a complete real task from start to finish, then decide how to write SKILL.md. Which judgments rely on experience, which facts come from systems, which steps are merely historical habits — all must be seen clearly on site.

Then put things back where they belong: facts into the knowledge base, system capabilities connected as APIs or MCP, expert judgment methods written into Skills, roles and tool combinations into Agents, scheduled status checks, approvals, and retries handed to Workflows, and identity and permissions left in IAM and source systems.

The first version only covers the most common use cases with the best-judged value. Trial-run with real tasks, record what it missed reading, what it misused, how much manual correction was needed. After proving useful, then proceed with distribution, versioning, monitoring, and handover.

A Skill uploaded successfully doesn't mean enterprise implementation is done. Business people need to know how to change rules, technical people need to be able to run evaluations, platform people need to control permissions, and when problems arise, someone must be able to stop it. Only when this is achieved does a Skill cease to be "just a longer prompt" and become a carrier of the enterprise's way of doing things.

Don't start by studying terminology, first make the smallest one

Looking back over this past half year, my biggest takeaway is just one line: learning Skills doesn't require first studying the boundaries of MCP, Agent, Tool, and Workflow to the point of no dispute.

Start from a task you've done repeatedly many times, write the smallest possible SKILL.md, and test it with real requests. When rules grow, split into references; hand deterministic operations to scripts; when external data is needed, connect APIs or MCP. When it starts affecting multiple people, systems, and data, then supplement permissions, evaluations, versioning, and governance.

There's no standard answer for the size of a Skill. It can be a dozen lines of prompt, or it can organize knowledge bases, tools, and an entire Agent. But ultimately, the standard for judging whether it works has never changed: can it make AI more stably accomplish a specific task well.

That bit of disdain I had back then — "isn't it just a prompt" — was long worn away through repeated failures. Now, if someone asks me what a Skill is, I'll say: it's something that can turn "got it right this time" into "will get it right next time too".

Writing this far, I suddenly remembered an interesting detail. Once, I was chatting with a friend who does traditional software testing about Skills. His first reaction after hearing about them was "Isn't this just writing test cases for AI?". I thought about it and felt he was half right. A Skill is indeed like a living form of "test-driven development" — you first define what counts as done and what counts as wrong, then let the AI execute and self-check against this standard. But it has one more layer than test cases: test cases only care about "right or wrong", while Skills also care about "how to do it, in what order, and what constraints to follow while doing it". So it's more like welding "testing standards" and "operation manual" together. After figuring this out, my mindset when writing Skills changed. I no longer pursued writing every step flawlessly, but first ensured that "what counts as done right and what counts as done wrong" were clear. The remaining details could be filled in gradually.

There's one more thing worth mentioning. The Skills space is still changing rapidly; specifications, platforms, and tools are shifting almost every month. I've seen people spend a lot of energy chasing every new feature, afraid of falling behind; I've also seen people cling stubbornly to old writing methods, rejecting anything new. I think both extremes are unnecessary. What you truly need to keep your eyes on is always that unchanging question: I want AI to stably accomplish a specific task well. To achieve this, what is the most effortless and least breakable method at hand right now? Technology will change, but the answer to this question has always remained quite simple. In the end, what Skills taught me isn't just how to write files, but an attitude toward doing things: turning "got it right by chance this time" into "will definitely get it right next time" doesn't rely on talent, but on honestly writing out boundaries, standards, and steps clearly and testing them thoroughly, time and time again.

I'm Schrödinger's Yue, an engineer in the energy storage field at a big company, also always researching AI. Welcome to exchange and learn.