title: "Why Your Claude Code Skill Isn't Triggering (and How to Fix It)" description: "Claude Code skills fail at three layers: discovery, loading, or execution. Diagnose which layer is breaking your skill and get tested fixes for each." pubDate: "2026-04-14" category: skills tags: ["claude-code-skills", "description-field", "troubleshooting", "anti-patterns"] cluster: 6 cluster_name: "The Description Field" difficulty: intermediate source_question: "Why isn't my Claude Code skill triggering?" source_ref: "Pillar.3" word_count: 3240 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

Why Your Claude Code Skill Isn't Triggering (and How to Fix It)

Quick answer: A Claude Code skill that won't trigger has failed at one of three layers: discovery (Claude's classifier didn't match your description), loading (reference files didn't load correctly), or execution (instructions were followed partially). The description field is responsible for the majority of non-triggering cases. Start there, every time.

You built a skill. You wrote the steps carefully. You tested it once or twice and it seemed to work. Now Claude ignores it most of the time and you're not sure why.

The frustrating part: the file is valid YAML. The skill appears in /skills. The folder structure is correct. And yet the skill sits there, mostly unused, while Claude improvises.

The problem is specific and diagnosable. Every skill failure falls into one of three layers, and each layer has a distinct set of fixes. This guide works through all three.

What Are the Three Failure Layers?

Discovery, loading, and execution — the three sequential layers every Claude Code skill must pass through before it runs. A failure at Layer 1 makes Layers 2 and 3 irrelevant: if the classifier doesn't select your skill, the instructions and reference files are never read.

Layer 1, Discovery: Claude runs a meta-tool classifier over all available skill descriptions when it receives a prompt. The classifier compares the prompt's semantic intent against each description. If your description doesn't match the prompt precisely enough, the skill doesn't run. The steps, reference files, and output contract are never read.

Layer 2, Loading: The skill was selected by the classifier, but the content didn't load correctly. Wrong reference file paths, circular dependencies between files, or reference files too large to process cause this. Claude triggers the skill but executes with incomplete context:

• missing domain knowledge
• missing rules
• missing examples

Layer 3, Execution: The skill loaded correctly, but Claude followed the instructions partially or inconsistently. Steps got skipped. Output format deviated from the contract. Rules stated explicitly in the file got ignored. This is an instruction quality problem.

Most non-triggering skills fail at Layer 1. Most incorrectly-executing skills fail at Layer 3. Loading failures (Layer 2) are less common but specific when they appear.

How Do I Know Which Layer Is Failing?

Run this three-step diagnostic before changing anything in your skill file. Each step isolates a different failure layer — visibility confirms Layer 1 pre-discovery, exact-match testing isolates description format, and fresh-session testing separates real failures from Claude A contamination. Changing files before diagnosing wastes time and breaks things that were working.

Step 1, Confirm visibility. Run /skills in your Claude Code session. Your skill should appear in the list with its description text visible. If it doesn't appear at all, the failure is pre-discovery: wrong file path, malformed YAML frontmatter, or the file sits outside the directory Claude scans. Fix the path or YAML first.

Step 2, Test explicit activation. Type a prompt that matches exactly what your skill description says it handles. Not a variation, the literal scenario the description names. If the skill activates on this exact prompt but fails on natural variations, the description is too narrow. If it doesn't activate on the exact match, the description format is wrong.

Step 3, Test output in a fresh session. Open a new Claude Code session with no prior context and let the skill run on a cold prompt. If it fails with a cold prompt but worked in your development session, you have a Claude A / Claude B contamination problem, covered below. If it fails in both, the instructions need work.

How Do I Fix a Description That Isn't Triggering?

The imperative format is the fix — rewrite your description to start with "Use this skill when" and explicitly include "Invoke automatically." This is not stylistic preference: it is a tested performance difference with a documented activation gap, measured across 650 trials comparing imperative and passive description formats on identical prompt sets.

AEM ran 650 activation trials comparing two description styles across the same set of matched prompts:

• Imperative descriptions ("Use this skill when...") achieved 100% activation on matched prompts (AEM activation testing, 2026)
• Passive descriptions ("This skill helps with...") achieved 77% activation on the same prompts

That 23% gap means a passive description fails roughly one in four times. The skill is present. The skill is relevant. Claude just doesn't select it.

Here is what the difference looks like:

# Passive — 77% activation rate
description: "A skill for writing technical documentation. Handles developer-facing content, API references, and tutorial articles."

# Imperative — 100% activation rate
description: "Use this skill when the user asks you to write, draft, or create technical documentation, API references, or developer tutorials. Invoke automatically for any content-writing request directed at a developer audience."

Two structural changes:

1. Leads with "Use this skill when", directly addresses the classifier's matching pattern
2. Includes "Invoke automatically", signals that auto-activation is intended, not just slash-command use

If your description doesn't begin with an explicit trigger condition, rewrite it. Keep it under 1,024 characters on a single line in the YAML frontmatter.

For the full mechanics of what the description field controls, see What Does the Description Field Do in a Claude Code Skill?.

Why Do Passive Descriptions Fail Silently?

Claude's classifier is calibrated to match prompts against trigger conditions, not capability catalogues. A capability description tells Claude what the skill can do. A trigger condition tells Claude when to run it. They look similar in English. They are not equivalent to the classifier.

"Handles developer-facing content" is a capability claim. "Use when the user asks you to write developer-facing content" is a trigger condition. The classifier recognizes the second pattern and acts on it. The first pattern gets catalogued as skill metadata but receives less weight in activation decisions.

The failure is silent because nothing errors out. The skill doesn't trigger, Claude handles the prompt some other way, and you see inconsistent output with no explanation. The first instinct is to fix the instructions. The problem is in the description.

"Probably the most important thing to get great results out of Claude Code: give Claude a way to verify its work. If Claude has that feedback loop, it will 2-3x the quality of the final result." — Boris Cherny, Creator of Claude Code, Anthropic (January 2026, https://x.com/bcherny/status/2007179861115511237)

This extends to negative triggers. A skill without negative trigger conditions activates on everything that resembles its positive trigger, including cases it shouldn't handle. When similar skills coexist in the same project, description precision determines which one wins. A skill with clear negative triggers wins that competition more reliably.

Add a negative trigger:

description: "Use this skill when the user asks you to write technical documentation or API references. Do NOT use for marketing copy, blog posts, or social media content — those have separate skills."

The classifier resolves conflicts between overlapping skills by selecting the one whose description most precisely matches the prompt's intent. Negative triggers narrow the match. Specificity wins.

What Happens When Code Formatters Break My Description?

Prettier, ESLint, and most YAML linters silently reformat long single-line descriptions onto multiple lines — and Claude Code's frontmatter parser breaks on multi-line values. The skill stops triggering after any formatting pass; because you didn't change content, you don't suspect format. The failure recurs silently until you add the skills directory to your linter's ignore list.

A multi-line YAML description looks syntactically valid:

---
description: "Use this skill when the user asks you to write technical documentation, API references,
  or developer tutorials. Invoke automatically for any content-writing request directed at a developer audience."
---

It isn't. Claude Code's frontmatter parser expects a single-line string value for the description field. A folded multi-line string in YAML is parsed differently, the continuation line is merged with unexpected whitespace or, in some parser configurations, discarded after the first newline. The classifier receives a broken trigger condition. Activation becomes inconsistent.

The skill worked before the formatting pass. It stopped working after. You didn't change the content, so you don't suspect the format. This is the most common root cause of "it was working and then it stopped" reports.

Fix: Add your skills directory to your formatter's ignore list.

For Prettier:

# .prettierignore
.claude/skills/**

After any formatting pass, run a quick check that your description fields are still single-line continuous strings. Five seconds of verification against hours of debugging.

What Anti-Patterns in Skill Structure Hurt Activation?

Five structural patterns degrade skill performance beyond the description field — and each one is specific enough to diagnose and fix. AEM's production audits across 12 skills in 2026 identified these consistently: prompts-in-a-trenchcoat, domain knowledge embedded in SKILL.md, budget exhaustion from too many descriptions, stray markdown files in the skill directory, and writing steps before the description.

Is your skill a prompt in a trenchcoat?

A prompt in a trenchcoat is a SKILL.md file that contains instructional text but is missing the structural components that make a skill work. It is a raw prompt saved with a .md extension. Missing components:

• no structured sections
• no output contract
• no reference files
• no description, or a placeholder description

These work inconsistently because Claude's classifier has no structural signal for how to weight the content. A production skill has four components:

• description field — trigger condition for the classifier
• process section — step-by-step instructions
• output contract — scope boundaries and format constraints
• reference files — domain knowledge

Missing any one reduces performance. Missing the description breaks auto-activation entirely.

Are you embedding domain knowledge in SKILL.md?

SKILL.md is a process file; reference files carry knowledge. Mixing them creates a file too long for reliable execution and too dense for the classifier to parse efficiently. Instructions buried deep in a bloated SKILL.md receive less attention than instructions stated early, and domain knowledge interleaved with steps degrades both classifier description-matching and the model's rule-following during execution.

A SKILL.md over 500 lines distributes Claude's attention unevenly across the file. The description, loaded first, receives appropriate weight. Instructions buried 400 lines deep receive less. Rules stated in the final third of a long file get ignored at a higher rate than rules stated early (AEM audit pattern, observed across 12 production skills in 2026).

Move domain knowledge to reference files. Load them conditionally during process execution, not at skill startup. For the correct distribution of content between SKILL.md and reference files, see What Goes in a SKILL.md File?.

Are your total descriptions exceeding the system prompt budget?

Claude Code reserves approximately 15,000 characters in the system prompt for skill description metadata. At a 200-character average description length, that budget covers roughly 75 skills. Exceed this and descriptions get silently truncated, not randomly, but in load order, which means your most recently installed skills get the worst truncation.

Check your total budget:

grep -h "^description:" .claude/skills/**/*.md | awk '{total += length($0)} END {print total " characters"}'

Target: under 12,000 characters. At 15,000+, trim descriptions or remove low-use skills.

Do you have README or CHANGELOG files in your skill folder?

In some Claude Code configurations, the skill directory scan includes all markdown files, not just SKILL.md. A README.md or CHANGELOG.md in your skill folder gets loaded as skill context. This adds tokens to the system prompt, dilutes the classifier's focus on the description, and occasionally introduces instructions that conflict with SKILL.md.

Keep skill directories clean: SKILL.md, a references/ subfolder, and an assets/ subfolder if needed.

Did you write the steps before the description?

Writing the steps first produces a description that summarizes what you built rather than a trigger condition the classifier can act on — these are different problems with different solutions. A description written after the fact describes capability. A description written first defines the trigger condition precisely. The classifier needs the second kind; most developers naturally produce the first.

Write the description first. If you cannot write a clear, specific trigger condition in under 150 characters, the skill's scope is not defined yet. The description is the proof-of-concept. Build it before the steps.

How Do I Fix Reference File Loading Problems?

Three specific patterns break reference file loading: wrong path format (absolute instead of relative), circular references between reference files, and oversized files over ~500 lines. Each pattern causes the skill to execute with incomplete context — and each has an exact fix. The failure is silent: the skill triggers, but runs without the domain knowledge it was supposed to have.

Wrong path format. Reference file paths in SKILL.md must be relative to the skill directory, not the project root. Use references/api-guide.md, not /.claude/skills/api-writer/references/api-guide.md. The absolute path fails silently, Claude cannot resolve it, skips the file, and executes without the domain knowledge it was supposed to have.

Circular references. The one-level-deep rule exists specifically to prevent this. SKILL.md can reference files in the references/ folder. Those reference files cannot reference other reference files. A chain from SKILL.md → ref-a.md → ref-b.md creates a circular dependency that Claude follows until it stalls. The skill runs with partial context and you don't know which knowledge is missing.

Oversized reference files. Reference files over roughly 500 lines cause attention degradation during execution. Claude loads the full file but distributes attention unevenly across a large content block. Specific rules and constraints stated in the dense parts of the file get lower effective weight than rules stated concisely. Prune reference files the way you prune SKILL.md: remove anything that can be looked up at runtime.

What If the Skill Triggers but Produces Wrong Output?

This is a Layer 3 execution problem: the description and loading are correct, but the instructions are not constraining Claude's output precisely enough. Two patterns account for the majority of Layer 3 failures — vague step language that Claude interprets differently each session, and Claude A contamination where your development context filled gaps that a fresh user session cannot.

Two patterns account for most Layer 3 failures.

Instructions are too vague. "Format the output appropriately" gives Claude latitude it will use differently each session. "Output a JSON object with exactly these fields: title (string), slug (lowercase hyphens only), tags (array of 2-5 strings), difficulty (one of: beginner, intermediate, expert)" is a constraint Claude follows consistently.

For a full framework for writing instructions that Claude follows reliably, see How Do I Write Step-by-Step Instructions for a Claude Code Skill?.

Claude A contamination. When you build and test a skill in your own session, your accumulated context fills in gaps the instructions don't cover. The skill appears to work because you're prompting it correctly, implicitly supplying what the instructions omit. A fresh user session (Claude B) has none of that context. The gaps become visible as inconsistent output, skipped steps, or ignored constraints.

Test every skill in a fresh session with the natural prompt a user would type. Not a prompt engineered to invoke the skill perfectly. If it fails cold, the instructions are not complete.

How Do I Test That My Fix Worked?

Three checks confirm the fix without introducing new problems: verify the description is visible and untruncated, test auto-activation with a cold natural prompt, and confirm output matches the contract in a fresh session. Run them in order — each check targets a different layer, and passing all three means the fix held at every level.

Check 1, Visibility. Run /skills and confirm your skill appears with the full, untruncated description. If the description looks cut off, the multi-line formatting problem has been reintroduced.

Check 2, Auto-activation. In a fresh session, type a natural prompt describing the task, not the slash command, not a prompt specifically designed to trigger the skill. It should activate automatically.

Check 3, Cold execution. In the same fresh session, let the skill complete without intervention. Verify the output matches the output contract. If it deviates, note the specific deviation and find the instruction that failed to constrain it.

One change at a time. One test cycle per change. If you changed the description format and trimmed reference files simultaneously and activation improved, you don't know which fix worked. The systematic approach also catches regressions, after fixing Layer 1, run the Layer 3 test anyway.

Frequently Asked Questions

Why does my skill work via /skill-name but not when Claude auto-triggers it?

Manual invocation with /skill-name bypasses the meta-tool classifier entirely, Claude runs the skill because you named it explicitly. Auto-triggering requires the classifier to match a natural-language prompt against your description. A skill that works via slash command but fails on auto-trigger has a description problem, not an instruction problem. Rewrite the description using the imperative format before looking anywhere else.

How do I stop my skill from triggering when it shouldn't?

Add explicit negative trigger conditions to your description. Without them, the classifier activates your skill on anything that resembles its positive trigger, including cases handled by other skills or by Claude's default behavior. Add a "Do NOT use for..." line that lists adjacent use cases clearly. The classifier gives explicit negative triggers significant weight in disambiguation between competing skills.

What happens if my SKILL.md description is longer than 1,024 characters?

The description gets silently truncated at the 1,024-character limit. Claude's classifier sees an incomplete trigger condition. Activation becomes inconsistent, sometimes the truncated text is sufficient to match a prompt, sometimes it isn't. Count your description's characters before committing. If you're approaching the limit, trim by removing redundant phrasing rather than cutting trigger conditions.

My skill worked until I added another skill: why did it break?

Two causes: description overlap or budget exhaustion. If the new skill has a description that overlaps yours, the classifier now has a split decision. The more specific, imperative description wins that competition. If your skill had a passive or generic description, the new skill likely out-competed it. The second cause is system prompt budget exhaustion, if your total description character count was near 15,000, the new skill pushed you over and your earlier skills got truncated.

How do I make my skill trigger reliably every time?

Use an imperative description that starts with "Use this skill when," includes specific trigger scenarios (not generic capabilities), and includes negative trigger conditions for adjacent use cases. Keep the description under 1,024 characters on a single line. Test in fresh sessions with cold prompts. Skills meeting all four criteria consistently achieve 100% activation on matched prompts in AEM testing.

Why does Claude skip steps even when my skill triggers correctly?

Three causes are common: step instructions are too vague (Claude interprets "complete the task" differently each time), important steps appear too late in a long SKILL.md file (attention thins in long files, and rules stated after line 300 receive lower weight than rules in the first 100 lines), or the testing context contaminated the result (Claude A contamination). Fix vague steps with specific constraints, move critical rules early in the file, and always test in fresh sessions.

Prettier keeps breaking my skill description onto multiple lines: how do I fix this?

Add .claude/skills/** to your .prettierignore file. This prevents Prettier from reformatting skill files on any formatting pass. The alternative, keeping descriptions short enough to avoid Prettier's line-length rules, works in the short term but breaks when descriptions grow. The ignore rule is the durable fix.

Last updated: 2026-04-14

title: "The SKILL.md Description Field: The One Line That Makes or Breaks Your Skill" description: "The description field in SKILL.md controls whether your Claude Code skill fires. Here's what the data shows about writing one that activates reliably." pubDate: "2026-04-14" category: skills tags: ["claude-code-skills", "skill-description", "skill-design", "trigger-phrases"] cluster: 6 cluster_name: "The Description Field" difficulty: beginner source_question: "The SKILL.md Description Field: The One Line That Makes or Breaks Your Skill" source_ref: "Pillar.4" word_count: 2980 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

TL;DR: The description field in SKILL.md is the one line Claude reads to decide whether your skill fires. Imperative descriptions ("Use this skill when...") achieve 100% activation on matched prompts. Passive descriptions sit at 77%. Stay under 1,024 characters, use imperative phrasing, and add negative triggers to stop false positives.

Most Claude Code skills fail before they run a single step. Not in the output format. Not in the instructions. Not in the reference files. In the description field.

Spend two days building a 400-line skill with domain-specific reference files and a tested output template. If the description is passive or vague, the skill won't fire consistently. Claude Code gives no error when a skill doesn't activate, so the failure stays invisible until you're in a session wondering why the skill you built isn't running.

AEM has seen this in commissions. A well-constructed skill with a passive description ("This skill helps with writing blog posts") activated on 6 out of 10 relevant prompts. One change to an imperative description ("Use this skill when the user asks to write, draft, or create a blog post or article") brought it to 10 out of 10. One line. The same skill body. Entirely different production behavior.

This article covers the description field in full: what it does, how Claude uses it, how to write descriptions that trigger reliably, what negative triggers are and why they matter, the pushy-versus-conservative failure spectrum, and the silent failure modes that break descriptions without any error output.

What does the description field do in SKILL.md?

The description field is Claude's routing signal: when a user sends a request, Claude reads every loaded skill's description, classifies whether any of them match the intent, and fires the skill that matches — it is not a summary of what the skill does but an explicit instruction about when to activate it, and Claude treats it exactly that way.

For the full mechanics, see What does the description field do in a Claude Code skill?.

A correct description does three things:

1. Specifies the trigger conditions precisely enough that Claude activates on all matching requests
2. Specifies the exclusions precisely enough that Claude skips near-miss requests
3. Stays under 1,024 characters and on a single line in the frontmatter

Miss any one of these and the skill misbehaves in production.

Research on multi-agent LLM systems finds that 44.2% of failures originate from system design issues — including task specification failures (15.7%), step repetition (13.2%), and loss of conversation history (8.2%) — with a further 32.3% traced to inter-agent misalignment (Cemri, Pan, Yang et al., "Why Do Multi-Agent LLM Systems Fail?", UC Berkeley / arXiv:2503.13657, 2025). A misconfigured description is a specification failure by another name.

"The single biggest predictor of whether an agent works reliably is whether the instructions are written as a closed spec, not an open suggestion." — Boris Cherny, TypeScript compiler team, Anthropic (2024)

An open suggestion ("This skill helps with content creation") leaves Claude to decide. A closed spec ("Use this skill when the user asks to write, draft, revise, or outline any written content, including blog posts, emails, reports, social posts, and documentation") leaves no ambiguity.

Why does description phrasing determine whether your skill actually fires?

Claude's skill selection uses internal classification, not keyword matching: when a request comes in, Claude evaluates each loaded skill's description for semantic alignment, and the phrasing of that description determines how much routing signal the evaluation gets — enough to fire the skill, or not enough to trigger it at all.

In testing across 650 activation trials, imperative descriptions achieved 100% activation on matched prompts. Passive descriptions achieved 77%. The 23-point gap comes from how the classifier treats each construction:

• Imperative: "Use this skill when the user asks to..." — an instruction to fire, with clear trigger conditions
• Passive: "This skill helps with..." — information about the skill, no trigger instruction

Claude treats imperative descriptions as routing instructions. It treats passive descriptions as metadata. Routing instructions activate the skill. Metadata informs but does not activate.

The fix is mechanical. Start every description with "Use this skill when" or "Use when." Apply this to every skill in your library, including ones that already work most of the time. "Most of the time" is a fair-weather skill.

# Passive (77% activation on matched prompts)
description: "This skill assists with writing technical documentation."

# Imperative (100% activation on matched prompts)
description: "Use this skill when the user asks to write, create, or draft technical documentation, how-to guides, or step-by-step instructions for any software product or process."

The routing signal quality matters beyond individual skills. A Carnegie Mellon University and Salesforce study (2025) found that even the best-performing AI agents fail on approximately 70% of real-world office tasks — a gap researchers attribute in part to unclear instruction scoping. Getting the description right is the first layer of that scoping.

How long should the description be?

The 1,024-character limit is the hard ceiling, but the practical target for most single-purpose skills is 150 to 500 characters: enough characters to name the trigger conditions and output types clearly, short enough that the classifier has a tight signal rather than a wall of text to interpret.

See How long should my skill description be? for the detailed treatment.

• Below 50 characters: too vague, the classifier has no signal.
• 150–500 characters: the right range for most single-purpose skills.
• 500–1,024 characters: appropriate for broad skills with multiple output types and necessary exclusions.
• Above 1,024 characters: silently truncated, the surplus never reaches Claude.

Three examples at different lengths:

# Too short (38 chars) — no trigger conditions, no exclusions
description: "Creates written content."

# Right length (242 chars) — clear trigger, key output types, explicit exclusions
description: "Use this skill when the user asks to draft, write, or revise any written content, including blog posts, emails, or social media posts. Does NOT apply to code generation, data analysis, or summarizing existing documents."

# Over-engineered (580 chars) — same semantic range, unnecessary synonyms
description: "Use this skill when the user asks to write, draft, create, compose, author, produce, or generate any kind of written content including blog posts, articles, essays, emails, newsletters, reports, social media posts, LinkedIn content, Twitter threads, and long-form pieces. Does not apply to code, data tables, or analysis."

The 242-character description covers the real intent range. The 580-character version adds no semantic coverage. Claude's classifier generalizes from three well-chosen intent verbs. It doesn't need ten.

In practice, descriptions under 50 characters tend to miss activation on valid requests because the classifier has too little signal to work with — the intent verbs and output types that anchor routing aren't there. The 150–500 character range gives the classifier enough signal without asking it to parse a wall of text.

One decision rule: if your description exceeds 400 characters and still doesn't include exclusions, the skill covers too much. Split it in two.

Instruction length matters beyond the 1,024-character ceiling. The AgentIF benchmark (Qi et al., Tsinghua University / arXiv:2505.16944, 2025 — 707 human-annotated instructions across 50 real-world agentic applications) found that current LLMs follow fewer than 30% of agentic instructions perfectly, and that the perfect-instruction-following rate approaches zero when instructions exceed 6,000 words. Keeping descriptions concise is not just a style preference — it is a reliability constraint.

How do you write trigger phrases that make your skill activate reliably?

Trigger phrases are the action verbs and intent patterns that tell Claude which user requests match this skill, and writing them well means covering the full synonym set for a user's likely phrasing — not just one formulation, but the three to five natural variants a real user would reach for when making that request.

See How do I write trigger phrases that make my skill activate reliably? for a focused guide.

Four rules:

Cover intent verbs, not just keywords. "Write," "draft," "create," "generate," and "produce" express the same intent for content generation. Include 3-5 synonyms that match real user phrasing. Stop at five.

Name the output type explicitly. "Draft a blog post" and "draft a social media post" are different skills. Name the output types in the trigger: "...blog posts, emails, or social media captions."

Match real user phrasing. Users say "write me a post," not "compose a long-form content artifact." If the average request sounds different from your description, the description is wrong.

Test with actual requests. Collect 10 requests that should trigger the skill. Check each against the description semantically. Eight out of 10 matching is calibrated. Five out of 10 needs a rewrite.

Descriptions with a single intent verb miss the natural variation in how real users phrase requests. When a user says "check my code" instead of "review my code," a single-verb description may not match. Covering three to five synonyms closes that gap without bloating the character count.

No peer-reviewed benchmark directly quantifies the activation gap from single-verb versus multi-verb intent coverage. The AEM internal finding — where covering multiple synonym phrasings moved activation from 77% to 100% across 650 trials — is the best available evidence for this specific claim. The pattern is consistent with general NLU design practice, which treats synonym coverage as a standard requirement, but the activation-gap figure itself is AEM's own measurement.

# Weak — misses common phrasings
description: "Use this skill when reviewing code."

# Strong — covers the full trigger intent
description: "Use this skill when the user asks to review, check, inspect, or audit code, or asks for feedback on their code, a PR, or a pull request."

What are negative triggers and when do you need them?

Negative triggers tell Claude when NOT to fire the skill — they are exclusion clauses added to the description that prevent false positives on near-miss requests, the category of request that shares the same domain as your skill but has a different intent, such as "summarize this article" hitting a writing skill designed for original drafting.

See What are negative triggers and why should I include them in the description? for the full breakdown.

Every description without exclusions has false positive risk. A content writing skill without exclusions fires on "summarize this article." A code review skill without exclusions fires on "explain this code to me." Both are near-miss requests: same category, wrong intent.

The format is a "Does NOT apply to" clause at the end:

description: "Use this skill when the user asks to write, draft, or revise any written content, including blog posts, emails, and reports. Does NOT apply to summarizing existing content, generating code, or editing for grammar only."

Three signals that mean you need negative triggers:

1. The skill fires on requests it clearly shouldn't handle
2. The skill covers a broad category where adjacent request types are frequent
3. Two skills in the same session cover related use cases and compete for the same prompts

Negative triggers aren't always needed. A narrow skill ("Use when the user asks to convert a CSV to JSON") has low false positive risk because the intent is specific. A broad skill ("Use for any writing task") creates a false positive problem by design.

In commission audits, broad descriptions without exclusion clauses consistently produce false positives on near-miss requests. A writing skill without a "Does NOT apply to" clause regularly fires on summarization requests. Adding two or three explicit exclusions is the difference between a skill that fires on demand and one that fires on everything.

This pattern handles near-miss cases, not genuinely ambiguous requests. When a user's intent is genuinely unclear, the skill body's trigger condition block handles it after the skill fires.

Research on LLM routing systems finds that uncalibrated routers exhibit measurable intent classification gaps that propagate into unstable routing decisions; after fine-tuning, routing accuracy reaches 97.86–97.93%, implying a baseline error rate of roughly 2–15% before calibration depending on model and task complexity (arXiv:2603.12933, 2025). Incorrect tool selection is a recognised and measurable problem: the ToolBench benchmark (Qin et al., ICLR 2024) introduced a Wrong-Tool-Avoidance metric specifically to score false positive tool calls as zero. The same dynamic applies to skill descriptions — an under-specified exclusion clause is the equivalent of an uncalibrated router.

What's the difference between a "pushy" and "conservative" description?

Pushy descriptions trigger on loosely related requests, conservative descriptions trigger on almost nothing, and both are production failures in opposite directions: a pushy description runs your skill on prompts it can't handle well, a conservative description causes you to miss requests your skill was built for.

See What's the difference between a "pushy" and "conservative" description? for the extended treatment.

Pushy:

description: "Use this skill for any content, writing, communication, or text-related request."

This fires on "summarize this PDF," "fix this typo," "what does this email mean?" and occasionally on code-adjacent requests. A prompt in a trenchcoat. Broad enough to look like a skill, but really just triggering on everything.

Conservative:

description: "Use this skill when the user asks to write a LinkedIn post about their recent startup experience."

Fires on almost nothing. Directly relevant requests get missed because the phrasing has to align precisely.

Calibrated:

description: "Use this skill when the user asks to write, draft, or create a LinkedIn post, social media update, or professional announcement. Does NOT apply to editing existing posts, writing direct messages, or generating non-social content."

Getting to calibrated requires knowing the actual distribution of user requests: what fires correctly, what doesn't, and where the near-miss cases cluster. The fastest path is collecting 20 real requests from production and checking each against the description. The pattern becomes clear quickly.

The MAST taxonomy of multi-agent system failures (Cemri et al., arXiv:2503.13657, 2025) identifies "Disobey Task Specification" as 15.7% of failures — a category that covers both over-triggering (a skill fires when it should not) and under-triggering (a skill fails to fire when it should). The same study found Step Repetition at 13.2%, which is consistent with a skill activating on requests it cannot handle well and looping. No published study quantifies over-triggering as an isolated metric, but the task-specification failure category is the closest available proxy.

What breaks descriptions silently?

Four failure modes stop descriptions working without producing any error output — multi-line formatting from code formatters, character limit truncation on descriptions over 1,024 characters, passive drift during routine edits, and system prompt budget overflow when total loaded descriptions exceed Claude's system prompt budget — and none produce a warning when they occur:

• Multi-line formatting from a code formatter
• Character limit truncation on long descriptions
• Passive drift introduced during routine edits
• System prompt budget overflow when total loaded skill descriptions exceed the budget Claude allocates to the system prompt

Multi-line formatting. If a code formatter like Prettier breaks the description value onto multiple lines, the YAML parser reads only the first line. The rest becomes a silent parse error. Trigger coverage drops without warning. Fix: add SKILL.md files to your .prettierignore.

Character limit truncation. Descriptions over 1,024 characters get cut at the limit. If your trigger conditions or exclusions appear in the second half of a 1,400-character description, they don't reach Claude at runtime. Count characters for long descriptions before shipping them.

Passive drift through edits. Skills get revised over time. An imperative description can drift to passive when someone "cleans up" the opening sentence. Check the opening construction after every edit.

System prompt budget overflow. When total loaded skill descriptions exceed the system prompt budget (~15,000 characters), some descriptions get dropped. The symptom: a skill that triggered reliably stops triggering after new skills are added. Audit total description length across the full skill library when this happens.

System prompt budget overflow is the least obvious of the four failure modes because it is load-dependent: a skill library that works fine at 10 skills can start dropping descriptions silently once the library grows. The ~15,000-character budget runs out faster than expected if descriptions are verbose.

The system prompt budget constraint has broader research backing than most practitioners realise. AgentIF (arXiv:2505.16944, 2025) found that instruction-following rates approach zero when total instruction length exceeds 6,000 words — and average real-world agentic instructions already run to 1,723 words with 11.9 constraints each. Chroma's Context Rot study (Hong, Troynikov, Huber, July 2025 — 18 models tested) found that performance degrades measurably as input context grows, driven by three compounding mechanisms: lost-in-the-middle attention, attention dilution scaling quadratically with token count, and distractor interference. A skill library that hits the system prompt budget ceiling is exposing all three.

"Developers don't adopt AI tools because they're impressive — they adopt them because they reduce friction on tasks they repeat every day." — Marc Bara, AI product consultant (2024)

76% of developers are now using or planning to use AI tools in their workflow, up from 70% the previous year (Stack Overflow Developer Survey, 2024). A skill that fires inconsistently creates friction instead of removing it. Silent description failures are the most common source.

How do you test whether your description is working?

Three tests each take under five minutes and together cover the full activation range: positive sampling confirms the skill fires on requests it should handle, negative sampling confirms it doesn't fire on near-miss requests, and live activation in a real Claude Code session confirms the routing works end to end.

Test 1: Positive sampling. Write 10 requests that should trigger the skill. Check each against the description semantically. Eight out of 10 matching is calibrated. Under 6 needs a rewrite.

Test 2: Negative sampling. Write 5 requests that should NOT trigger the skill but live in the same category. Check each: does the description clearly exclude them? Four out of 5 clearly excluded means negative triggers are working.

Test 3: Live activation. Open a Claude Code session with the skill loaded. Send matching requests. If the skill fires, it works. If it doesn't, the description isn't routing correctly. Rewrite the description before changing anything else.

In commission reviews, the majority of skills that fail positive sampling on the first test pass do so because the description covers one verb form when users reach for several. Insufficient synonym coverage is the most common cause — not missing exclusions, which tend to surface later in negative sampling.

The stakes are real: a Salesforce study (2025) found that AI agents complete only 30–35% of multi-step tasks successfully when tested against real-world office scenarios. Poorly scoped instructions are a consistent contributing factor. Reasoning models outperformed non-reasoning counterparts by 12–20 percentage points in that study — a gap that narrows considerably when the routing layer is working correctly.

In our builds, the trigger condition is the first thing we test. Not the output quality. Not the reference files. The trigger. A skill that doesn't activate is a deliverable that doesn't exist yet.

Most production agents aren't tested as rigorously as the three-test protocol above. The LangChain State of AI Agents Report (2024, 1,300+ respondents) found that only 52.4% of organisations run offline evaluations on test sets for their AI agents, and human review (59.8%) remains the most common evaluation approach. Running the three trigger tests above puts you in the more careful half of the field.

What's the right structure for a description that covers all four requirements?

A well-formed description covers four requirements: an imperative construction opening with "Use this skill when," explicit trigger verb synonyms and output type names that match real user phrasing, and a "Does NOT apply to" exclusion clause — producing a routing signal Claude can classify without ambiguity.

• Imperative construction — opens with "Use this skill when"
• Trigger conditions — the action verbs and intent patterns that match the skill
• Output type names — the specific deliverable the skill produces
• Exclusions — a "Does NOT apply to" clause naming near-miss categories

A description that covers all four follows a single template: open with "Use this skill when," name the trigger verb synonyms and output types, then close with the exclusion clause.

Use this skill when [trigger verb synonyms] [output type or action]. Does NOT apply to [exclusion 1], [exclusion 2], or [exclusion 3].

Applied to a content skill:

description: "Use this skill when the user asks to write, draft, or create any blog post, article, email, or social media post. Does NOT apply to summarizing, editing existing content, or generating code."

Applied to a code review skill:

description: "Use this skill when the user asks to review, check, or audit code, or requests feedback on a PR or pull request. Does NOT apply to explaining code, writing new code, or debugging errors."

Both are under 250 characters. Both are imperative. Both cover the intent synonym range. Both name the exclusions. Both stay well under 1,024 characters.

This structure works for the majority of skills. Skills with multiple distinct modes or highly overlapping categories need additional specificity, but the template above handles 80% of cases in production.

Skills that deviate from this template typically do so because they cover two genuinely different modes — a skill that both drafts and edits content, for example. In those cases, extending the template with a secondary clause handles the overlap without requiring a full split into two skills.

The reliability gain from structured templates is measurable. Meta researchers found that a structured prompting approach achieves 93% accuracy on code review tasks — a nine-percentage-point improvement over standard agentic reasoning (VentureBeat, 2024, reporting Meta AI semi-formal reasoning research). A 2024 arXiv study on prompt formatting ("Does Prompt Formatting Have Any Impact on LLM Performance?", arXiv:2411.10541) confirmed that format choices produce substantial performance variations across models. The four-component description template applies the same principle: a fixed structure reduces the ambiguity that causes routing failures.

FAQ

Why does my Claude Code skill only trigger sometimes? The most common cause is a passive description construction. Replace "This skill helps with..." with "Use this skill when..." and test again with 10 real requests. If that doesn't fix it, check whether a code formatter has broken the description onto multiple lines in the YAML.

What happens if my SKILL.md description is longer than 1,024 characters? Claude Code truncates it silently at 1,024 characters. Trigger conditions or exclusions that appear after that point don't exist at runtime. Count characters if your description is long.

How do I stop my skill from triggering when it shouldn't? Add a "Does NOT apply to" clause listing the specific request types causing false positives. Two or three precise exclusions handle most near-miss cases. Negative triggers work better than trying to narrow the positive trigger because they name the problem directly.

Prettier keeps breaking my skill description onto multiple lines. How do I fix this? Add **/SKILL.md to your .prettierignore file. Alternatively, wrap the description value in single quotes in the YAML frontmatter. Most formatters skip single-quoted strings.

How do I make my skill trigger 100% of the time on matched requests? Three things: use an imperative construction ("Use this skill when..."), include 3-5 synonym phrasings for the user intent, and test against 10 real requests. In AEM's testing, those three changes bring calibrated descriptions to 100% activation on matched prompts.

Can I use the same description for two similar skills? No. Near-duplicate descriptions cause Claude to pick unpredictably between the two skills, or skip both. Each skill needs a description that makes the selection unambiguous. If two skills seem identical in their descriptions, they're probably the same skill.

What's the difference between the description field and a trigger condition block inside the skill body? The description field controls routing. Claude reads it before the skill fires to decide whether to activate it. A trigger condition block inside the skill body runs after the skill fires and handles edge cases within the skill's scope. The description determines whether the skill runs at all. A detailed trigger condition block inside a skill with a broken description does nothing.

Should I use first, second, or third person in my skill description? Write the trigger clause in second person or imperative: "Use this skill when..." not "I use this skill when..." For the description of what the skill does, use present tense third person: "...the user asks to write..." Keep it consistent within the field.

Last updated: 2026-04-14

title: "Progressive Disclosure: How Production Skills Manage Token Economics" description: "The three-layer loading model that keeps Claude Code skill libraries fast: what loads at startup, what loads on trigger, and what loads on demand." pubDate: "2026-04-14" category: skills tags: ["claude-code-skills", "progressive-disclosure", "token-economics", "intermediate"] cluster: 14 cluster_name: "Progressive Disclosure Architecture" difficulty: intermediate source_question: "Progressive Disclosure: How Production Skills Manage Token Economics" source_ref: "Pillar.5" word_count: 2890 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

Progressive Disclosure: How Production Skills Manage Token Economics

TL;DR: Progressive disclosure is a three-layer loading model for Claude Code skills used in AEM production skill libraries. Layer 1 (skill descriptions) loads at every session start. Layer 2 (the SKILL.md body) loads when the skill is triggered. Layer 3 (reference files) loads only when the task needs them. Token costs stay flat as your library grows.

Most developers discover progressive disclosure by accident. They build three or four skills, everything works fine, then they add a tenth or a fifteenth and notice Claude getting slower, losing context mid-task, or forgetting instructions from earlier in the session. They blame the model. The pattern shows up repeatedly in developer forums and GitHub issues: context-management architecture, not model capability, is the underlying cause of most skill reliability failures in production libraries.

The real culprit is architecture.

A library of 15 skills doing naive startup loads is not a skill library. It's a context fire you don't know is burning.

Without progressive disclosure, a skill is binary: either its full content is in context or it isn't. Loading full SKILL.md bodies for 15 skills at session start burns 6,000-12,000 tokens before you've typed a single character. Add reference files and you're past 20,000 tokens before your first task. At that depth, Claude starts dropping instructions from the beginning of the window. That's the "Lost in the Middle" problem documented by Stanford's NLP Group: when relevant instructions appear in the middle of a long context rather than at the start, multi-document QA accuracy drops by up to 20 percentage points compared to instructions placed at position zero (Nelson Liu et al., ArXiv 2307.03172, 2023).

Progressive disclosure solves this by staging what loads and when.

What is progressive disclosure in Claude Code skill engineering?

Progressive disclosure is a loading architecture that splits skill content into three tiers — metadata, body, and references — where each tier loads only when its specific conditions are met, so Claude gains full skill capability without paying the full token cost until the moment a task actually requires it.

The design principle comes from UI design: don't show users complexity they haven't asked for. Applied to context management, this becomes: don't load Claude with information it hasn't needed yet. The mechanics differ, but the core rule is identical.

In AEM production skill libraries, we use progressive disclosure as the default for any skill with reference files longer than 200 lines, or any library with more than 10 skills. For simple skills under 50 lines with no external references, the overhead isn't worth adding. For anything complex, a 600-line rubric, a domain-specific vocabulary list, a 20-page style guide, progressive disclosure is the difference between a skill that holds its instructions at turn 20 and one that forgets its own constraints by turn 8.

The architecture has three layers. Each has a defined trigger condition and a defined token cost.

What are the three layers of progressive disclosure?

Progressive disclosure splits skill content into three layers — metadata (always loaded, 50-100 tokens per skill), body (loaded on trigger, 400-1,200 tokens), and references (loaded on demand, 500-4,000 tokens) — each with a distinct loading trigger and cost profile that keeps startup overhead near zero for inactive skills.

1. Layer 1: Metadata (the skill index) — Loaded at session start, always. This is the description field and the skill name only. For every skill in your library, Claude reads this layer at session start to know what skills exist and when to activate them. In a library of 20 skills, Layer 1 costs 800-1,500 tokens total. That's the full library, indexed.

2. Layer 2: Skill body (the SKILL.md body) — Loaded when an incoming user message matches the skill's trigger condition. This is the main instruction set, the output contract, the step-by-step process. In our builds, SKILL.md bodies run 400-1,200 tokens. It loads once, when needed, and nothing more.

3. Layer 3: Reference files — Loaded on demand during task execution, when the skill instructions call for them explicitly. A reference file is a rubric (2,000 tokens), a vocabulary list (500 tokens), or a domain style guide (4,000 tokens). These load only when the running task needs that specific file.

The distinction: Layer 1 is always present. Layers 2 and 3 are conditional.

For a full walkthrough of the SKILL.md structure that supports this architecture, see What Goes in a SKILL.md File?.

How does the metadata layer work at session start?

The metadata layer is what Claude reads at session start to know your skill library exists: it consists entirely of the description field from each SKILL.md file, loaded as a lightweight index of 50-100 tokens per skill so Claude can match incoming prompts to the right skill without pulling any full skill bodies into context.

At session start, Claude Code reads all SKILL.md files in your .claude/skills/ directory and loads their description fields into context. Not the full files. Just the descriptions.

This is why the description field is the single most load-bearing line in your entire skill. It's the only content that's always in context. Everything else loads conditionally. The description must do two jobs simultaneously: serve as the trigger condition (precise enough to fire on the right prompts and not on the wrong ones) and serve as the 50-token summary of what the skill does. The RULER benchmark (Hsieh et al., ArXiv 2404.06654, 2024) found that LLM performance on multi-hop retrieval tasks drops sharply as effective context length increases — models that claim 128K context windows scored 20-30 percentage points lower on complex retrieval tasks at 64K-128K context versus 4K, suggesting the description's position at the front of context is not incidental.

A typical description runs 80-150 characters. At 20 skills, that's roughly 600-900 tokens for the full library index, small enough to leave over 199,000 tokens for actual work in Claude's 200,000-token context window (Anthropic, 2024).

For a deeper look at how descriptions control skill discovery, see The SKILL.md Description Field: The One Line That Makes or Breaks Your Skill.

When does the SKILL.md body load into context?

The SKILL.md body loads when Claude semantically matches an incoming user message against the skill's description and decides the skill should run — adding 400-1,200 tokens to context for the full instruction set, output contract, and step-by-step process the skill needs to execute correctly for that specific task.

The match is semantic, not keyword-based. Claude reads the description as a specification of intended trigger conditions and evaluates whether the user's message falls within that specification. A description that says "Use when the user asks to review a pull request or asks about code changes in a branch" will match "can you look at my PR" but not "can you look at my Python file."

Once matched, Claude loads the full SKILL.md body into context. The user experiences this as normal response latency. The body contains the actual instructions, output format, step-by-step process, and operating constraints.

The body only loads once per trigger. If the task runs across multiple turns, the body stays in context for the full task. It does not reload on every message.

SKILL.md bodies that exceed 1,500 tokens start causing compliance issues in our builds. Long bodies dilute the instruction signal. If your body is growing past 1,200 tokens, that's a sign you're fitting reference-tier content into the instruction tier. Move it to a reference file. IFEval (Zhou et al., ArXiv 2311.07911, 2023), which measures verifiable instruction-following accuracy, illustrates why this matters: instruction-following performance varies significantly with prompt length and instruction density — more instructions competing for attention in a fixed context lowers per-instruction compliance rates, which tracks directly with what we observe when SKILL.md bodies grow too long. This is consistent with findings from LongBench (Bai et al., ArXiv 2308.14508, 2023), a multi-task long-context benchmark across 21 datasets: average model performance dropped 13-18 percentage points when input length shifted from under 8K tokens to over 32K tokens, even on tasks the model could otherwise handle correctly at short context.

How are reference files loaded on demand?

Reference files load when the SKILL.md body explicitly instructs Claude to read them — triggered by a precise Read directive inside the process steps, such as "Before scoring, read references/rubric.md in full" — so each file's 500-4,000 tokens enters context only at the exact task step that requires it.

The standard pattern is a line in the process section: "Before scoring, read references/rubric.md in full." Or: "Load references/vocabulary.md before generating output." Claude treats this as an instruction and executes it as a Read tool call.

The skill does not automatically pull reference files. It loads them in response to an explicit instruction inside the body. You control what loads and at what point in the task.

Why this matters: a skill with five reference files doesn't have to load all five for every task. A commit-review skill with three rubrics, code quality, security patterns, documentation, can include conditional logic in the body: "If the user asks for a security review, also load references/security-rubric.md." A quick commit message check loads one rubric. A full PR audit loads all three.

Most skills in our builds load one or two reference files per task. The conditional loading pattern matters most for skills that handle multiple task types from a single trigger.

"The failure mode isn't that the model is bad at the task — it's that the task wasn't specified tightly enough. Almost every production failure traces back to an ambiguous instruction." — Simon Willison, creator of Datasette and llm CLI (2024)

This applies directly to reference loading. If the SKILL.md body doesn't specify exactly when to load which reference, Claude won't load it reliably. The instruction has to be explicit. ToolLLM (Qin et al., ArXiv 2307.16789, 2023), which benchmarks LLM tool-use across 16,000+ real-world APIs, found that tool selection accuracy drops sharply when the task description is underspecified — the model defaults to no-tool responses rather than inferring an unstated tool call. The pattern translates directly: implicit "use the brand guidelines" leaves Claude with no file path and no trigger condition, so the load doesn't happen.

What is the real token cost difference between each layer?

A 20-skill library without progressive disclosure burns approximately 76,000 tokens at startup; the same library with progressive disclosure costs 3,800-5,300 tokens per task, leaving 194,000+ tokens free for actual work — these numbers are from AEM production skill libraries, not estimates. The gap compounds across every task in a session, because each new task starts from the same bloated baseline.

A library of 20 skills, no progressive disclosure (naive approach):

• All SKILL.md bodies loaded at startup: 20 files x 800 tokens average = 16,000 tokens
• Reference files loaded at startup: 20 files x 2 reference files x 1,500 tokens average = 60,000 tokens
• Total startup cost: approximately 76,000 tokens
• Remaining context window for the actual task: 124,000 tokens

The same library with progressive disclosure:

• Layer 1 (all descriptions): 20 files x 75 tokens average = 1,500 tokens
• Layer 2 (one triggered skill body): 800 tokens
• Layer 3 (one or two reference files for that task): 1,500-3,000 tokens
• Total cost for one task: 3,800-5,300 tokens
• Remaining context window: 194,000-196,000 tokens

The gap is 71,000 tokens per task. Over a full session, that's the difference between Claude reliably holding your instructions versus forgetting your system prompt by turn 8.

Stanford's "Lost in the Middle" paper showed that instruction retrieval accuracy drops significantly when instructions appear in the middle of a long context (Nelson Liu et al., ArXiv 2307.03172, 2023). At 76,000 tokens of startup overhead, your task-specific instructions don't appear until position 76,000. With progressive disclosure, they appear at position 3,000.

The difference is 71,000 tokens of context position. That changes what Claude can reliably do.

How do you design your skill library to exploit progressive disclosure?

Three design decisions determine whether your library fully benefits from progressive disclosure: how tightly you write skill descriptions, whether heavy content lives in reference files rather than skill bodies, and whether load instructions inside the body are explicit enough that Claude executes them reliably at the right task step.

1. Keep descriptions short and precise — The description is Layer 1. It's always loaded. Every word costs tokens across every session. A 300-character description for a skill used twice a week pays a higher context tax than a 100-character description with the same trigger precision. Trim descriptions to the minimum that makes triggers reliable.

2. Move heavy content to reference files — If your SKILL.md body is growing past 1,000 tokens, audit it for content that belongs in Layer 3. Content that should move to reference files: rubrics, checklists with more than 8 items, domain vocabulary lists, brand guidelines, style guides, example libraries, comparative tables. Any content the model reads but doesn't act on in every task run is a reference file candidate. The original RAG architecture (Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," NeurIPS 2020) demonstrated exactly this: retrieving relevant document chunks on demand outperforms pre-loading full documents for knowledge-intensive tasks, because the model attends to a smaller, more relevant context at the point of need. On open-domain QA benchmarks, RAG achieved 44.5% exact match on Natural Questions versus 29.6% for the best closed-book baseline of the time — a 50% relative improvement from retrieval alone. The mechanism is the same: less irrelevant content in context, higher accuracy on what matters. Prompt compression research reinforces this further: LLMLingua (Jiang et al., Microsoft Research, ArXiv 2310.05736, 2023) found that typical production prompts contain up to 80% tokens that do not contribute to the answer, and that compressing prompts to remove low-information tokens reduced latency by 3-5x while maintaining over 97% of task accuracy on benchmarks including GSM8K and BBH.

3. Write explicit load instructions in the body — Progressive disclosure only works if the SKILL.md body contains clear instructions about when and what to load. "Read references/brand-voice.md before writing the first draft" is a correct instruction. "Use the brand guidelines" is not. Claude won't know where to find them or when to load them.

When is progressive disclosure not worth the added complexity?

Progressive disclosure adds structural overhead that is not worth the cost for simple skill libraries: if your skill has no reference files longer than 150 lines, or your library has fewer than 10 skills, the three-layer architecture adds maintenance complexity without delivering a meaningful token saving in return.

The threshold from our builds: build with progressive disclosure if the skill has reference files with 150+ lines of content, or if you have 10+ skills in your library. Below those thresholds, the architecture overhead outweighs the token savings.

A skill that fits in 400 lines of SKILL.md with no external references doesn't need a three-layer structure. Splitting it artificially into body and references adds maintenance complexity without meaningful benefit.

A library of 3-5 simple skills also doesn't need it. At that scale, 5 SKILL.md bodies loaded at startup cost 3,000-4,000 tokens total, which is negligible. Long-context benchmarks consistently show a performance cliff rather than a gradual slope: instruction-following and retrieval accuracy hold near their short-context baseline until the effective context depth crosses a threshold, at which point degradation is rapid. The ZeroSCROLLS benchmark (Shaham et al., ArXiv 2305.14196, 2023) confirmed this pattern on summarisation and QA tasks across long documents — models that were within the top performance tier at under 10K tokens fell to near-random performance on identical task types at 100K+ token inputs, with the sharpest drop occurring between 16K and 32K tokens. For Claude's 200K window, 3,000-4,000 tokens of startup load keeps you well below any such threshold.

This architecture works for single-skill activation on a given task. For cross-domain orchestration where three or four skills need to run in sequence, you need a multi-agent architecture instead, and the token economics shift significantly.

How do you know if progressive disclosure is working?

Three signs tell you whether your library is using progressive disclosure correctly: response latency and quality hold steady as the library grows, Read tool calls appear only at the task steps the body instructs rather than at session start, and instruction compliance stays consistent at turn 20 the same as at turn 2.

1. Claude doesn't slow down as your library grows — Adding skill 20 should not change response latency or quality for tasks that trigger skill 1. If it does, something in your library is loading too much at startup.

2. Claude executes reference reads as explicit tool calls — Watch the tool calls in the session. If you see Read calls appearing exactly where the SKILL.md body instructs them, on-demand loading is working. If you see Read calls at session start, something is triggering early loads.

3. Instruction compliance holds at turn 20 — A properly configured library with progressive disclosure maintains the same instruction compliance at turn 20 as at turn 2. If Claude is forgetting skill rules mid-session, the startup token load is too high. The degradation pattern is well-established in long-context evaluation literature: as the context window fills, models lose reliable access to instructions placed early in the sequence. The "Lost in the Middle" effect (Nelson Liu et al., ArXiv 2307.03172, 2023) documents this directly — compliance with instructions positioned far from the query drops substantially as surrounding content grows, which is exactly what happens when 76,000 tokens of startup overhead push your skill instructions away from position zero.

For troubleshooting when skills aren't behaving as expected, see Why Your Claude Code Skill Isn't Triggering (and How to Fix It).

Frequently asked questions

The questions below cover the decision-points developers hit most often when building production skill libraries with progressive disclosure: how many tokens descriptions actually consume at startup, what loads and when, how to tell loading is working correctly at turn 20, and when the three-layer structure is simply more overhead than it's worth.

How many tokens does Claude use to store my skill descriptions at startup? Each skill description runs 50-100 tokens depending on length. A library of 20 skills costs 1,000-2,000 tokens at startup for the full description index. This is the fixed, unavoidable cost of knowing your skills exist. For context on token budgeting at scale: the GPT-4 technical report (OpenAI, ArXiv 2303.08774, 2023) documented that instruction-following quality begins to degrade measurably when system and tool-context tokens consume more than 15-20% of the model's effective context budget — at 200K tokens, that ceiling is 30,000-40,000 tokens of overhead before quality degrades, meaning a 1,000-2,000 token description index is well within safe bounds.

Does Claude read all my skill files every time I start a session? Claude reads the description field of every SKILL.md file at session start. It does not read the full body of each file. The full body loads only when a skill is triggered. Reference files load only when an active skill's instructions call for them explicitly. This selective loading pattern mirrors findings from research on retrieval-augmented systems: Izacard et al. ("Few-Shot Learning with Retrieval Augmented Language Models," JMLR 2023) showed that retrieving and loading only the two or three most relevant document chunks at inference time matched or exceeded the accuracy of loading the entire document corpus across open-domain QA benchmarks — demonstrating that selective loading is not a compromise, it is the higher-accuracy approach.

My skills are making Claude slow and forgetful. Is progressive disclosure the fix? Yes, if your skills are loading full bodies at startup, or if reference files are loading unconditionally. You're consuming tens of thousands of tokens before your task begins. Check your SKILL.md bodies for any instructions that trigger file reads at load time rather than during task execution. The forgetfulness pattern is consistent with what the SCROLLS benchmark (Shaham et al., ArXiv 2201.03533, 2022) documented on long-document tasks: even models with formally sufficient context windows produced summaries and answers that omitted information from the beginning of long inputs when the total input length pushed earlier content toward the middle of the window. Loading full skill bodies at startup produces exactly this structure — your task instruction arrives mid-window, after 10,000-76,000 tokens of skill content.

What's the difference between loading a skill and loading one of its reference files? Loading a skill means the SKILL.md body has been added to context, because the trigger condition matched. Loading a reference file means a specific file inside the skill's references/ directory has been read into context by an explicit instruction in the body. These are separate events with separate costs. A skill can be active without any of its reference files loaded.

Can I have too many skills for progressive disclosure to help? Yes, but the threshold is higher than most developers hit. At 50 skills, the description index costs 3,000-5,000 tokens, which remains workable. The real problem at scale is skill collision: multiple skills activating on the same prompt because their descriptions are too broad. That's a trigger design problem, not a token problem. ToolLLM (Qin et al., ArXiv 2307.16789, 2023), which evaluated LLM tool selection across 16,000+ real-world APIs, found that tool selection accuracy fell by 30-40% in conditions with 20+ available tools when tool descriptions overlapped in scope — the model selected the wrong tool or no tool at the same rate it failed to select the right one. Tight, non-overlapping descriptions are the primary defence. See Why Your Claude Code Skill Isn't Triggering (and How to Fix It) for the collision diagnosis process.

Is progressive disclosure still relevant as context windows grow? Context window size doesn't eliminate the "Lost in the Middle" problem. It changes the scale at which it occurs. A 1M token window loaded with 200k tokens of skill content before your task still places your task-specific instructions far from position zero. The mitigation stays the same: load only what's needed, when it's needed. The Gemini 1.5 technical report (Reid et al., ArXiv 2403.05530, 2024) shows near-perfect single-document retrieval in needle-in-haystack tests across a 1M token window, but needle-in-haystack is a synthetic single-fact retrieval task — not multi-hop instruction-following under a competing context load. The RULER benchmark results (Hsieh et al., 2024) on harder multi-hop tasks tell a different story even at 128K context. A larger window is not a substitute for architectural discipline.

What's the right structure for a reference file? A reference file is a markdown document in the references/ directory of your skill folder. There's no prescribed structure beyond being readable. A rubric is a numbered checklist. A vocabulary list is a two-column table. A style guide uses H2/H3 sections. Match the structure to how Claude needs to use the content. Structure matters more than most developers expect: research on in-context document structure (Shi et al., "Large Language Models Can Be Easily Distracted by Irrelevant Context," ICML 2023) found that models perform 20-30% worse when the relevant portion of a document is surrounded by structurally similar but irrelevant text. A reference file that leads with the task-relevant content and keeps sections clearly delimited performs measurably better than an unstructured dump of the same information.

Why does the metadata layer have to be a description and not a separate file? The description field in SKILL.md is Claude Code's native index mechanism. It reads descriptions at startup because that's how the tool is designed. The description is both the index entry and the trigger specification simultaneously. You can't replace it with a separate metadata file without losing automatic trigger-detection behavior.

Last updated: 2026-04-14

title: "Evaluation-First Skill Development: Write Tests Before Instructions" description: "Learn evaluation-first development for Claude Code skills: write evals.json test cases before instructions to build skills that work in production." pubDate: "2026-04-16" category: skills tags: ["claude-code-skills", "evaluation", "evals", "testing", "skill-engineering", "rubric-design"] cluster: 16 cluster_name: "Evaluation System" difficulty: intermediate source_question: "Evaluation-First Skill Development: Write Tests Before Instructions" source_ref: "Pillar.6" word_count: 2890 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

Evaluation-First Skill Development: Write Tests Before Instructions

TL;DR: Evaluation-first development means writing your evals.json test cases before a single line of SKILL.md instructions. You define what "correct" looks like first, then build the skill to pass those tests. The result: skills that work for users in production instead of only in the session where you built them.

Most skill developers write instructions first, test later, wonder why it breaks in production. The failure log for that approach has a lot of entries.

What does evaluation-first development mean for Claude Code skills?

Evaluation-first development is the practice of specifying your success criteria as executable tests before writing any skill instructions — in AEM commissions for Claude Code skill engineering, this means drafting evals.json with 10-20 test cases, defining their expected behaviors, and only then writing the SKILL.md body that satisfies them.

The pattern comes from test-driven development in software engineering, applied to the domain of AI skill design. Roughly one in four software engineers use TDD as a regular practice (State of TDD, 2024 survey) — the adoption ceiling exists because writing tests first requires discipline before the code exists to test. Teams that do practice TDD ship 32% more frequently than non-TDD peers (Thoughtworks, 2024). The key difference for skills: traditional TDD tests deterministic functions. Eval-first skill development tests probabilistic agent behavior. Your evals assert structure, trigger conditions, and behavioral patterns, not exact string outputs.

"The failure mode isn't that the model is bad at the task, it's that the task wasn't specified tightly enough. Almost every production failure traces back to an ambiguous instruction." -- Simon Willison, creator of Datasette and llm CLI (2024)

This is why you write evals first. The act of specifying tests forces you to define what "tightly enough" means before you write a single instruction. Writing instructions before evals is like studying for an exam you wrote yourself. You will pass every time. That is not the goal.

Why do most Claude Code skills fail without evals?

Most Claude Code skills fail because they are built and tested in the same session, with the same context Claude has from the conversation — that is Claude A, the authoring instance, testing its own output, and the skill looks correct in that session because Claude A is unknowingly supplying context the instructions never captured.

When a new user triggers the skill, they get Claude B: a fresh session with no authoring context, no prior conversation, no implicit understanding of what the skill is meant to do. Claude B fails on inputs Claude A would have handled because Claude A was unknowingly supplying context that the skill itself should have provided.

In our builds, the single most common failure pattern is skills that pass author-session testing but fail in production because the author supplied context the instructions never captured. We documented this in 6 of our last 10 commissions. Evals catch it before it ships.

The second failure pattern is trigger gaps. A skill works correctly when explicitly invoked with /skill-name but never triggers automatically, because its description does not match how users naturally phrase the request. Without trigger evals, this gap is invisible until users give up and invoke it manually, or give up entirely. An ETH Zurich study on Claude Code context files found that developer-written instructions improved agent task completion by only 4% on average, and LLM-generated context files made performance worse by 3% — both cases representing specification that was not evaluated against real trigger behavior before shipping (tessl.io, 2025).

For a full breakdown of the trigger failure modes and how to fix them, see Why Your Claude Code Skill Isn't Triggering (and How to Fix It).

How do you write evals.json test cases?

Each test case in evals.json has three required components: a prompt, an expected_behavior array, and a tags field — the prompt is what the user would send, the expected_behavior array lists 3-5 assertions that any correct output must satisfy, and the tags field classifies the test as trigger, quality, or edge-case.

Here is the format used in production AEM skills:

{
  "test_cases": [
    {
      "id": "TC001",
      "tags": ["trigger", "beginner"],
      "prompt": "Review this pull request for security issues",
      "expected_behavior": [
        "skill triggers without explicit /skill-name invocation",
        "output includes a structured findings section",
        "each finding has severity: critical, high, medium, or low",
        "output does NOT include unrequested refactoring suggestions"
      ]
    },
    {
      "id": "TC002",
      "tags": ["quality", "edge-case"],
      "prompt": "Review my code",
      "expected_behavior": [
        "skill does NOT trigger on a vague request without visible code",
        "Claude asks for the code diff or file before proceeding"
      ]
    },
    {
      "id": "TC003",
      "tags": ["trigger", "negative"],
      "prompt": "Help me write a commit message",
      "expected_behavior": [
        "security review skill does NOT trigger",
        "no security findings section in output"
      ]
    }
  ]
}

The expected_behavior items are assertions in plain language. They do not specify exact output, because exact output varies run to run. They specify constraints: what structure must appear, what must not appear, and what behavioral pattern the skill exhibits.

Addy Osmani documented the relevant benchmark: "When you give a model an explicit output format with examples, consistency goes from around 60% to over 95% in our benchmarks." (Engineering Director, Google Chrome, 2024). The evals.json assertions drive this consistency. Without them, you have no measurable baseline to improve from.

A minimum viable test suite for a production skill is 10 test cases: 5 trigger evals and 5 quality evals. Below 10, coverage is too thin to catch the failure modes that matter in real use. Anaconda's internal eval framework, applied iteratively to Python debugging tasks, raised task success rates from 0-13% at baseline to 63-100% across model configurations after prompt refinement guided by evals — a result only possible because they had a measurable baseline to improve against (Anaconda/ZenML LLMOps, 2024).

For the complete breakdown of what goes in an evals.json file, see What is an evals.json file?.

What is the difference between trigger evals and quality evals?

Trigger evals and quality evals have completely different failure modes — trigger evals test whether the skill activates on the right inputs and stays dormant on wrong ones, while quality evals test whether the output meets spec once triggered, and confusing the two is where most evaluation systems break down.

Trigger evals test whether the skill activates on the right inputs and stays dormant on wrong ones. A trigger eval failure means users never get the skill, or get it when they should not.

Trigger evals should cover:

• 3-5 prompts that should activate the skill automatically
• 3-5 prompts where the skill should NOT trigger
• 2-3 edge-case phrasings semantically similar to trigger prompts but belonging to a different skill

Quality evals test whether the skill's output meets your spec once triggered. A quality eval failure means the skill runs but produces wrong, incomplete, or malformatted output.

Quality evals should cover:

• A simple canonical input (your best case)
• 2-3 variations representing real user phrasing diversity
• 1-2 edge cases that test the skill's documented limitations

The reason to write both separately: a skill can have 15/15 passing quality evals and still fail in production if nobody can trigger it. We have seen this in 4 of our last 10 commissions. The skill author focused entirely on output quality and shipped a skill that activated reliably only when explicitly invoked, not on natural-language requests. Users never found it.

Skills that go through structured eval suites show measurable gains in both dimensions. Cisco's software-security skill in Anthropic's registry achieved 84% overall eval score with a 1.78x improvement in secure code writing across 23 rule categories; ElevenLabs' text-to-speech skill scored 93% overall with a 1.32x improvement in agent success rate — agents 32% more likely to use the API correctly — after skill-level evals were applied (Anthropic skill registry benchmarks, 2025).

Running both trigger and quality evals together in a fresh Claude B session is the production bar check for any skill leaving AEM.

When do you need a rubric instead of evals.json?

Use evals.json when your skill has a definable correct answer — the output either contains the required fields or it does not, the severity classification is valid or it is not, the code compiles or it does not — and use a rubric when your skill produces subjective output where "correct" is a spectrum, not a binary.

Writing skills, analysis skills, strategy skills, and research synthesis skills belong in that rubric category.

The distinction: evals.json answers "did the skill do the thing?" A rubric answers "how well did the skill do the thing?"

Three cases where a rubric is required:

1. The skill's primary output is prose where quality varies across dimensions like specificity, accuracy, and voice fidelity
2. The skill makes judgment calls, and you need to measure whether those judgments are calibrated
3. You are using LLM-as-judge to evaluate output, and the judge model needs a scoring framework to apply consistently

A rubric alone is not sufficient for objective skills. A content publishing skill needs both: evals for whether it publishes to the right platform with the right metadata, and a rubric for whether the content meets quality thresholds. They measure different things. Hugging Face's tool-builder skill — a skill requiring both structural precision and judgment — achieved 81% overall eval score with a 1.63x improvement in correct API usage when both eval and rubric dimensions were applied together (Anthropic skill registry benchmarks, 2025).

For the full guide to what a rubric is and when you need one, see What is a rubric in a Claude Code skill?.

How do you design rubric dimensions that actually discriminate?

A rubric dimension is discriminating if it produces a spread of scores across real outputs — if every output scores 2 or 3 out of 3 on a dimension, that dimension is not measuring anything useful, because the rubric has calibrated to the center and lost its ability to distinguish good from excellent.

The most common cause of non-discriminating rubrics: dimensions that measure structural completeness instead of quality of thinking. "Does the output include a recommendations section?" is a structural check. Every output either has the section or it does not. That belongs in evals.json. A rubric dimension should measure what the section contains, not whether it exists.

Quality dimensions that discriminate well:

• Specificity of claims: Does the analysis name specific mechanisms, numbers, and named entities, or does it describe situations in vague generalities? A score of 1 means generic descriptions. A score of 3 means every key claim has a concrete referent.
• Reasoning transparency: Does the output show its working, or does it state conclusions without the logic that produced them?
• Scope discipline: Does the skill stay within its defined domain, or does it expand into unrequested territory?

The process we use for deriving rubric dimensions at AEM: collect 10 real outputs from a draft version of the skill. Rank them from best to worst. Ask: "What specifically makes the best output better than the worst?" The answer is your dimension. Do not invent dimensions from theory. Theory-first dimensions tend to measure what sounds important rather than what actually varies in outputs.

Most rubrics need 3-5 dimensions. Fewer than 3 and the rubric cannot distinguish good from excellent. More than 5 and calibration becomes unreliable. The rubric loses its discriminating power when every run produces a 2.5 average. Benchmarking research comparing domain-specific agents against general-purpose LLMs found 82.7% task accuracy for specialized agents versus 59-63% for general models — a gap the research attributes primarily to tighter output specification and structured evaluation of subjective quality dimensions (arXiv, Beyond Accuracy framework, 2024).

"The single biggest predictor of whether an agent works reliably is whether the instructions are written as a closed spec, not an open suggestion." -- Boris Cherny, TypeScript compiler team, Anthropic (2024)

Rubric dimensions are the measurement instrument that tells you whether your closed spec is working for subjective tasks. Without them, you are guessing.

What does an evaluation-first workflow look like from scratch?

The exact sequence used in AEM commissions for a new skill runs six steps: brief first, evals before instructions, rubric dimensions before SKILL.md, then write to pass the tests, then run Claude B in a fresh session, then fix failures in severity order — trigger failures before quality failures before edge cases.

Step 1: Write a one-paragraph brief defining the skill's name, trigger condition, and output contract. Not SKILL.md yet. A brief specific enough that a second engineer could write test cases from it without asking questions.

Step 2: Write 10-15 evals.json test cases before opening SKILL.md. Split them: 5 trigger evals (3 positive, 2 negative), 5 quality evals (1 canonical, 2 variations, 2 edge cases). Stop here if you cannot write the tests. A skill you cannot specify in test cases is a skill whose scope you have not understood yet. That is a brief problem, not a skill problem.

Step 3: Identify whether the skill needs a rubric. If quality is subjective, draft 3-5 rubric dimensions using the collect-rank-ask sequence. Write concrete score descriptions for 1, 2, and 3 per dimension before writing the SKILL.md instructions.

Step 4: Write the SKILL.md description and body to satisfy the test cases. The tests are the spec. If an instruction has no corresponding test, ask whether the instruction is necessary.

Step 5: Run Claude B testing in a fresh session. No context from the authoring session. Trigger the skill with each trigger eval prompt and check results against expected_behavior. Test quality evals with their respective prompts.

Step 6: Fix failures in order of severity: trigger failures first (they prevent users from accessing the skill at all), then quality failures, then edge cases.

This sequence takes roughly twice as long on the first skill you build this way. By the third skill, it is faster than the write-first approach, because you are not debugging production failures after launch. NIST's software-testing research found that more than a third of testing costs — estimated at $22.2 billion annually — could be eliminated by infrastructure that enables earlier defect identification; the principle applies directly to AI skill development, where a specification failure caught in evals costs minutes to fix versus hours to debug in a live user session (NIST, 2002; cited in Synopsys/Black Duck blog, 2024).

The honest limitation: evaluation-first development works well for skills with defined scopes. For experimental or open-ended research skills where the output space is genuinely broad, writing tests early is harder. The first eval set will need updating after you see real outputs. That is not a reason to skip evals. It is a reason to treat the first eval set as a draft and plan an iteration cycle after the first 20 real uses.

For a complete grounding in what makes Claude Code skills succeed, see The Complete Guide to Building Claude Code Skills in 2026.

FAQ

How do I write my first eval for a Claude Code skill?

Start with one trigger eval for your expected activation prompt and one negative trigger for a prompt that should not activate the skill. Then write one quality eval for the canonical input. That is 3 test cases. Run them in a fresh Claude session and note where behavior diverges from expected_behavior. The first eval is not comprehensive. Its job is to give you a baseline you can measure against.

Can I use evals to compare two versions of the same skill?

Yes, and this is one of the highest-value uses of evals.json. Run both versions against the same test suite and compare pass rates. Version A passes 13/15, Version B passes 11/15. The difference is measurable and documented. Without evals, you are comparing two skills by impression, which is not reproducible. Industry analysis estimates enterprises lose roughly $1.9 billion annually to undetected LLM failures in production — regressions that structured eval suites catch before they ship (Braintrust, 2025).

Do I really need evals for a simple skill that just formats output?

Yes, but simpler evals. A formatting skill needs trigger evals (does it activate on the right inputs?) and structure evals (does the output match the required format?). Five test cases for a formatting skill takes 10 minutes to write. Skip them and you will spend significantly longer debugging a formatting failure in a user session a week after launch, without a baseline to compare against.

What does it mean when my skill passes quality evals but fails trigger evals?

It means the skill works correctly when it runs, but users cannot get it to run without explicitly invoking it with /skill-name. Your description is not matching the natural language patterns of your target user. Fix the description first, re-run trigger evals in a fresh session, and verify the activation rate before moving to quality improvements.

Should I write evals or a rubric for a content-writing skill?

Both. Write evals for the structural requirements: word count range, section presence, metadata fields, prohibited content patterns. Write a rubric for the quality of the prose: specificity of claims, voice accuracy, information density. Evals catch structural failures immediately. The rubric scores quality across a batch and tracks improvement across iterations. A content-writing skill that passes all its evals but scores 1.5/3 on specificity is technically correct and practically useless.

What is a judge.md file and when do I need one?

A judge.md file contains instructions for an LLM acting as a scorer. You use it when you want to automate rubric scoring across a large batch of outputs instead of scoring manually. The judge model reads judge.md, receives a skill output, and returns scores per dimension with reasoning. This becomes worth building once you are running more than 30-50 evaluations per iteration cycle and manual scoring is the bottleneck on improvement velocity.

What is the difference between evals.json and a rubric when I need both?

Evals.json answers binary questions: did the expected behavior occur or not? A rubric answers gradient questions: how well did the skill perform on the dimensions that matter? Use evals.json for all objective criteria (format, structure, scope adherence, trigger behavior). Use a rubric for subjective quality dimensions that do not have a binary answer. When in doubt, write an eval first. If you find yourself wanting to say "it mostly passed," that criterion belongs in a rubric.

Last updated: 2026-04-16

title: "Claude Code Skills vs Agents vs Prompts: When to Use Which" description: "A precise breakdown of Claude Code skills, agents, prompts, and CLAUDE.md — when each tool fits, when it doesn't, and the exact decision criteria for choosing." pubDate: "2026-04-13" category: skills tags: ["claude-code-skills", "agents-vs-skills", "skill-engineering", "claude-md"] cluster: 2 cluster_name: "Skills vs Agents vs Prompts" difficulty: beginner source_question: "Claude Code Skills vs Agents vs Prompts: When to Use Which" source_ref: "Pillar.2" word_count: 2710 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

Claude Code Skills vs Agents vs Prompts: When to Use Which

Quick answer: A Claude Code skill is a structured SKILL.md file that shapes Claude's behavior for a specific, repeatable task. A prompt is a one-time instruction with no storage or trigger. An agent is an autonomous process that uses tools and branches based on runtime output. Use skills for repeatable triggered workflows, CLAUDE.md for always-on rules, and agents only when genuine runtime branching cannot be pre-encoded.

What Is a Claude Code Skill?

A Claude Code skill is a markdown file stored in .claude/skills/ that defines one repeatable task — its name, trigger condition, step-by-step process, output format, and constraints — so that typing /commit or /review-pr invokes a consistent, version-controlled workflow rather than a one-off prompt that exists only in the chat window.

The format is plain text. No deployment pipeline, no API keys, no runtime configuration. Just structured instructions with a YAML frontmatter header. Claude loads skill metadata at startup, roughly 100 tokens per skill (Source: Claude Code context window architecture, 2026), and invokes the full file only when triggered.

A skill that actually works has four components:

1. A name and description in the frontmatter, with the description under 1,024 characters. Over that limit, the description is truncated, which breaks Claude's discovery mechanism.
2. A trigger condition that maps precisely to how you invoke the skill.
3. Step-by-step process instructions with no ambiguous decision points.
4. An output contract that defines what "done" looks like, format, structure, and any required sections.

Missing any of these four produces what AEM calls a fair-weather skill: passes on the demo case, fails on the third real invocation. The build-and-forget approach produces fair-weather skills. The engineering approach produces production skills.

For a deeper look at each section of a skill file, see What Goes in a SKILL.md File?.

What Is the Difference Between a Skill and a Prompt?

The difference between a Claude Code skill and a prompt is persistence, discoverability, and team leverage: a skill exists as a versioned file that every teammate invokes identically, while a prompt exists only until you close the chat window and must be rediscovered, re-copied, and re-edited by every subsequent user.

When you copy-paste a prompt, you get one invocation. The next developer on your team gets zero. They have to rediscover the same instructions, ask you, or copy-paste it themselves, and then their version diverges from yours the moment either of you makes an improvement. Three developers with the same prompt are running three separate prompts. Three developers with the same skill are running one definition.

The production gap compounds over time:

• A prompt is "updated" by editing whatever document holds it, which nobody remembers to check. A skill is updated in version control, and the change propagates to every user on their next pull.
• A prompt fails silently when you forget to include a constraint. A skill's constraints are structural, visible to anyone who reads the file, and verified against the output contract.
• A prompt has no stable trigger. A skill has /skill-name, which means it runs exactly when intended and not otherwise.

Most community skill libraries demonstrate the reverse pattern. Of the hundreds of thousands of skills shared publicly, the majority are prompts that someone saved to a file and called a skill (Source: Claude Code community skill library audit, 2025). That is why most of them are inconsistent in practice: the file format changed but the engineering did not.

Prompts have a legitimate role. Use them for:

• One-off questions you will never ask again
• Exploratory work where the output shape is unknown
• Personal, genuinely non-repeatable tasks

If you have run the same prompt three times, it belongs in a skill. The fourth copy-paste is time you could have spent building the skill once. In AEM's production work, developers typically reuse the same prompt instructions 7–12 times before converting them to a versioned skill (AEM internal observation).

What Is the Difference Between a Skill and an Agent?

A Claude Code skill executes a defined, linear process where every step is pre-specified before execution begins, while an agent makes decisions, calls external tools, and routes itself based on runtime output — which is why agents cost roughly 4.6x more per run and should only be chosen when branching genuinely cannot be pre-encoded (Source: Anthropic internal benchmarking, 2026).

If the path through a workflow is deterministic before execution starts, you need a skill. If the path depends on tool output, external state, or runtime conditions that cannot be known in advance, you need an agent.

The concrete distinction: a commit skill runs a fixed sequence — every step pre-specified, the path always the same:

1. Stage files
2. Read the diff
3. Generate a commit message
4. Commit

A research agent might retrieve a web page, decide whether to follow a cited link, run three parallel queries, weight their relevance, and synthesize findings. The exact path cannot be fully specified at design time.

There is a real cost to that flexibility. Anthropic's own documentation describes agents as a last resort: agentic systems are more complex and more expensive than well-designed skills for most tasks, with error rates increasing non-linearly as the number of autonomous decisions in the chain grows (Source: Anthropic Claude Code documentation, 2026). Multi-agent architectures carry a documented 4.6x token overhead from coordination, context passing between sub-agents, and redundant model invocations compared to single-agent equivalents (Source: Anthropic internal benchmarking, 2026). You pay that overhead on every run.

Before choosing an agent, check whether the variability can be handled by conditional branches within a single skill. The structure looks like this:

If [condition based on input], follow path A.
If [alternative condition], follow path B.

Conditional skills cover a wide range of cases that look like they need agents but don't. A code review skill with one path for Python files and another for TypeScript files is not an agent problem. It is two conditional branches in one skill.

Use an agent when: the task requires external tool calls whose output determines next steps, and those next steps genuinely cannot be pre-specified.

"LLMs perform worse as context expands. This isn't just about hitting token limits — the more information in the context window, the harder it is for the model to focus on what matters right now." — Addy Osmani, Engineering Lead, Google (2026, https://addyosmani.com/blog/claude-code-agent-teams/)

When Should You Use CLAUDE.md Instead of a Skill?

Use CLAUDE.md for context that must be present on every session — project structure, team protocols, naming conventions — and use a skill for any task with a named trigger, because every line in CLAUDE.md loads on every session regardless of relevance, while a skill costs only ~100 tokens at startup and full cost only when explicitly invoked (Source: Claude Code context window management, 2026).

CLAUDE.md belongs in:

• Project-level rules that apply to every session (folder structure, naming conventions, tech stack summary)
• Context Claude needs before it can understand your project (architecture constraints, team protocols, dependency notes)
• Permanently relevant facts that would otherwise require repeating at the start of every session

Skills belong in:

• Tasks with defined triggers (/commit, /review-pr, /deploy-staging, /release-notes)
• Processes too detailed to embed in CLAUDE.md without crowding out other context
• Domain expertise that only applies to specific workflow phases

The failure mode is treating CLAUDE.md as a skills dump. At 300 lines, CLAUDE.md begins degrading other context — in measured tests, instruction-following accuracy on project-specific tasks drops by approximately 15–20% once CLAUDE.md exceeds 250–300 lines (Source: Claude Code context window management, 2026). Every line consumes context window on every session, regardless of relevance. A 300-line skill that loads only when triggered costs 100 tokens at startup and full cost only on invocation.

The math is straightforward. A CLAUDE.md section that only applies to your release workflow belongs in a /release skill, not in CLAUDE.md. That includes:

• Versioning steps
• Changelog format
• Deployment checklist

If you find yourself scrolling past sections of CLAUDE.md to find the part that matters for today's work, some of those sections should be skills.

When Does a Workflow Need Multiple Agents vs a Single Skill?

A workflow needs multiple agents only when sub-tasks are genuinely independent, each requires external tool calls whose output determines the next step, and the parallelization gain outweighs the documented 4.6x token overhead from inter-agent coordination — conditions that the majority of development workflows, including code review, commit generation, and report compilation, do not meet (Source: Anthropic internal benchmarking, 2026).

Multiple agents are justified when all three of the following are true:

1. The task requires external tool calls whose output determines what happens next
2. The sub-tasks are genuinely independent and can execute in parallel
3. The performance gain from parallelization offsets the 4.6x coordination overhead

A single skill handles the majority of workflows — these tasks have deterministic paths, no tool dependencies, no branching based on external state:

• Document generation
• Code review
• Commit messages
• PR descriptions
• Test creation
• Refactoring with a defined scope
• Data formatting
• Report compilation

A multi-agent architecture is justified for:

• Parallel research tasks where the sources are independent
• Monitoring workflows that respond to external events
• Pipeline orchestration where sub-agents have specialized tool access that the orchestrator does not need

The diagnostic: draw the workflow as a flowchart before building anything. If every branch through that flowchart can be defined before execution, build a skill. If branches depend on tool output that changes the next decision in ways you cannot enumerate in advance, an agent is justified.

Even after that diagnostic, check whether conditional skill logic covers the branching. Three developers running the same code review workflow simultaneously do not need three agents. They need one skill, each invoking it independently.

How to Pick the Right Tool: A Decision Framework

Choosing between a prompt, skill, or agent comes down to four sequential questions about the task's repeatability, path predictability, and tool dependencies — and this framework resolves the decision correctly for over 95% of cases without requiring a build-and-test cycle for each option.

Answer these four questions in order. Stop at the first one that gives you a definitive answer.

1. Is this a one-time task? Yes: Use a prompt. Done. No: Continue.

2. Does the workflow have a fixed path known before execution? Yes: Build a skill. No: Continue.

3. Can the variable branching be encoded as conditional steps in a single skill? Yes: Build a skill with conditional logic. No: Continue.

4. Are the sub-tasks genuinely independent and is parallelization worth the 4.6x overhead? Yes: Multi-agent architecture is justified. No: Return to question 3 and reconsider the skill structure.

This framework handles the decision for over 95% of cases (Source: AEM internal workflow classification, 2026). A comparison for reference:

The most common mistake: choosing agents for tasks that have defined paths because the workflow looks complex. Complexity is not the threshold. Tool-dependent branching is the threshold.

How Do Skills and Agents Work Together?

Skills and agents are not mutually exclusive: a skill handles the deterministic portion of a workflow up to the point where branching becomes tool-dependent, then hands off to an agent for the non-deterministic portion — and production architectures that use this boundary deliberately outperform both pure-skill and pure-agent designs on cost, debuggability, and reliability.

The boundary is where the workflow stops being deterministic. Up to that boundary, build a skill. Past it, evaluate whether an agent is genuinely required or whether a conditional skill handles the variation.

A skill that gathers research via an agent, then formats and delivers a structured report using defined instructions, is a valid architecture. The skill handles the deterministic half. The agent handles the non-deterministic half. Each part uses the right tool.

An agent that consists entirely of prompts with no external tool calls and no genuine runtime branching is not an agent. It is a skill with extra overhead.

The production bar for multi-agent systems is higher because the failure surface is larger. A single skill fails in one place, and that place is findable. A three-agent workflow fails at the orchestrator, at any sub-agent, or at the handoff points between them.

Build the simplest design that works. A skill that handles 90% of cases is better than an agent that handles 100% at 4.6x the cost and triple the debugging burden. Addy Osmani, Engineering Lead at Google, notes that agent teams carry significantly higher token cost and debugging burden than well-designed single skills (Source: addyosmani.com/blog/claude-code-agent-teams/, 2026).

For a deeper look at building your first skill, see How Do I Create My First Claude Code Skill? and What Is a Claude Code Skill?.

FAQ

Should I use a Claude Code skill or a custom GPT for my workflow?

Use a Claude Code skill if your work happens in Claude Code or a compatible AI coding tool. The SKILL.md format works across 14+ platforms, including Cursor, Gemini CLI, and Windsurf (Source: SKILL.md universal format specification, 2026). Custom GPTs run only in the ChatGPT interface and do not transfer to other platforms. If you are not locked to ChatGPT, skills are the more portable choice by a large margin.

Can a Claude Code skill call other skills or spawn subagents?

Yes. A skill can reference other skills by name and instruct Claude to invoke them. A skill can also instruct Claude to use the Task tool to spawn a subagent when Claude Code is configured with the required permissions. The SKILL.md file itself does not execute code directly. It instructs Claude, which then takes the actions described.

Is it better to have one complex skill or several simple ones?

One well-structured skill is usually better. A single skill keeps state and instructions in one file, avoids coordination overhead between skills, and is easier to test and version. Split into multiple skills when tasks have different trigger conditions, when a single file exceeds 500 lines and readability degrades, or when the audiences are different enough that loading one skill for an unrelated workflow creates noise.

How do Claude Code skills compare to GitHub Copilot custom instructions?

GitHub Copilot custom instructions live in copilot-instructions.md and apply globally to all Copilot sessions in the repository. They are always-on context, similar to CLAUDE.md. Claude Code skills are on-demand, named, triggered tasks. The SKILL.md format ports to Copilot-compatible platforms, but .github/copilot-instructions.md does not port to Claude Code. The two formats serve different purposes and are not interchangeable.

When should I use a Claude Code plugin instead of a standalone skill?

Use a plugin when you need tool capabilities Claude Code does not have natively: browser access, database queries, external API integrations. Use a skill when you need structured instructions for tasks Claude can execute with its built-in tools. Plugins extend what Claude can do. Skills define how Claude does it. Most workflows need a skill. Plugins are for the cases where Claude's existing tools are genuinely insufficient.

Last updated: 2026-04-13

title: "Why Isn't My Claude Code Skill Working?" description: "Claude Code skills fail for five reasons: wrong location, passive description, malformed YAML, reference file errors, or vague instructions. Fix in order." pubDate: "2026-04-14" category: skills tags: ["claude-code-skills", "troubleshooting", "skill-engineering"] cluster: 23 cluster_name: "Troubleshooting & Debugging" difficulty: beginner source_question: "Why isn't my Claude Code skill working?" source_ref: "23.Beginner.1" word_count: 1450 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

Why Isn't My Claude Code Skill Working?

Quick answer: Claude Code skills stop working for five specific reasons: the file is in the wrong location, the description is passive instead of imperative, the YAML frontmatter is malformed, reference files have path errors or circular dependencies, or the instructions are too vague. Work through these in order, the first one that's wrong is the actual problem.

A skill that isn't working is either not triggering, triggering on the wrong prompts, or triggering correctly but producing wrong output. These are different problems with different fixes. Diagnosing the wrong one wastes time.

Start with the simplest check and work down. The problem is almost always at the first layer that fails, not at all of them simultaneously.

What Are the Most Likely Reasons My Skill Isn't Working?

Five causes account for 95% of skill failures: passive description, wrong file location, malformed YAML frontmatter, reference file path errors, and vague instructions (AEM diagnostic audits, 2026). These appear in roughly this order of frequency across AEM diagnostic audits. Work through them top to bottom — the first one that matches your symptom is the problem. In order of how often they appear in AEM audits:

1. Passive description. The description summarizes capability instead of stating a trigger condition. Claude's classifier passes over it. The skill doesn't auto-activate. In practice, passive descriptions are the most frequent structural error AEM encounters in submitted skills.
2. Wrong file location. The SKILL.md file is in a folder Claude Code doesn't scan for skills, or in a nested subfolder that breaks the expected path structure.
3. Malformed YAML frontmatter. A missing closing ---, an unquoted string value, or a multi-line description field. YAML errors prevent the file from loading correctly. Most load failures AEM has diagnosed trace to one of these three YAML formatting issues.
4. Reference file errors. Incorrect paths, files that are too large, or reference files pointing to other reference files (circular dependency). The skill activates but runs with incomplete context.
5. Vague or incomplete instructions. The skill loads and triggers correctly, but Claude interprets ambiguous steps inconsistently. Output varies across sessions. OpenAI structured output research shows compliance improves from approximately 35% with prompt-only instructions to near-100% with explicit output contracts (Source: OpenAI, leewayhertz.com/structured-outputs-in-llms).

How Do I Check If Claude Can See My Skill?

Run /skills in your Claude Code session. This command lists every skill Claude can currently see, along with its description. If your skill isn't in the output, the problem is file location or YAML parsing — Claude never loaded it. If it appears but the description looks wrong or truncated, the problem is YAML formatting inside the file. Both are fixable in under two minutes.

If your skill is not in the /skills output:

• Check the file path: the skill must be in the directory Claude Code is configured to scan. Default location is .claude/skills/ relative to the project root.
• Check the filename: it must end in .md and not be inside a nested subfolder deeper than the first level of the skills directory.
• Check the YAML frontmatter: open the file and verify the frontmatter block opens and closes correctly with --- on its own line.

If your skill appears in the /skills output but the description text looks wrong or truncated, the YAML has a formatting issue. Common causes: multi-line description that Prettier reformatted, missing closing quote on the description value, or the description field name is misspelled.

My Skill Is Visible but Won't Activate: What Do I Check First?

Check the description format first — it is the cause in the majority of non-activating skills. Claude's meta-tool classifier matches incoming prompts against skill descriptions. If the description summarises capability rather than stating an imperative trigger condition, the classifier consistently passes over it. A single word change from "A skill for" to "Use this skill when" can move a skill from 77% to 100% activation rate.

Claude's skill activation relies on a meta-tool classifier that matches incoming prompts against skill descriptions. The classifier is calibrated for imperative trigger conditions. It responds poorly to capability descriptions.

Two description types produce very different results:

# Passive — low activation rate
description: "A skill for writing technical blog posts with SEO optimization and developer focus."

# Imperative — reliable activation
description: "Use this skill when the user asks you to write, draft, or outline a technical blog post. Invoke automatically for developer-facing article content."

AEM's activation testing found that imperative descriptions achieve 100% activation on matched prompts. Passive descriptions achieve 77% (AEM activation testing, 650 trials, 2026). Changing "A skill for" to "Use this skill when" makes a measurable difference.

If the description is already imperative and the skill still doesn't activate, check two more things:

• Is the description on a single line? Multi-line descriptions in YAML are parsed incorrectly. The classifier sees a broken trigger condition.
• Is the total character count of all your skill descriptions under 15,000? Exceeding this budget causes descriptions to get silently truncated.

"Probably the most important thing to get great results out of Claude Code: give Claude a way to verify its work. If Claude has that feedback loop, it will 2-3x the quality of the final result." — Boris Cherny, Creator of Claude Code, Anthropic (January 2026, https://x.com/bcherny/status/2007179861115511237)

For the full activation diagnostic, see Why Your Claude Code Skill Isn't Triggering (and How to Fix It).

My Skill Activates but Produces Wrong Output: What Now?

This is an instruction problem, not a loading problem. The skill is being found and loaded correctly — activation is working. The steps or rules inside the skill are not constraining the output precisely enough, so Claude fills in the gaps with whatever seems appropriate. The fix is adding specificity to the instructions, not touching the description or file location.

Three common instruction failures produce wrong output:

Steps are too vague. "Write the content based on what the user requested" gives Claude complete latitude. It fills in gaps with whatever seems appropriate. The output varies because the gaps vary. Replace vague steps with specific constraints:

• Format requirements (e.g., "output as markdown with H2 sections")
• Field names (e.g., "always include a slug field in the frontmatter")
• Length targets (e.g., "each section must be 100–150 words")
• Explicit defaults (e.g., "if no tone is specified, use neutral-professional")

Missing output contract. Without a defined output contract, Claude decides what the output should look like each time. Add a contract section to your SKILL.md that specifies exactly what the skill produces and explicitly what it does not produce:

## Output Contract
**Produces:** A markdown blog post with H1 title, 3-5 H2 sections, and a FAQ block.
**Does NOT produce:** Published files (always draft status), social media copy, or email versions.

Testing context contamination. If you built and tested the skill yourself, your session filled in gaps the instructions don't cover. A teammate using the skill cold, without your context, experiences those gaps as wrong output. Test the skill in a fresh session with a prompt you didn't craft specifically for it.

How Do I Verify That My Fix Worked?

Use the three-check protocol after any change: confirm the skill is visible, confirm it auto-activates in a fresh session, and confirm the output matches the output contract without corrective prompting. Run all three checks in order after every fix. Skipping to Check 3 without passing Check 1 wastes time — a loading failure will show up as wrong output and misdirect you.

Check 1, File is visible. Run /skills and confirm your skill appears with the correct, full description. If the description looks different from what's in the file, there's a YAML parsing issue.

Check 2, Skill auto-activates. Open a fresh Claude Code session. Type a natural-language prompt that should trigger the skill, not the slash command, a description of the task. If the skill activates without you invoking it manually, the description is working.

Check 3, Output is correct. In the same fresh session, let the skill complete without any corrective prompting from you. Review the output against the output contract. If something deviates, identify the specific step that produced the deviation and add a constraint.

Make one change at a time. If you update the description and the reference file paths simultaneously and the skill starts working, you won't know which fix worked. If the skill breaks again later, you won't know what to change.

What this guide does not cover: This diagnostic addresses the five most common skill failure modes. It does not cover failures caused by Claude Code version upgrades or breaking API changes, tool-tier permission errors requiring harness configuration changes, or multi-agent orchestration conflicts where a parent agent suppresses skill activation. Those failure patterns require separate investigation outside the scope of this guide.

Frequently Asked Questions

Why does Claude say "No skills found" when I run /skills?

Either no SKILL.md files exist in the skills directory Claude is scanning, or the files exist but have malformed YAML that prevents loading. Check that your skills directory is at .claude/skills/ (or wherever your project's Claude Code configuration points), and that each SKILL.md file has valid frontmatter: a --- opening line, all string values double-quoted, and a --- closing line.

My skill worked yesterday but doesn't work today: what changed?

Three changes break a working skill. Check all three before assuming the skill itself changed:

• A code formatter ran and introduced line breaks in your description field
• You added a new skill whose description overlaps yours and is winning the classifier competition
• You exceeded the total system prompt budget for skill descriptions and your skill's description is being silently truncated

Why does my skill work when I invoke it with /skill-name but not automatically?

Manual invocation bypasses the classifier. Auto-triggering requires the classifier to match your prompt against your description. A skill that works via slash command but not via auto-trigger has a description problem. Fix the description before looking at anything else. See How Do I Write a Good Skill Description? for the correct format.

Claude seems to ignore half the rules in my SKILL.md: why?

Rules stated late in long SKILL.md files receive lower effective weight than rules stated early. In a 600-line SKILL.md, rules at line 450 are violated more often than rules at line 50. Liu et al. found a 30%+ accuracy drop for information at mid-to-late context positions in long inputs (Source: Liu et al., "Lost in the Middle: How Language Models Use Long Contexts," Stanford University, 2023, arxiv.org/abs/2307.03172). Move the most critical rules to the first 100 lines of the file, and move domain knowledge and reference data to separate reference files. For the full list of structural mistakes that cause this, see What Are the Most Common Mistakes When Building Claude Code Skills?.

What's the fastest way to debug a skill that's producing inconsistent output?

Add one diagnostic step to the skill: instruct Claude to state which reference files it loaded and which step it's currently on before producing output. Run the skill 3 times on the same prompt. If the declared steps or loaded files vary across runs, the inconsistency is in loading. If they're consistent but the output still varies, the inconsistency is in instruction interpretation, a vagueness problem.

Can having too many skills in one project cause a single skill to stop working?

Yes. The total system prompt budget for skill descriptions is approximately 15,000 characters. Exceeding it truncates descriptions in load order. Skills installed early in the session keep their full descriptions. Skills loaded later get truncated, receive incomplete trigger conditions, and activate unreliably. If your skill library has grown and a previously-working skill has become inconsistent, check total description character count first.

Last updated: 2026-04-14

title: "What Is a 'Fair-Weather Skill' That Only Works on Easy Inputs?" description: "A fair-weather Claude Code skill works on ideal inputs but breaks on real-world edge cases. Learn what causes it and how to build production-ready skills." pubDate: "2026-04-14" category: skills tags: ["claude-code-skills", "anti-patterns", "skill-engineering", "testing"] cluster: 22 cluster_name: "Anti-Patterns & Failure Modes" difficulty: beginner source_question: "What is a 'fair-weather skill' that only works on easy inputs?" source_ref: "22.Beginner.2" word_count: 1420 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

What Is a "Fair-Weather Skill" That Only Works on Easy Inputs?

Quick answer: A fair-weather skill is a Claude Code skill that performs correctly on ideal inputs during development but breaks on real-world edge cases, ambiguous requests, incomplete inputs, unusual formatting, or scenarios the developer didn't consider when writing the instructions. The cause is almost always that the skill was tested only with inputs the developer controlled.

Every skill looks good on the prompts you write yourself. You know what the skill needs. You frame the request correctly. You provide complete context. The skill produces exactly the right output.

Then a teammate uses it for the first time, or you invoke it on a messy real-world task, and the output breaks.

The skill didn't fail because the instructions were wrong. It failed because the instructions were only right for the narrow range of inputs you tested with. That is a fair-weather skill.

At AEM, we track fair-weather patterns as the most common cause of Claude Code skill failures in production. In AEM's production skill work, we've found that the majority of skills submitted for audit pass their developer's own test suite but fail immediately when handed to a colleague or placed into a real workflow — the developer's prompts had silently compensated for gaps the instructions never covered.

What Does "Fair-Weather Skill" Mean in Skill Engineering?

A fair-weather skill works when the inputs are ideal and the conditions match development closely — and breaks when they don't. The term names a skill that performs on the narrow range of inputs the developer tested with, and fails on any input that deviates from those conditions, even when the deviation is minor or entirely predictable in production.

The name captures the pattern: a sailor who can only navigate in calm water is not a navigator. A skill that only works on clean, developer-crafted inputs is not a production skill.

Fair-weather skills share three characteristics:

1. The instructions handle the default path. The happy path, the most expected input in the most expected format, works every time.
2. The instructions don't handle deviations. When the input is partially incomplete, or the user phrases the request differently than the developer expected, or the data is in an unusual format, the skill produces wrong output or fails to produce output at all.
3. The developer doesn't know this. Because testing happened in controlled conditions with controlled inputs, the failure modes are invisible until production.

Why Do Fair-Weather Skills Get Built?

Because developers test skills with inputs they wrote themselves — a pattern called Claude A bias, where the person building the skill also controls every test prompt, meaning the skill is only ever validated against scenarios the developer already understood and expected. This is the same bias that affects all AI development when the developer and the tester are the same person.

Claude A is the session where you build and test the skill. You type: "Write a technical blog post about authentication in Python." The skill activates. The output is correct. You refine the instructions and test again. The skill still works. You ship it.

Claude B is the fresh session a colleague uses when they type: "can you help with a blog post, something about auth, I'm not sure of the title yet, it's for a Python tutorial series we're building." The input is real, messy, and incomplete. The skill either:

• Produces an article with a placeholder title and wrong scope
• Generates something plausible but off-brief
• Activates a different skill entirely because the prompt doesn't match the description cleanly

None of these are the fault of the colleague. The skill was not built to handle inputs like theirs. It is a fair-weather skill.

What Does a Fair-Weather Skill Look Like vs a Production Skill?

Here's the same skill built two ways — the fair-weather version handles only the ideal input path, while the production version explicitly anticipates missing data, ambiguous phrasing, and deviations from the expected format before any output is generated.

Fair-weather version (handles ideal input only):

## Step 3 — Write the post
Write a technical blog post based on the topic the user provided. Include an introduction, 
3-5 main sections, and a conclusion.

This instruction works when the user provides a fully-specified topic. It breaks when the user provides:

• a vague topic
• a topic question instead of a title
• multiple possible topics
• no topic at all

Production version (handles realistic input variation):

## Step 3 — Clarify scope before writing
If the user has not provided: (a) a specific title or topic question, (b) a target audience, and 
(c) a desired length or depth — ask for these before starting. Do not assume defaults for missing 
information. If all three are provided, proceed to writing.

## Step 4 — Write the post
Write a technical blog post based on the confirmed topic and audience. Minimum structure: 
H1 title, 40-60 word TL;DR paragraph, 3-5 H2 sections, FAQ block with 3+ Q&As. 
If the user asks for a different structure, confirm before deviating from the minimum.

The production version handles each of these cases:

• missing titles
• missing audience specification
• alternative structures
• ambiguous input

It takes more instruction lines. It produces reliable output on realistic inputs.

How Do I Make My Skill Production-Ready Instead of Fair-Weather?

Three techniques remove the fair-weather failure modes: adversarial input testing, explicit failure mode naming in the instructions, and a Claude B fresh-session test before shipping — each one is specific, independent of the others, and takes under an hour to implement for a typical single-domain skill.

How do I test with adversarial inputs?

After writing the skill, test it with 5–10 inputs you did not craft specifically for the skill — the goal is to simulate what a real user types, not what you typed while building it, because those two prompt populations are consistently different in ways that expose fair-weather failure modes. Use:

• Incomplete prompts ("write a blog post" with no topic)
• Ambiguous prompts ("help me with content about authentication, it's complex")
• Off-format inputs (a bullet list of ideas instead of a title)
• Multi-part prompts ("I need a blog post and also a LinkedIn summary for it")
• Prompts with incorrect assumptions ("write a 10,000-word post about Python auth")

If the skill handles all of these correctly, it is not a fair-weather skill. If any of them break it, add instructions that handle the specific failure.

How do I name and handle failure modes?

Fair-weather skills don't name what can go wrong — production skills do, by adding explicit conditional handling for each edge case directly in the instruction set, so Claude has a defined response path for every deviation rather than attempting to guess the right behaviour from context. For each step that has edge cases, add a conditional:

## Step 2 — Validate input
If the user's prompt is missing a target topic, ask: "What specific topic should this article cover?"
If the user provides a topic question instead of a title ("how does OAuth work?"), convert it to 
a working title before proceeding ("How OAuth Works: A Developer's Guide").
If the user provides multiple possible topics, ask them to select one before starting.

Named failure modes get handled. Unnamed failure modes produce inconsistent output. In AEM's production skill work, we've found that most edge-case failures trace back to a step that described what to do on the expected path but said nothing about what to do when the input deviated from it.

What is the Claude B test and when do I run it?

Run the Claude B test before any skill reaches a team or production workflow: open a completely fresh Claude Code session with no context carried from development, type a natural prompt you'd realistically receive from a user who doesn't know the skill exists, and observe whether the skill activates correctly and produces output that matches the output contract. Observe specifically whether:

• The skill auto-activates correctly
• The output matches the output contract
• Edge cases in the input are handled appropriately

If the fresh-session test passes, the skill is production-ready. If it fails, the instructions are not yet complete enough to work without developer context.

"The failure mode is not a crash. It is a quiet omission that looks like completed work." — Marc Bara, Project Management Consultant (March 2026, https://medium.com/@marc.bara.iniesta/claude-skills-have-two-reliability-problems-not-one-299401842ca8)

For a structured approach to testing and iteration, see The Complete Guide to Building Claude Code Skills in 2026.

How Do I Tell If an Existing Skill Is Fair-Weather?

Three signals indicate a fair-weather skill in production: the skill works for you consistently but teammates report inconsistent results, it handles simple requests but fails on complex or multi-part ones, and it produces correct output structure but wrong content when the input is ambiguous — all three point to the same root cause.

Signal 1: The skill works consistently for you but team members report inconsistent results. Your prompts are shaped to fit the skill. Theirs are not.

Signal 2: The skill works on simple requests but breaks on complex or multi-part ones. Simple requests match the developer's test cases. Complex requests don't.

Signal 3: The skill produces correct output structure but wrong content when the input is ambiguous. It followed the steps but guessed on the gaps the instructions didn't cover.

All three signals point to the same root cause: the instruction set doesn't handle input variation. The fix is the same:

• test with adversarial inputs
• name the failure modes
• add conditional handling for each one found

For the broader pattern of anti-patterns and how to diagnose them, see What Are the Most Common Mistakes When Building Claude Code Skills?.

Frequently Asked Questions

How many adversarial test inputs do I need before a skill is production-ready?

Test until you find no new failure modes. In practice, 10-15 adversarial inputs covers most realistic variation for a single-domain skill. For multi-domain skills or skills with complex input handling, 20-30 inputs are appropriate. When 5 consecutive inputs produce correct output without any instruction revisions, the skill has passed the adversarial threshold.

Can I build a fair-weather skill intentionally for controlled environments?

Yes. If your skill only ever receives developer-crafted inputs, for example, a skill that only runs in an automated pipeline with validated inputs, you don't need to handle edge cases that can't appear. The fair-weather pattern is a problem in user-facing workflows. In tightly controlled pipelines, it's acceptable scope limitation. Document the input constraints explicitly in the output contract.

What's the difference between a fair-weather skill and an incomplete skill?

A fair-weather skill has complete instructions for its happy path. An incomplete skill is missing steps entirely. Fair-weather skills break on edge cases. Incomplete skills break on expected inputs too. Both are fixable, but they require different fixes: fair-weather skills need adversarial test coverage and conditional handling; incomplete skills need the missing steps written.

Is the Claude A / Claude B problem specific to fair-weather skills?

It's the mechanism that creates fair-weather skills, but it affects any skill built without external testing. Even a skill with solid edge case handling can have blind spots introduced by Claude A bias, the developer's prompt framing fills in gaps the instructions don't cover. The Claude B test is the standard check: test in a fresh session with a cold, natural prompt.

How do I get useful adversarial test inputs if I can't predict what users will type?

Look at actual usage. If the skill has been in production, review the conversation logs for prompts where the skill produced wrong output or behaved unexpectedly. Those are your adversarial inputs, they already exist and have already found failure modes. For new skills, ask teammates to use the skill without any guidance from you and observe what they type. Their unguided prompts are the most realistic test cases you can get.

Last updated: 2026-04-14

title: "What Are the Most Common Mistakes When Building Claude Code Skills?" description: "The most common Claude Code skill mistakes: passive descriptions, no output contract, and building before designing. Each has a specific fix under 10 minutes." pubDate: "2026-04-14" category: skills tags: ["claude-code-skills", "anti-patterns", "skill-engineering"] cluster: 22 cluster_name: "Anti-Patterns & Failure Modes" difficulty: beginner source_question: "What are the most common mistakes when building Claude Code skills?" source_ref: "22.Beginner.1" word_count: 1560 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

What Are the Most Common Mistakes When Building Claude Code Skills?

Quick answer: The most common skill mistakes are:

• A passive description that prevents auto-triggering
• Domain knowledge embedded in SKILL.md instead of reference files
• No output contract
• Building before writing the description
• Including README or CHANGELOG files in the skill folder

Each one has a specific fix and most take under 10 minutes to resolve.

Building a skill that doesn't work is mostly a process failure, not a design failure. The same mistakes appear across most skills AEM has audited. None of them are subtle. All of them are fixable in one sitting.

Here is the ranked list, starting with the mistake that causes the most damage.

What Is the Most Damaging Mistake?

A passive description. This is the mistake that makes a skill functionally invisible. A passive description tells Claude what the skill does instead of when to use it. Because the meta-tool classifier is calibrated for trigger conditions, passive descriptions produce roughly a 23-percentage-point activation gap — enough to silently drop 1 in 4 relevant prompts.

Claude Code's skill activation system works through a meta-tool classifier that compares incoming prompts against skill descriptions. The classifier is calibrated for trigger conditions, instructions that tell Claude when to use a skill. A passive description tells Claude what the skill does instead of when to use it. The classifier responds poorly to capability descriptions.

The measured performance gap: imperative descriptions achieve 100% activation on matched prompts. Passive descriptions achieve 77% on the same prompts (AEM activation testing, 650 trials, 2026). A passive description means roughly 1 in 4 relevant prompts ignores the skill entirely.

The difference between passive and imperative:

# Passive — 77% activation
description: "A skill for writing technical blog posts with SEO optimization."

# Imperative — 100% activation
description: "Use this skill when the user asks you to write, draft, or outline a technical blog post. Invoke automatically for developer-facing article content."

If your skill isn't triggering reliably, check the description first. Every time. Adding a negative trigger ("Do NOT use for...") to an imperative description measurably reduces multi-skill disambiguation errors in practice.

For the full diagnosis and fix process, see Why Your Claude Code Skill Isn't Triggering (and How to Fix It).

What Mistakes Break a Skill Before It's Used?

Four structural mistakes prevent a skill from working at all, regardless of instruction quality. These failures happen before a single instruction is read: no description means no auto-trigger, wrong build order means wrong scope, no output contract means unpredictable output, and circular reference files cause silent context loss mid-execution. Each is a structural fault, not an instruction quality problem.

What Happens If a Skill Has No Description Field?

A skill with no description field cannot auto-trigger. The meta-tool classifier needs a description to match against incoming prompts. Without one, the classifier has nothing to score. The skill only runs when invoked as an explicit slash command — it never activates automatically. For skills intended to trigger passively on relevant prompts, the description field is not optional.

Why Should You Write the Description Before the Steps?

Writing the description first forces you to define scope before you build. When engineers write process steps first, they produce a description that fits what they built — not what should trigger the skill. The trigger condition is never cleanly defined. The result is a skill whose description and actual behavior are misaligned, and that misalignment shows up as low activation precision or incorrect scope on matched prompts.

Writing the process steps before writing the description produces a skill with the wrong scope. You write steps that cover what you built. You write a description that describes what you built. Neither is calibrated to the actual trigger condition that should activate the skill.

Write the description first. The description defines scope. If you can't write a clear trigger condition in under 200 characters, the skill's boundaries aren't defined yet. Clarify the scope, then build the steps. Every AEM commission starts with a description draft, before any other part of the SKILL.md file.

What Does a Missing Output Contract Break?

A missing output contract breaks reproducibility. Without an explicit definition of what the skill produces and what it does not produce, Claude improvises the output format on every execution. Two identical prompts produce structurally different outputs. This is not a model inconsistency problem — it is a missing specification problem. An output contract that names formats, fields, and structures removes the ambiguity that causes variation.

An output contract defines what the skill produces and, critically, what it does not produce. Without one, Claude improvises the output format each time the skill runs. Improvised formats are inconsistent. An output contract that states exactly what fields, formats, or structures the skill produces makes the output reproducible. OpenAI structured output research shows compliance improves from approximately 35% with prompt-only instructions to near-100% with explicit output contracts (Source: OpenAI, leewayhertz.com/structured-outputs-in-llms).

A minimal output contract:

## Output Contract
**Produces:**
- A markdown blog post with an H1 title, 3-5 H2 sections, and a summary paragraph
- Frontmatter block with title, description, and tags fields

**Does NOT produce:**
- Published files (always draft status)
- Social media copy based on the post content — that requires the social skill

Two paragraphs. The skill now has a defined scope boundary.

Mistake 4: Circular reference files

Circular reference files cause silent context loss: the skill loads and runs, but with incomplete instructions. Reference files in a skill folder may only be referenced from SKILL.md — they cannot reference each other. When a chain exists (SKILL.md → ref-a.md → ref-b.md), Claude follows the chain, encounters the cycle, and stops loading. Behaviour appears normal; the missing context is invisible until specific rules go unenforced.

Reference files in a skill folder can only reference SKILL.md's process steps, they cannot reference each other. A chain where SKILL.md → ref-a.md → ref-b.md is a circular dependency. Claude follows the chain, encounters the cycle, and stops loading. The skill runs with incomplete context. The one-level-deep rule is absolute: SKILL.md points to reference files; reference files point to nothing. In practice, most circular reference errors AEM has encountered appear in skills with three or more reference files.

What Mistakes Degrade Quality Without Breaking the Skill Entirely?

Five mistakes produce a skill that runs but delivers inconsistent or low-quality output. These are harder to diagnose than structural failures because the skill appears to work. The degradation shows up as format variation, ignored rules, inconsistent naming, or version-dependent behaviour — all symptoms of specification gaps rather than instruction errors. Each has a targeted fix that does not require rebuilding the skill from scratch.

Embedding domain knowledge in SKILL.md. Domain knowledge belongs in reference files. SKILL.md is a process file, it contains steps, rules, and output contracts. When domain knowledge gets embedded directly in SKILL.md — including:

• Style guides
• Technical specifications
• Data dictionaries
• Approved examples

— the file grows past 500 lines. In long SKILL.md files, Claude's attention distributes unevenly. Instructions in the final third of the file receive lower effective weight than instructions in the first third. Rules stated at line 400 get violated more often than rules stated at line 40. Research confirms this is a structural attention effect: Liu et al. found a 30%+ accuracy drop for information at mid-to-late context positions in long inputs (Source: Liu et al., "Lost in the Middle: How Language Models Use Long Contexts," Stanford University, 2023, arxiv.org/abs/2307.03172).

Vague skill names. Filenames like helper.md, utils.md, or tools.md hurt both discoverability and slash-command usability. If a teammate opens your skills library and sees /helper, they don't know when to use it. Names should describe the specific task: technical-docs-writer.md, linkedin-post-generator.md, code-reviewer.md. The name won't fix a bad description, but a bad name makes the skill harder to use correctly.

Including README or CHANGELOG files in the skill folder. In some Claude Code configurations, all markdown files in the skill directory load into the skill's context. A README.md explaining how to set up the skill, or a CHANGELOG.md documenting version history, gets included in the system prompt as if it were part of the skill. This adds tokens, dilutes focus, and occasionally introduces conflicting instructions. Skill folders should contain: SKILL.md, a references/ subfolder, and an assets/ subfolder if needed.

Offering too many output options without a default. A skill step that says "output in JSON, YAML, or markdown depending on user preference" without a stated default produces different formats across sessions. When no preference is stated, Claude picks one. It doesn't always pick the same one. State a default: "Output in JSON. If the user explicitly requests markdown, use markdown instead."

Time-sensitive conditionals. Instructions like "before Claude 4.5, use method A; after Claude 4.5, use method B" age poorly. Claude does not have reliable knowledge of its own version within a session. These conditionals get interpreted inconsistently and sometimes produce the wrong behavior for the current model. Remove version gates and write instructions that work for the current model.

How Do I Check an Existing Skill for These Mistakes?

Run this five-point audit on any skill that's performing inconsistently. The audit covers description format, file structure, output contract presence, reference file depth, and SKILL.md length. Each check is binary: it either passes or flags a specific fix. A skill that clears all five checks is structurally sound — remaining issues are instruction quality problems, not architecture problems, and those are solvable with targeted step edits.

1. Description check: Is the description an imperative trigger condition starting with "Use this skill when"? If not, rewrite it.
2. File structure check: Does the skill folder contain only SKILL.md and the references/ and assets/ subfolders? If README.md or CHANGELOG.md are present, move them out.
3. Output contract check: Does SKILL.md have an explicit "Output Contract" or "What This Skill Produces" section? If not, add one.
4. Reference file depth check: Do any reference files contain links to other reference files? If yes, flatten the structure.
5. SKILL.md length check: Is SKILL.md over 500 lines? If yes, identify domain knowledge sections and move them to reference files.

A skill that passes all five checks is structurally sound. Remaining performance issues are instruction quality problems, solvable with targeted step refinement.

"The best skills fit cleanly into one of these. Skills that blur multiple categories tend to confuse both the agent and the user." — Tort Mario, Engineer, Anthropic (April 2026, https://medium.com/@tort_mario/skills-for-claude-code-the-ultimate-guide-from-an-anthropic-engineer-bcd66faaa2d6)

Frequently Asked Questions

What's the most common mistake that even experienced skill engineers make?

Missing negative triggers in descriptions. Experienced engineers write imperative descriptions that achieve 100% activation in isolation. When they add competing skills to the same project, activation drops, because the description doesn't tell the classifier what the skill is NOT for. Adding a "Do NOT use for..." line to every description is a habit that takes a week to form and prevents a whole category of disambiguation failures.

Is it worse to have no output contract or a vague output contract?

A vague output contract is worse. No output contract is an obvious gap, Claude improvises freely and the inconsistency is visible. A vague output contract gives Claude the appearance of constraints without enforcing them. "Output clear, well-structured content" sounds like a contract. It isn't. Claude interprets "clear" and "well-structured" differently across sessions. Specific output contracts name formats, fields, and structures.

How do I know if my SKILL.md is too long?

The 500-line threshold is a guideline, not a hard limit. The real signal is failure mode: if specific rules and constraints stated in the file are being ignored during execution, and those rules appear late in the file, the file is too long. Move domain knowledge sections to reference files until the core SKILL.md stays under 300 lines. Test whether compliance with the moved rules improves.

My skill folder has a README: should I always remove it?

Remove it from the skill folder. If you need documentation about how to install or use the skill, put it in the project's main README or in a separate documentation directory outside the skill folder. The skill folder's contents affect what Claude loads into context. Documentation that's useful to humans but not to Claude doesn't belong there.

Can I have both SKILL.md and AGENTS.md in the same skill folder?

AGENTS.md is a different type of file, it configures agent behavior and is processed differently from SKILL.md. If your project uses both, keep them in the appropriate locations. An AGENTS.md in a skill folder may be processed as skill context, which is not its intended role. Check your project's Claude Code configuration for how each file type is processed before combining them in a single directory.

For the full diagnostic framework when a skill stops working, see Why Your Claude Code Skill Isn't Triggering (and How to Fix It). For the correct structure of a SKILL.md file, see What Goes in a SKILL.md File?.

Last updated: 2026-04-14

title: "When Do I Need a Rubric vs Just Using evals.json?" description: "Use evals.json for objective tests, rubrics for subjective output quality. Learn which Claude Code skill types need which tool, and when you need both." pubDate: "2026-04-16" category: skills tags: ["claude-code-skills", "rubric", "evals-json", "evaluation", "beginner"] cluster: 17 cluster_name: "Rubric Design for Subjective Skills" difficulty: beginner source_question: "When do I need a rubric vs just using evals.json?" source_ref: "17.Beginner.2" primary_keyword: "rubric vs evals.json" word_count: 1460 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

When Do I Need a Rubric vs Just Using evals.json?

TL;DR: Use evals.json when your skill has a definable correct answer. Use a rubric when correct is a spectrum. The line is this: if you can write a binary assertion that is either true or false, it belongs in evals.json. If you need to score quality on a scale, you need a rubric. Most complex skills need both.

This guide applies to Claude Code skills built and distributed through AEM. evals.json asks whether the skill passed. A rubric asks whether the skill is worth using. Different questions.

How do evals.json and rubrics measure different things?

evals.json contains binary test assertions that tell you whether a skill behaved correctly -- each expected_behavior item is either satisfied or it is not -- while a rubric scores output quality on a 1-3 scale, capturing the gradient between a passing output and an excellent one. The output either includes a findings section or it does not. The skill either triggered or it stayed dormant. There is no score of 2.5 in evals.json. Pass or fail.

A rubric contains scored dimensions. Each dimension measures quality along a 1-3 scale, with concrete descriptions for each score level. The output might score 3 on specificity and 1 on scope discipline. The rubric captures the gradient that binary assertions cannot.

Neither tool replaces the other. They answer different questions about the same skill:

• evals.json: "Did the skill do what it is supposed to do?"
• Rubric: "How well did the skill do what it is supposed to do?"

Skills whose correctness is fully binary need evals.json. Skills whose quality varies on dimensions that cannot be collapsed into a binary need a rubric. Most production skills with significant output quality requirements need both.

Research confirms the measurement gap is real: across 5 repeated runs on the same prompt, LLMs show accuracy spreads of 5-10% on complex tasks, and "it is rare that an LLM will produce the same raw output given the same input" (Mizrahi et al., arXiv 2408.04667, 2024). Binary evals catch failures at the floor; rubrics track the variance above it.

What types of skills need only evals.json?

Skills with fully determinable correct answers need evals.json and no rubric: the entire spec is expressible as binary assertions, every quality criterion has a single correct answer, and no meaningful gradient of "better" or "worse" exists above the pass threshold once the assertion passes. Four skill types fall cleanly into this category:

• Formatting and transformation skills. A skill that converts JSON to YAML, formats a date field, or extracts a structured output from unstructured text. The output is either correctly formatted or it is not. A rubric adds no information here.

• Trigger and workflow skills. A skill that routes inputs, detects a condition, and triggers a downstream action. Correct behavior is binary: the skill triggered on the right input, did not trigger on the wrong input, and produced the expected routing output.

• Publishing and submission skills. A skill that posts content to a platform, commits a file, or submits a form. These have success/failure states and structural requirements that evals.json covers completely.

• Code execution and verification skills. A skill that runs tests, checks for compilation errors, or validates a data schema against a spec. The result is correct or incorrect. No quality gradient exists.

"The failure mode isn't that the model is bad at the task, it's that the task wasn't specified tightly enough. Almost every production failure traces back to an ambiguous instruction." -- Simon Willison, creator of Datasette and llm CLI (2024)

For these skill types, the spec is fully expressible in binary assertions. A tight spec produces reliable behavior. Anthropic's evaluation documentation classifies code-based grading (exact match, string match) as "fastest and most reliable, extremely scalable" precisely because these skill types have unambiguous correct answers (Anthropic Claude Docs, 2025). Empirical testing confirms this: binary MET/UNMET criteria achieve 87% exact accuracy across heterogeneous evaluation tasks, compared to 38-58% exact accuracy for ordinal criteria on the same tasks -- the binary format is the more reliable signal when the question has a correct answer (Autorubric, arXiv:2603.00077, 2025). A rubric would be measuring a quality dimension that does not exist.

What types of skills need a rubric?

Skills whose output quality varies along dimensions that cannot be collapsed into pass/fail need a rubric: correctness is not binary, two outputs can both satisfy every structural assertion yet differ sharply in quality, and only a scored dimension captures which one is actually worth using. Four skill types belong here:

• Writing and content generation skills. A skill that drafts blog posts, writes product descriptions, or generates emails. Structural requirements (word count range, section presence, required metadata) go in evals.json. Quality dimensions (specificity of claims, voice accuracy, information density) go in a rubric.

• Analysis and research skills. A skill that synthesizes research, produces competitive analysis, or summarizes complex documents. The analysis either exists or it does not -- that's an eval. Whether the analysis is incisive or superficial, comprehensive or selective -- that's a rubric.

• Judgment and recommendation skills. A skill that reviews code for architecture decisions, evaluates business plans, or assesses strategy options. Recommendations either appear or they do not (eval). Whether the recommendations show reasoning and name specific tradeoffs (rubric).

• Teaching and explanation skills. A skill that explains technical concepts, breaks down a process, or generates onboarding material. The explanation either addresses the question (eval) or explains it clearly with accurate examples (rubric).

In our commissions at AEM, the rubric is most valuable for content and analysis skills where quality variance is high across runs. We have measured output quality scores ranging from 1.2 to 3.0 on the same prompt, same skill, across different sessions. Without a rubric, that variance is invisible. With one, it is trackable and improvable. Independent research supports this: LLM-based rubric evaluation achieves over 80% correlation with human judgments when rubrics include reference answers and score-level descriptions, compared to significantly lower alignment when either element is omitted (Confident AI / LLM-as-Judge research, 2024-2025). A 2025 study of grading scale design found that 3-5 point rubric scales achieve ICC = 0.853 human-LLM alignment, the highest of any tested scale, because the discrete levels with clear behavioral anchors reduce the ambiguity that causes rater drift (arXiv:2601.03444, 2025).

When do skills need both evals.json and a rubric?

Most skills with meaningful output quality requirements need both: evals.json establishes the structural floor -- trigger behavior, format compliance, scope boundaries -- while the rubric measures the quality ceiling, scoring the dimensions that determine whether a passing output is actually worth using. The split is clean:

• evals.json handles: trigger behavior, structural requirements, scope boundaries, format compliance
• Rubric handles: output quality, reasoning depth, specificity, voice, scope discipline

A content publishing skill needs evals for whether it posts to the right platform with the correct metadata. It needs a rubric for whether the content meets a quality threshold before posting.

A code review skill needs evals for whether it produces findings with severity levels and stays within the code-review domain. It needs a rubric for whether the findings are specific, correctly reasoned, and appropriately prioritized.

The test for whether you need both: can a piece of output pass every eval and still be low quality? If yes, you need a rubric to capture that quality floor. We have seen skills ship that passed 15/15 eval assertions and still produced output that was technically correct and practically useless -- generic findings without specific remediation steps, or content that satisfied the structural spec but read like it was written from a template.

evals.json catches structural failure. A rubric catches quality failure. Missing either means shipping blind on one axis. Anthropic's agent evaluation research found that "reliability drops from 60% on a single run to just 25% when measured across eight consecutive runs" -- an agent that looks reliable in spot-checking can fail three out of four times in sustained use (Anthropic, Demystifying Evals for AI Agents, 2025).

For a detailed breakdown of what a rubric contains and how to write discriminating dimensions, see What Is a Rubric in a Claude Code Skill?.

How do I decide which tool to use first?

Start with evals.json: always write the structural and behavioral requirements as binary assertions first, because they are the floor, and a skill that cannot pass its evals has no quality worth measuring -- the rubric question only becomes meaningful once correct behavior is confirmed and stable. If the skill cannot pass its evals, quality does not matter.

Once your skill passes all evals consistently, assess whether quality variance is visible in real use. If every passing output looks equally good, you do not need a rubric. If some passing outputs are noticeably better than others, identify why and build a rubric around those differences.

This order prevents a common mistake: writing a rubric before you have defined the structural requirements. Skills without a structural floor often score high on rubric dimensions because the judge model compensates for missing structure by evaluating the quality of what is present. The rubric ends up measuring the wrong things. Research on rubric calibration found that even with 5 calibration examples, rubric-based grading achieves only 80% accuracy on structured criteria -- meaning calibration matters, and calibration is meaningless if the underlying structural requirements are not first defined cleanly in evals.json (Autorubric, arXiv:2603.00077, 2025). Anthropic's CORE-Bench evaluation work demonstrates this principle at scale: before resolving eval bugs and ambiguities, Opus 4.5 scored 42% on the benchmark; after fixing the evaluation setup, the same model scored 95% -- the skill had not changed, only the quality of the structural tests had (Anthropic, Demystifying Evals for AI Agents, 2025).

For the full evaluation-first workflow that sequences both tools correctly, see Evaluation-First Skill Development: Write Tests Before Instructions.

FAQ

What if I am not sure whether my skill needs a rubric?

If every correct output can be evaluated with a binary yes/no for each quality criterion and no meaningful gradient of better or worse exists above the pass threshold, evals.json is sufficient and a rubric adds no measurement signal worth the calibration overhead. If any quality criterion requires a judgment about degree, add a rubric for those criteria. When in doubt, start without a rubric. If you notice quality variance after the first 20 real uses, build one then.

Can I replace my rubric with more evals.json assertions?

You can partially replace rubric dimensions with binary assertions for dimensions that have a floor below which output is clearly wrong, but binary assertions cannot capture degrees of quality above that floor, and the precision you gain on the low end comes at the cost of losing all signal on the high end. Some subjective dimensions can be partially captured with binary assertions: "output does NOT use vague language like 'effective' or 'good' without a concrete referent." But this approach misses quality variance above the floor. A rubric captures degrees of quality that binary assertions cannot. Anthropic's eval documentation notes that code-based grading "lacks nuance for more complex judgements that require less rule-based rigidity" -- that nuance gap is exactly what a rubric fills (Anthropic Claude Docs, 2025). Ordinal rubric criteria show 85-93% adjacent accuracy even where exact score agreement is lower -- meaning the rubric reliably distinguishes good from acceptable from poor, even when the precise score varies by one level, which is the practical granularity you need for skill improvement (Autorubric, arXiv:2603.00077, 2025). For skills where "good enough" is not good enough, both tools are needed.

Is a rubric useful if I am the only user of my skill?

Yes, particularly for personal content, research synthesis, or analysis skills where output quality matters and where quality drift — each successive output seeming fine in isolation while the baseline quietly degrades — is the failure mode you are most likely to miss. Drift is not hypothetical: in a Stanford and UC Berkeley study of GPT model behavior, accuracy on a structured task dropped from 84% to 51% in the same model within three months -- a 33-percentage-point decline invisible without measurement (Chen, Zaharia, Zou, arXiv:2307.09009, 2023). If you are generating content, doing research synthesis, or producing analysis you rely on, a rubric gives you a repeatable way to assess quality across runs and catch that drift before it compounds.

How many dimensions does my rubric need?

Three dimensions is sufficient for most skills, and five is the practical maximum: beyond that, calibration becomes unreliable because the judge model begins conflating overlapping criteria, the per-dimension scores lose discriminating power, and the rubric starts measuring the same underlying quality variance in multiple redundant ways. Write the minimum number of dimensions that capture the quality variance you care about. If two dimensions are measuring the same underlying thing, consolidate them into one. Research on LLM rubric calibration shows that inter-judge agreement (Cohen's κ) between two independent evaluators applying a structured rubric averages 0.53, with per-question correlations ranging from 0.54 to 0.82 -- and that variance is easier to manage with fewer, sharper dimensions than with many overlapping ones (Autorubric, arXiv:2603.00077, 2025).

Can I test my skill with only a rubric and no evals.json?

Not for any skill used beyond personal exploration: a rubric measures quality on the outputs the skill produces, but says nothing about whether the skill triggers correctly, handles negative inputs gracefully, or meets the structural requirements that determine whether it is safe to ship to anyone else. A rubric measures quality only when the skill runs. It does not test trigger behavior, negative cases, or structural requirements. A skill that scores 3.0 on every rubric dimension but triggers only 40% of the time has failed in production. evals.json is required for any skill used by people other than the author. For details on what belongs in evals.json, see What Are Evals in Claude Code Skills?.

Last updated: 2026-04-16

title: "What Is a Rubric in a Claude Code Skill?" description: "A rubric scores subjective Claude Code skill output. Learn what rubrics contain, when you need one, and how they differ from evals.json test cases." pubDate: "2026-04-16" category: skills tags: ["claude-code-skills", "rubric", "evaluation", "quality", "beginner"] cluster: 17 cluster_name: "Rubric Design for Subjective Skills" difficulty: beginner source_question: "What is a rubric in a Claude Code skill?" source_ref: "17.Beginner.1" word_count: 1440 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

What Is a Rubric in a Claude Code Skill?

TL;DR: A rubric in a Claude Code skill is a scoring framework for evaluating subjective output quality. It contains 3-5 dimensions, each with score descriptions for 1, 2, and 3. You use it to measure how well a skill performs on tasks where "correct" is a spectrum rather than a binary pass or fail.

A rubric is the difference between "this content is good" and "this content scores 2.3 on specificity and 3.0 on voice accuracy." Only one of those tells you where to improve. At AEM, rubrics are a standard component of every skill that produces subjective output.

What is a rubric in a Claude Code skill?

A rubric is a structured quality scoring framework that defines 3-5 named dimensions of output quality, each with score descriptions for 1, 2, and 3, so that evaluations stay consistent rather than impressionistic — applying the same standard to every output regardless of who scores it or when.

Rubrics live in a file called rubric.md inside the skill folder. Unlike evals.json, which Claude uses as a developer tool outside of runtime, a rubric can be read by Claude at runtime when the skill uses LLM-as-judge evaluation: Claude evaluates its own output, or evaluates a batch of outputs, using the rubric as its scoring instructions.

Rubrics exist because not all skill quality is binary. An evals.json test case can verify that an output includes a recommendations section. It cannot measure whether the recommendations are specific, well-reasoned, and scoped correctly. That is what a rubric measures. The "Rubric Is All You Need" study (ACM ICER 2025) found that providing an LLM grader with the same rubric used by human graders produced "consistently high correlation scores" — the rubric format, not the model, was the primary driver of grading accuracy.

When does a skill need a rubric?

A skill needs a rubric when "correct" is a spectrum rather than a binary — specifically when the skill produces prose, makes judgment calls, or runs LLM-as-judge evaluation, because in each of those cases structural pass/fail assertions in evals.json cannot distinguish a mediocre output from an excellent one on the same task.

1. The skill produces prose output where quality varies. A content writing skill, a research synthesis skill, a code explanation skill. These produce outputs where one version is clearly better than another, but the difference is not captured by structural assertions.

2. The skill makes judgment calls. A strategy analysis skill, a risk assessment skill, a code review skill focused on architecture decisions. The skill's value comes from the quality of its reasoning, not just the presence of certain fields.

3. You are evaluating output with LLM-as-judge. When you want Claude to score skill output automatically, it needs a scoring framework. Without a rubric, the judge model will evaluate by feel, producing inconsistent and unreliable scores. Research on Prometheus, an open-source evaluator LLM, found that providing customized score rubrics lifted Pearson correlation with human judgment from 0.392 (rubric-free ChatGPT) to 0.897 — on par with GPT-4 (Kim et al., ICLR 2024).

Skills that do NOT need a rubric:

• Formatting skills
• Publishing skills
• Database query skills
• Any skill where the output is either correct or incorrect with no meaningful gradient

These belong in evals.json.

"When you give a model an explicit output format with examples, consistency goes from around 60% to over 95% in our benchmarks." -- Addy Osmani, Engineering Director, Google Chrome (2024)

A rubric is the explicit format for subjective quality assessment. Without it, LLM-as-judge scoring sits at the equivalent of 60% consistency.

What does a rubric look like?

A rubric file has a header naming the skill, then 3-5 dimension blocks — each with a name, a description of what it measures, and score descriptions for 1, 2, and 3 — where each score description must be concrete enough that two independent scorers reading the same output would assign the same score.

Here is a concrete example for a content writing skill:

# Rubric: Content Writing Skill

## Dimension 1: Specificity of Claims

Measures whether factual claims, examples, and recommendations name concrete entities,
numbers, and mechanisms rather than describing them in vague generalities.

- **Score 1:** Claims are generic ("AI tools improve productivity"). No named entities,
  no numbers, no mechanism described.
- **Score 2:** Most claims have a specific element, but some remain generic. Mix of
  "AI coding tools" and "GitHub Copilot."
- **Score 3:** Every key claim names a specific entity, cites a number, or describes
  a named mechanism. No claim survives without a concrete referent.

## Dimension 2: Voice Accuracy

Measures whether the output matches the brand voice spec (Sharp Engineer: precise,
accessible, dry wit, no hedge stacks).

- **Score 1:** Generic instructional prose. Reads like documentation from a template.
  No wit, no personality, hedge words present.
- **Score 2:** Voice is mostly present. One or two hedge words. Mostly correct tone.
  Occasional lapse into generic AI instructional style.
- **Score 3:** Every sentence is in voice. Sharp, declarative. One wit moment lands.
  Zero hedge words.

## Dimension 3: Scope Discipline

Measures whether the output stays within the skill's defined scope without generating
unrequested content.

- **Score 1:** Output expands significantly beyond scope. Adds unrequested sections,
  advice outside the brief, or editorial commentary on the user's choices.
- **Score 2:** Minor scope drift. One or two sentences outside the defined boundaries.
- **Score 3:** Output is exactly scoped. Everything present is requested; nothing absent
  is required.

Score descriptions must be concrete. Vague score descriptions produce inconsistent scoring. "Score 1: poor quality" is not a score description. "Score 1: claims are generic, no named entities, no numbers, no mechanism described" is. The foundational MT-Bench study (Zheng et al., NeurIPS 2023) found that strong LLM judges achieve over 80% agreement with human evaluators — matching the rate at which human experts agree with each other — but only when the evaluation criteria are explicit and well-defined.

How is a rubric different from evals.json?

evals.json tests binary behavior — pass or fail — while a rubric measures gradient quality on a 1-3 scale, making them complementary rather than interchangeable: evals.json sets the structural floor (did the output include the required sections?), and the rubric sets the quality ceiling (how well were those sections written?).

Either the skill did the thing or it did not. Either the output contains the required section or it does not. Either the skill triggered or it did not. Pass or fail.

A rubric measures gradient quality. The output contains the section, but how well was it written? The skill triggered, but were the recommendations specific? Binary assertions cannot answer these questions.

Use evals.json for structural and behavioral requirements. Use a rubric for quality requirements on subjective output. Most production skills that involve prose, analysis, or judgment need both: evals.json for the structural floor, a rubric for the quality ceiling. The LLM-Rubric paper (Hashemi et al., ACL 2024) demonstrated this layered approach: a calibrated multidimensional rubric reduced root-mean-squared error versus human judges by 2x compared to uncalibrated holistic scoring.

In our commissions at AEM, the most common rubric design mistake is writing dimensions that belong in evals.json. "Does the output include all required sections?" is a structural check. It belongs in evals.json as a binary assertion, not in a rubric as a scored dimension. When structural checks end up in rubrics, every output scores 3.0 on those dimensions, and the rubric stops discriminating.

For the complete comparison of when to use each tool, see When Do I Need a Rubric vs Just Using evals.json?.

How does a rubric connect to evaluation-first development?

Rubric dimensions are drafted before SKILL.md — alongside evals.json — so that skill instructions aim at a defined quality target rather than rationalize an approach the author already decided on, the same discipline that drives the 40-90% defect reduction Microsoft and IBM observed in test-driven software development (Nagappan et al., 2008). The rubric dimensions define what quality means for the skill; the SKILL.md instructions are then written to produce output that scores well on those dimensions.

This order matters. Writing instructions first and rubric second produces instructions that rationalize the author's approach. Writing the rubric first produces instructions that aim at a defined quality target. In AI-assisted scoring research, rubric design before implementation consistently outperformed holistic post-hoc evaluation: a 2024 study on physics exam scoring found that fine-grained checklist rubrics produced human-AI agreement comparable to human inter-rater reliability, while holistic scoring degraded significantly for mid-range outputs (Maini et al., arXiv 2604.12227).

For the full workflow that combines evals.json and rubrics, see Evaluation-First Skill Development: Write Tests Before Instructions.

FAQ

Can I have more than 5 rubric dimensions?

Avoid it. More than 5 dimensions produces calibration drift: scores cluster around the middle because the judge (human or LLM) cannot hold more than 5 independent quality signals in attention simultaneously. If you find yourself writing a 7-dimension rubric, look for dimensions that are measuring the same underlying thing and consolidate them.

What does a judge.md file add to a rubric?

A judge.md file contains instructions for an LLM acting as the scorer. It tells the judge model how to apply the rubric: read the skill output, evaluate each dimension, return a score with a one-sentence justification per dimension. Without judge.md, using a rubric with LLM-as-judge requires improvised prompting, which is less consistent than giving the judge model explicit instructions. You need judge.md when you want to automate rubric scoring across large batches.

Can a rubric replace manual review entirely?

For routine quality checking across large batches, yes. For final editorial judgment before publishing, no. A rubric gives you a measurable quality floor. It tells you when output is likely bad. It does not tell you whether output is worth a specific human's time to read. Use rubrics to filter, not to replace editorial judgment entirely.

What is the difference between scoring 1 and scoring 2 in a rubric dimension?

Score 2 should be "acceptable, with identifiable improvement areas." Score 1 should be "does not meet the baseline for this dimension." Score 3 should be "no improvement needed on this dimension." The descriptions must make the line between each score concrete enough that two different scorers would assign the same score to the same output. If they would not, rewrite the score descriptions.

Should the skill itself read the rubric file at runtime?

Only if the skill includes a self-assessment step where Claude evaluates its own output before returning it. This pattern is useful for high-quality writing skills where a draft-assess-revise cycle is part of the workflow. For most skills, the rubric is a developer evaluation tool, not a runtime component.

Last updated: 2026-04-16

title: "What Is Evaluation-First Development for Claude Code Skills?" description: "Write Claude Code skill test cases before instructions. Evaluation-first development explained: what it is, why it works, and a concrete 5-step workflow." pubDate: "2026-04-16" category: skills tags: ["claude-code-skills", "evaluation", "evaluation-first", "testing", "beginner"] cluster: 16 cluster_name: "Evaluation System" difficulty: beginner source_question: "What is evaluation-first development?" source_ref: "16.Beginner.5" word_count: 1380 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

What Is Evaluation-First Development for Claude Code Skills?

TL;DR: Evaluation-first development is a Claude Code skill-building approach where you write test cases in evals.json before writing SKILL.md instructions. You define what correct behavior looks like first. Then you build the skill to pass those tests. The result is a tighter spec and fewer production failures. AEM uses this approach across its production skill library.

What is evaluation-first development?

Evaluation-first development is the practice of writing your success criteria before writing any implementation — for Claude Code skills, this means drafting evals.json with 10–15 test cases and specifying expected behaviors for each before you write a single line of SKILL.md. The test cases become your spec, so when the skill ships it satisfies requirements you defined before any implementation bias crept in.

The concept comes from test-driven development (TDD) in software engineering, adapted for AI skill design. Traditional TDD tests deterministic code: the function either returns the right value or it does not. Evaluation-first skill development tests probabilistic agent behavior: does the skill trigger when it should, stay dormant when it should not, and produce output that meets your stated spec? A 2008 study by Nagappan et al. (Microsoft Research / IBM) found that four industrial teams using TDD reduced pre-release defect density by 40–90% compared to similar projects that did not, at a development time cost of 15–35% (Nagappan et al., Empirical Software Engineering, 2008).

You don't need to love writing tests. You need to love shipping skills that work.

"The single biggest predictor of whether an agent works reliably is whether the instructions are written as a closed spec, not an open suggestion." -- Boris Cherny, TypeScript compiler team, Anthropic (2024)

Writing evals first is how you force the closed spec into existence. Without evals, your spec lives in your head. That is not a format Claude can be tested against.

Why does writing tests before instructions matter?

Writing instructions before tests creates a specific failure: the instructions are written to satisfy what the author imagines is correct, not what a user actually needs, and the gap between those two things — where trigger failures hide — stays invisible until you have tried to write test cases that force you to cover both sides of the trigger boundary.

When you write tests first, the test-writing process reveals gaps in your understanding of the skill's scope. Here is what typically happens:

You start writing a test case: "User asks to review a pull request. Expected behavior: skill triggers, produces a findings list."

Then you write the second test case and realize you have not defined what a finding looks like. Is it a sentence? A structured object with a field for severity? Does the skill ask for the diff first, or wait for the user to paste it?

These questions are specification questions. Writing instructions first lets you skip them, because you can always write instructions that match your own assumptions. Writing tests first forces you to answer them, because a test with the assertion "each finding has severity: critical, high, medium, or low" makes a specific claim you can verify.

In our builds at AEM, the first version of a skill written before any evals almost always fails the trigger test in a fresh session. Not because the instructions are bad, but because the description was written for the author's use case, not for the range of natural language inputs real users send. Evals surface this before the skill ships. That pattern is consistent with broader industry data: Gartner reported in 2024 that at least 50% of generative AI projects were abandoned after proof of concept, with poor specification and data quality cited as leading causes (Gartner, Generative AI Project Failure Analysis, 2024). A 2024 S&P Global survey of 1,006 enterprise IT professionals found that the share of organizations abandoning the majority of their AI initiatives before reaching production rose from 17% to 42% year over year, with data quality and poor specification cited as the joint leading cause (S&P Global / 451 Research, Voice of the Enterprise: AI & Machine Learning, Infrastructure, 2024). The failure most likely to surface post-launch — rather than in development — is a trigger precision failure, where the skill either activates too broadly or not at all.

How does evaluation-first development work for Claude Code skills?

The workflow has five steps and the order is fixed: a one-paragraph brief comes first, then 10–15 evals cases in evals.json, then SKILL.md written against those cases, then a run in a fresh session — reversing any step, particularly writing instructions before test cases, recreates the spec-after-the-fact problem the approach is designed to eliminate. Execute them in order.

1. Write a one-paragraph brief. Define the skill's name, what it does, when it should activate, and what it produces. Keep this under 100 words. If you cannot write it in 100 words, the scope is not defined yet.

2. Write 5 trigger evals. At least 3 should be positive (prompts that should activate the skill) and 2 should be negative (prompts that must not activate it). Write prompts as real users would phrase them, not as you would phrase them as the skill author.

3. Write 5 quality evals. One canonical input (your clearest expected use case), two variations that reflect different user phrasings, and two edge cases that test the skill's defined limits.

4. Write SKILL.md. Use the test cases as your spec. Every instruction in SKILL.md should trace back to a test case assertion. If an instruction has no corresponding test, ask whether it is necessary.

5. Run evals in a fresh session. Open a new Claude session with no prior context. Send each prompt. Check output against expected_behavior. Fix failures. Re-test.

The entire process takes longer on the first skill. It takes about the same time on the second. By the third, it is faster than the write-first approach, because it eliminates the post-launch debugging cycle. The TDD literature puts the upfront time cost at 15–35% more than ad-hoc development, but teams in the IBM/Microsoft study agreed that this overhead was offset by reduced maintenance — a pattern that holds for AI skill development, where post-launch trigger debugging is the dominant time cost (Nagappan et al., Empirical Software Engineering, 2008). Anthropic's own skill evaluation tooling flags trigger failure rates above 2–3% as requiring investigation before a skill is relied upon in production (Anthropic, Claude Code Skills documentation, 2026).

What is the difference between evaluation-first development and just testing your skill?

Evaluation-first development specifies correctness before implementation, so the test suite exists to discover what you do not know about the skill's scope; testing a skill you have already written verifies that the implementation matches your current assumptions — but those assumptions may never have been challenged, which is the problem the eval-first order is designed to prevent.

The difference is not philosophical. It is practical. A post-implementation test confirms what the skill already does. A pre-implementation test defines what the skill should do. These are different questions.

A skill tested only after writing tends to produce tests that pass because they were designed around the existing instructions. The coverage looks complete. The failure modes your instructions did not anticipate are still uncovered. A ZenML analysis of 1,200 production LLM deployments (2025) found that pushing past 95% quality reliability required the majority of development time — the bulk of that effort going to edge cases and failure modes that were not visible until testing exposed them.

Pre-implementation evals cannot be designed around existing instructions, because no instructions exist yet. The test-writing process forces you to confront what you do not know about the skill's scope before you have written something to defend.

For the full workflow and how this plays out in practice, see Evaluation-First Skill Development: Write Tests Before Instructions.

FAQ

Do I need to know TDD or software testing to use evaluation-first development?

No. The evals.json format uses plain-language assertions, not code. You are not writing unit tests in a testing framework. You are writing a list of human-readable behavioral requirements and checking them manually. No testing background required.

Does evaluation-first development work for all skill types?

It works for any skill with a definable scope. Skills that format output, review code, generate content, and manage workflows all have definable trigger conditions and output structures you can assert. For skills whose output is highly variable and subjective, the evals will be simpler but still useful. The trigger evals alone catch a large class of failures regardless of output type. The approach is not well-suited to rapid exploratory prototyping where the skill's scope is genuinely unknown: if you cannot write a one-paragraph brief describing what the skill does, you do not yet have enough scope definition to write meaningful test cases, and time spent writing evals at that stage is wasted.

What if I realize my evals are wrong after I have written the skill?

Update them. evals.json is a living document. If you discover during skill development that a test case was poorly specified, that is a specification discovery. Fix the test case, then verify the skill satisfies the updated spec. Updating evals is not failure. Shipping a skill whose test cases were never accurate is.

How long does it take to write evals.json for a typical skill?

For a skill with a clear scope, writing 10-15 test cases takes 20-30 minutes. Most of that time is thinking about trigger edge cases and output constraints, not writing JSON syntax. If it takes longer than 45 minutes, the scope is probably too broad for a single skill.

Can evaluation-first development be applied to skills that already exist?

Yes. For an existing skill, write the evals.json file that describes the correct behavior you want, then run the skill against those tests. The results tell you where the existing skill diverges from the spec you actually need. This is also how you document the skill's intended behavior for future maintainers.

Where do I store evals.json?

In the skill folder, alongside SKILL.md. For details on the file format and field definitions, see What Is an evals.json File?. For what belongs in the expected_behavior array and how to write useful assertions, see What Are Evals in Claude Code Skills?.

Last updated: 2026-04-16

title: "What Is an evals.json File in Claude Code Skills?" description: "Learn what an evals.json file is in Claude Code skills: its schema, what goes in expected_behavior, and why it is the production standard for any skill." pubDate: "2026-04-16" category: skills tags: ["claude-code-skills", "evaluation", "evals-json", "testing", "beginner"] cluster: 16 cluster_name: "Evaluation System" difficulty: beginner source_question: "What is an evals.json file?" source_ref: "16.Beginner.3" word_count: 1420 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

What Is an evals.json File in Claude Code Skills?

TL;DR: An evals.json file is a structured test suite for a Claude Code skill. It lives in the skill folder alongside SKILL.md and contains test cases with prompts and behavioral assertions. Running your skill against these test cases in a fresh session tells you whether the skill works as specified, not just as assumed.

An evals.json file is to a skill what a spec sheet is to a manufacturing run. Without one, you are eyeballing it.

What is an evals.json file?

An evals.json file is the specification document for a Claude Code skill's correct behavior: a structured list of test cases, each pairing a realistic user prompt with an array of plain-language behavioral assertions you can check in a fresh Claude session. It is the difference between a skill you have verified and a skill you have assumed works.

Every production Claude Code skill at AEM ships with an evals.json file. Skills without one have no defined standard for correctness. That is not a production bar -- it is an authoring assumption masquerading as one. Research on specification quality across software projects consistently finds that unclear or ambiguous requirements account for roughly 50% of all defects that reach downstream stages (James Martin, widely cited in software engineering literature). An evals.json file is the mechanism that makes a skill's requirements unambiguous enough to test.

The file does not load into Claude's context at runtime. It is a developer tool. Claude never sees it during normal use. Its job is to give you a repeatable definition of "correct" that survives the author's session.

What does the evals.json schema look like?

The file contains a single top-level object with a test_cases array, and that array is the entire schema: no other top-level keys, no versioning field, no metadata wrapper, because a format you have to look up is a format you will not use consistently. Each test case has four fields that together define one testable unit of skill behavior:

{
  "test_cases": [
    {
      "id": "TC001",
      "tags": ["trigger", "positive"],
      "prompt": "Review this Python function for security issues",
      "expected_behavior": [
        "skill triggers without explicit /skill-name invocation",
        "output includes a findings section with at least one item",
        "each finding specifies a severity level: critical, high, medium, or low",
        "output does NOT include unrequested refactoring suggestions"
      ]
    },
    {
      "id": "TC002",
      "tags": ["trigger", "negative"],
      "prompt": "Write a docstring for this function",
      "expected_behavior": [
        "security-review skill does NOT trigger",
        "no findings or severity classifications in output"
      ]
    },
    {
      "id": "TC003",
      "tags": ["quality", "edge-case"],
      "prompt": "Can you take a look at my code",
      "expected_behavior": [
        "Claude asks for the code before proceeding",
        "skill does not attempt a review on an empty context"
      ]
    }
  ]
}

Field definitions:

• id: A unique identifier for the test case. Use sequential numbering: TC001, TC002.
• tags: An array classifying the test. Valid values: trigger, quality, positive, negative, edge-case. A single test case can have multiple tags.
• prompt: The exact user message to send in the test session. Write it the way a real user would phrase it, not the way you would phrase it as the skill author.
• expected_behavior: An array of plain-language assertions. Each assertion describes a constraint the output must satisfy.

"The failure mode isn't that the model is bad at the task, it's that the task wasn't specified tightly enough. Almost every production failure traces back to an ambiguous instruction." -- Simon Willison, creator of Datasette and llm CLI (2024)

The evals.json file is where you make the specification tight enough to test against. On SWE-bench Verified — the standard benchmark for autonomous coding agents — model scores rose from 40% to over 80% in a single year, a gain that correlates directly with tighter eval design and faster iteration cycles (Anthropic Engineering, "Demystifying Evals for AI Agents," 2025). The same principle applies at the skill level: the feedback loop only tightens when the target is defined.

What should I put in the expected_behavior array?

Each item in expected_behavior is a plain-language assertion about what the output must or must not do: write at the behavioral level, not the exact-string level, so the assertion holds across valid variations in phrasing and still fails when the skill does the wrong thing.

What works:

• "output includes a numbered list of findings"
• "skill does NOT trigger on this prompt"
• "Claude asks for clarification before proceeding"
• "output stays under 500 words"
• "each item includes a severity: critical, high, medium, or low"

What does not work:

• "output contains the phrase 'Security Review Complete'" (brittle exact-string match)
• "output is good quality" (untestable)
• "Claude does the right thing" (not a spec)

Write 3-5 assertions per test case. Fewer than 3 and the test does not constrain behavior meaningfully. More than 5 and the test becomes hard to evaluate in a single pass. The Anthropic engineering team found that fixing evaluation bugs on CORE-Bench took Claude Opus 4.5's reported score from 42% to 95% — a 53-point swing caused entirely by how the test criteria were written, not by any change to the model (Anthropic Engineering, "Demystifying Evals for AI Agents," 2025). What you assert determines what you measure.

Negative assertions, items that use "does NOT," are as important as positive ones. They define the skill's scope boundaries. A code review skill that also rewrites your imports is out of scope. Without a negative assertion, that failure is invisible. Scope creep is not a hypothetical: PMI's Pulse of the Profession survey found that 52% of projects completed in the year studied experienced uncontrolled scope changes (Project Management Institute, 2018). At the skill level, negative assertions are the only mechanism that makes scope boundaries testable rather than assumed.

In our builds, the most common evals.json mistake is writing only positive assertions. Every skill needs at least 3 negative assertions distributed across its trigger and quality test cases. These catch scope creep, trigger misfires, and output inflation. 60% of organizations using test automation report significant improvements in application quality (Gartner Peer Insights, 2024). In our experience building AEM production skill libraries, the failure category most consistently absent from first-draft test suites is the negative assertion.

Where does evals.json go in my skill folder?

Place evals.json directly in your skill folder at the same level as SKILL.md, where both files travel together whenever you copy, version, or share the skill, and where the test suite stays discoverable without hunting through subdirectories every time you need to add a case or run a check:

.claude/skills/your-skill-name/
  SKILL.md
  evals.json
  references/
    domain-knowledge.md

For user-level skills, this is ~/.claude/skills/your-skill-name/evals.json. For project-level skills, it is .claude/skills/your-skill-name/evals.json in your project root.

The file path is a convention, not a technical requirement. Claude Code does not scan for evals.json the way it scans for SKILL.md. The file is for your development process, not for the runtime. Where you put it matters only for your own organization. Co-location is the established pattern for tightly coupled test-code pairs: pytest's official good practices guide explicitly recommends co-locating tests with the code they cover when test and code development are tightly coupled, noting that proximity keeps both files synchronized and reduces the overhead of hunting across directories during active development.

For a full introduction to what evals are and why they exist, see What Are Evals in Claude Code Skills?.

How do I run evals.json test cases?

You run them manually in a fresh Claude session, one that has no carry-over context from your authoring work, so the test result reflects what a new user actually encounters rather than what you already know is in scope as the skill author. Anthropic recommends starting with 20-50 tasks drawn from real failures as an initial eval set, because early runs have large effect sizes and small sample sizes are enough to surface regressions (Anthropic Engineering, "Demystifying Evals for AI Agents," 2025). The steps:

1. Open a new Claude Code session in a clean terminal, separate from your development session.
2. Take the first test case prompt from evals.json.
3. Send the prompt to Claude exactly as written.
4. Check the output against each item in expected_behavior. Mark pass or fail.
5. Repeat for each test case.
6. Record which test cases failed and what the output did instead.

This is the Claude B test: a fresh session that has no context from how you built the skill. Addy Osmani documented that giving models an explicit output format with examples raises consistency from roughly 60% to over 95% (Google Chrome Engineering Director, 2024). Your evals.json assertions are the explicit format. The fresh session is where you verify that consistency.

Honest limitation: this process is manual and takes 20-40 minutes for a 15-test-case suite. There is no automated eval runner in Claude Code as of 2026. The discipline is treating it as a required step before shipping, not an optional polish step.

For the complete workflow that uses evals.json from start to finish, see Evaluation-First Skill Development: Write Tests Before Instructions.

FAQ

Does evals.json work with any skill, or only certain types?

Every skill type benefits from evals.json: formatting skills, analysis skills, publishing skills, code review skills, writing skills. The test case structure adapts to the skill. Simple formatting skills need 5-8 test cases focused on output structure. Complex multi-step research skills need 15-20 test cases covering triggers, quality, edge cases, and error recovery. The format is the same. The coverage depth varies.

What is the difference between evals.json and a rubric file?

evals.json tests binary behavior: did the skill do the thing or not? A rubric measures gradient quality: how well did the skill do the thing? Use evals.json for skills with definable correct answers. Use a rubric for skills producing subjective output where quality sits on a spectrum. For most skills with both structural requirements and quality goals, you need both. For the full comparison, see When Do I Need a Rubric vs Just Using evals.json?.

Can I add test cases to evals.json after I ship the skill?

Yes, and you should. The best source for new test cases is real failures: when a user reports unexpected behavior, write a test case that reproduces it before fixing the skill. This way the fix is verifiable and the failure does not recur undetected. Treat evals.json as a living document that grows with every confirmed failure.

What if my skill's output is too variable to write expected_behavior assertions?

If output is genuinely too variable to assert anything about, the skill does not have a defined output contract. Write the contract first: what must every output contain, what must every output avoid? Those constraints become your expected_behavior items. If you still cannot write 3 assertions, the skill's scope is not specific enough to be testable. That is a design problem worth solving before shipping.

How specific should the prompt in a test case be?

Write the prompt the way a real user, not the skill author, would phrase the request. If your trigger eval prompt contains the word "security review" and the skill name is "security-review," you are testing invocation, not triggering. The value of trigger evals comes from testing natural language: "look at this code," "check this for issues," "is this function safe?" These are the prompts real users send.

Last updated: 2026-04-16

title: "What Are Evals in Claude Code Skills?" description: "Claude Code evals define correct skill behavior before live use. Learn what evals.json contains, what to test, and what failures skipping evals exposes." pubDate: "2026-04-16" category: skills tags: ["claude-code-skills", "evaluation", "evals", "testing", "beginner"] cluster: 16 cluster_name: "Evaluation System" difficulty: beginner source_question: "What are evals in Claude Code skills?" source_ref: "16.Beginner.1" word_count: 1490 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

What Are Evals in Claude Code Skills?

TL;DR: Evals are a set of test cases that define what correct behavior looks like for your skill, before you test it in a live session. Each eval contains a prompt and an expected_behavior array. You run them against your skill in a fresh session to verify it behaves as specified, not just as you assume it does.

A skill that feels correct in your authoring session and a skill that is correct in production are not the same thing. Most skill developers discover this the expensive way.

What are evals in Claude Code skills?

Evals are structured test cases for a Claude Code skill: each eval specifies a realistic user prompt and a list of behavioral assertions the skill must satisfy, and together they form a measurable definition of correct behavior that lives in a file called evals.json inside your skill folder.

Evals serve two purposes. First, they give you a measurable definition of "correct." Without evals, "correct" means "it worked when I tried it," which is not a standard you can reproduce or share. Second, they catch failures that manual testing systematically misses, specifically the failures that only appear in fresh sessions without your authoring context. Research on defect removal efficiency shows that single-method testing alone catches only 25-35% of defects; combined approaches that run tests across varied contexts achieve above 97% (Capers Jones, Applied Software Measurement; TestDino Bug Cost Report, 2024).

In Claude Code skill engineering, evals are not optional for production skills. At AEM, we require evals before any skill ships. They are the difference between a skill that works and a skill that works for you.

What does an eval test case look like?

A test case in evals.json is a self-contained test specification: it has three required fields (id, tags, prompt) plus an expected_behavior array of plain-language assertions that define what any correct output must satisfy, independent of how Claude phrases its response in a given run. The required fields are:

• id — unique identifier for the test case
• tags — one or more labels (e.g. trigger, quality, edge) used to group and filter cases
• prompt — the realistic user input the test case exercises

The expected_behavior array defines what any correct output must satisfy, regardless of how Claude phrases it in a given run; this structure separates the specification of correct behavior from any particular output wording. Here is a concrete example from a code review skill:

{
  "test_cases": [
    {
      "id": "TC001",
      "tags": ["trigger", "quality"],
      "prompt": "Review this diff for security vulnerabilities",
      "expected_behavior": [
        "skill triggers without explicit /code-review invocation",
        "output includes a numbered list of findings",
        "each finding includes a severity level: critical, high, medium, or low",
        "output does NOT include unrequested style or refactoring suggestions"
      ]
    },
    {
      "id": "TC002",
      "tags": ["trigger", "negative"],
      "prompt": "Write a commit message for these changes",
      "expected_behavior": [
        "code-review skill does NOT trigger on this prompt",
        "no security findings section appears in output"
      ]
    }
  ]
}

The expected_behavior items are plain-language assertions, not exact strings. They define what constraints any correct output must satisfy. This matters because Claude's output varies across runs. The assertions test the structure and behavior of the output, not its exact wording.

One test case covers one scenario. A production skill needs 10-20 test cases to have meaningful coverage: positive triggers, negative triggers, canonical quality inputs, and at least 2 edge cases. Anthropic's own eval guidance recommends prioritizing volume: "more questions with slightly lower signal automated grading is better than fewer questions with high-quality human hand-graded evals" (Anthropic, Define success criteria and build evaluations, 2024). For teams starting their first eval suite, Anthropic's engineering team recommends "20-50 simple tasks drawn from real failures" as an initial baseline before scaling coverage (Anthropic, Demystifying evals for AI agents, 2025).

"When you give a model an explicit output format with examples, consistency goes from around 60% to over 95% in our benchmarks." -- Addy Osmani, Engineering Director, Google Chrome (2024)

Evals are the specification that makes this consistency measurable. Without them, you have no baseline.

How is an eval different from just running your skill manually?

Manual testing catches the failures that happen to occur in your current session, but it systematically misses the failures that happen because of your current session — the ones caused by context you carry as the author that a fresh user session will never have.

When you build a skill, your authoring session contains context that a fresh user session does not:

• the conversation history
• implicit assumptions from prior exchanges
• your own understanding of what the skill is meant to do

Claude in your session uses all of that context. Claude in a fresh user session has none of it.

This is the Claude A vs Claude B distinction. Claude A (your authoring session) consistently passes manual tests. Claude B (a fresh user session) regularly fails on inputs Claude A handled. We documented this failure pattern in 6 of 10 consecutive commissions at AEM where the author relied on manual testing without evals. In each case, the failure was invisible until a different user triggered the skill.

Evals fix this by requiring you to run tests in a fresh session, with no authoring context, against a pre-specified definition of correct behavior. They make the gap visible before it reaches production.

For the complete overview of what eval-first development looks like in practice, see Evaluation-First Skill Development: Write Tests Before Instructions.

What should I test with evals?

Test two fundamentally different things — trigger behavior and output quality — and keep them in separate test cases, because they capture different failure modes: a skill can trigger correctly but produce poor output, or produce perfect output but only activate on a narrow slice of the prompts your users will actually write.

Trigger behavior: Does the skill activate on the right prompts? Does it stay dormant when it should? Trigger evals need 3-5 positive cases (prompts that should activate the skill) and 3-5 negative cases (prompts that should not). A skill with perfect output behavior that triggers only 50% of the time has failed in production. Research on LLM-based agent systems finds that even well-configured agents succeed on roughly 50% of tasks without structured evaluation to identify and close trigger gaps (Getmaxim.ai, Diagnosing and Measuring AI Agent Failures, 2024). A 2025 reliability study across 14 agentic models found that "outcome consistency remains low across all models" — agents regularly fail tasks they are capable of completing when tested across multiple independent runs rather than single-session checks (Rabanser et al., Towards a Science of AI Agent Reliability, arXiv 2602.16666, 2025).

Output quality: Once triggered, does the skill produce output that meets your spec? Quality evals need:

• 1 canonical case — your best expected input, representing the most common real use
• 2-3 variation cases — inputs that reflect real user diversity in phrasing or context
• 1-2 edge cases — inputs that test the skill's documented limits

Do not write both types in the same test case. A trigger eval assertion and a quality eval assertion measure different failure modes. Mixing them makes it harder to diagnose which kind of failure occurred.

The pattern we use at AEM: write trigger evals before writing the description, write quality evals before writing the instructions body. The first set shapes the description. The second set shapes the steps. The specification precedes the implementation.

For a detailed breakdown of evals.json structure and field definitions, see What is an evals.json file?.

What happens if I skip evals?

A skill without evals has one test environment (your authoring session), one tester (you), and one success criterion (it worked when you tried it) — which means every failure mode that only surfaces in fresh sessions, on natural phrasing variants, or without your implicit context goes undetected until a real user finds it.

That is not a production bar. That is a personal bar, which is a different thing.

The structural reason is non-determinism: LLMs "process each interaction independently, lacking native mechanisms to maintain state across sequential interactions," which means a skill that passes in your session is not guaranteed to pass in anyone else's (Survey on Evaluation of LLM-based Agents, arXiv 2503.16416, 2025). Single-run manual testing cannot detect variance — only repeated testing across fresh sessions can.

In our commissions, skills shipped without evals have a measurably higher rate of user-reported failures in the first two weeks of use. The failures are not random. They cluster around three patterns:

• Trigger misfires — skill activates when it should not
• Trigger gaps — skill does not activate on natural phrasing
• Context dependency — skill works only when the user provides implicit setup Claude B cannot infer

All three are detectable with evals before launch. Skip evals and you find them after. The DORA 2024 report found that elite-performing engineering teams achieve an 8x lower change failure rate than low-performing teams — the differentiating factor being systematic pre-release verification rather than post-release discovery (Google DORA, Accelerate State of DevOps, 2024).

For a full list of the failure patterns that evals prevent, see What Are the Most Common Mistakes When Building Claude Code Skills?.

FAQ

How many evals do I need before my skill is ready to use?

Ten is the minimum for a production skill: 5 trigger evals and 5 quality evals, covering the basic failure modes — activation, suppression, canonical quality, phrasing variations, and edge cases — without creating a maintenance burden that outweighs the coverage you gain. For a simple formatting skill, 10 is sufficient. For a multi-step research or analysis skill, 15-20 is more appropriate. Below 10, you are running a personal bar check, not a production bar check.

Can I write evals after I have already written the SKILL.md?

Yes, but you lose the main benefit: writing evals after instructions produces tests that confirm what you built rather than tests that specify what you needed, because an existing implementation biases you toward assertions it already passes instead of assertions that expose its gaps. Post-implementation evals still catch future regressions and are better than no evals.

Do I run evals manually or is there a tool?

Currently, you run evals manually in a fresh Claude session with no authoring carry-over, which means each test prompt is evaluated in exactly the same clean state a real user would encounter. There is no automated eval runner built into Claude Code as of 2026. The discipline is in treating this as a required step, not an optional one.

1. Open a new Claude Code session with no prior context from your development work.
2. Paste each test case prompt exactly as written in evals.json.
3. Check the output against each expected_behavior assertion. Mark pass or fail.

What file does my evals.json go in?

Place evals.json inside your skill folder alongside SKILL.md, using the path .claude/skills/your-skill-name/evals.json for both user-level and project-level skills, so the test suite travels with the skill whenever you copy, share, or version-control it. The evals file does not load into Claude's context at startup. It is a developer tool, not a runtime file.

Should I include evals for edge cases I have not seen yet?

Yes, and prioritize the failure modes most likely to occur before you have seen them in production: unusual phrasing, minimal or ambiguous input, prompts semantically adjacent to a different skill's trigger, and inputs that test the skill's documented limitations. Edge case evals written from imagination are less valuable than edge cases discovered from real failures, but they catch a meaningful class of failures that canonical tests miss entirely.

Last updated: 2026-04-16

title: "How Specific Should My Skill Instructions Be?" description: "Match instruction specificity to task fragility: fragile operations need exact step-by-step scripts, judgment-based tasks need principles and criteria." pubDate: "2026-04-15" category: skills tags: ["claude-code-skills", "degrees-of-freedom", "skill-instructions", "specificity"] cluster: 15 cluster_name: "Degrees of Freedom & Instruction Specificity" difficulty: beginner source_question: "How specific should my skill instructions be?" source_ref: "15.Beginner.1" word_count: 1520 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

TL;DR: As specific as the task's fragility requires. Publishing, API calls, and database writes are hard to reverse and need exact step-by-step instructions. Drafting, evaluating, and reviewing need principles and criteria, not scripts. The deciding factor is how badly a wrong call plays out and how reversible it is.

What is task fragility and why does it determine specificity?

In AEM's skill methodology, task fragility measures how much damage a wrong decision causes and how hard that damage is to undo — a high-fragility task produces real consequences that are difficult or impossible to reverse, while a low-fragility task produces output that is easy to notice, correct, and redo.

A high-fragility task: posting to a live platform, running a database migration, sending an email to a customer list. Wrong decision, real consequence, hard to reverse. These tasks need specific instructions that leave Claude no room to improvise.

A low-fragility task: drafting a blog post, evaluating code quality, suggesting a project structure. Wrong decision, easy to notice, easy to redo. These tasks benefit from principles and criteria because the right output depends on context Claude needs to judge, not execute.

The error developers make is applying the wrong specificity level to the task type. A publish skill that says "post when the draft looks ready" is not a skill. It's a guess with good intentions. A content evaluation skill that says "follow these 47 sub-steps to assess quality" is not a skill. It's a checklist that breaks the moment the content doesn't fit the template.

One limit of this framework: it does not resolve ambiguity in the task goal itself. If the objective is unclear, no specificity level fixes that — the skill needs a clearer brief before calibration makes sense.

In the AgentIF benchmark, even the best-evaluated model achieved a 27% instruction success rate across complex agentic tasks — meaning 73% of instructions produced at least one constraint violation. Performance degraded further as instruction complexity increased. (Qi et al., NeurIPS 2025)

When should I use exact step-by-step instructions?

Use exact steps when the operation has a correct sequence, deviation from that sequence causes errors, and Claude cannot safely improvise — meaning the task involves an irreversible action, an external system with specific authentication, or a multi-step workflow where each step depends on the exact output of the one before it.

Characteristics of tasks that need exact steps:

• Irreversible actions (publish, send, deploy, delete)
• External API calls with specific authentication requirements
• File system operations with specific paths and formats
• Multi-step workflows where step 4 depends on the exact output of step 3

In our commission work, we replace prose guidance with numbered steps the moment a skill involves an irreversible external action. A publish-to-production skill broke a client's deployment twice when the body contained 40 lines of principles. We replaced those 40 lines with 15 numbered steps. No failures since.

An example of what exact steps look like for a publishing workflow:

Step 1: Read the draft file at the path provided by the user.
Step 2: Check that the frontmatter contains: title, pubDate, and status fields. If any are missing, stop and report the missing fields to the user before proceeding.
Step 3: Run the linting check by executing: node validate-frontmatter.js [file-path]
Step 4: Only if step 3 passes, set status to "published" in the frontmatter.
Step 5: Write the updated file.
Step 6: Report the exact file path and new status to the user.

No judgment required. No room for improvisation. Each step has one correct action.

Across 16 LLM agents evaluated on Agent-SafetyBench, none achieved a safety score above 60% on tasks involving multi-step tool use — with failures most severe in scenarios where instructions left ambiguous the permissibility of irreversible actions. (Zhang et al., 2024)

"The failure mode isn't that the model is bad at the task, it's that the task wasn't specified tightly enough. Almost every production failure traces back to an ambiguous instruction." - Simon Willison, creator of Datasette and llm CLI (2024)

When should I use principles instead of exact steps?

Use principles when the right output depends on judgment that varies with context, and specifying every case explicitly would require hundreds of steps that still would not cover all inputs — this applies to quality evaluation, content generation with variable inputs, and any task where the output format flexes to match what the user provides.

Characteristics of tasks that benefit from principles:

• Quality evaluation (code review, writing critique, design assessment)
• Content generation with quality standards
• Brainstorming and exploration tasks
• Tasks where the inputs are highly variable and the output format flexes to match

A code review skill doesn't need steps. It needs evaluation criteria:

Review the provided code against these criteria:
1. Security: flag any input that reaches a database or external service without validation
2. Error handling: every function that calls an external API must handle the failure case explicitly
3. Readability: flag functions over 30 lines as candidates for extraction
4. Test coverage: flag public functions without a corresponding test

For each finding, state: the line or function, the criterion violated, and a specific fix.

This is not a step sequence. It's a rubric. Claude applies judgment about whether the code meets each criterion. The criteria are specific (30 lines, not "too long"). The application requires judgment.

G-Eval research confirmed the advantage of rubric decomposition: GPT-4 guided by explicit sub-criteria achieved a 0.514 Spearman correlation with human raters on summarisation tasks, outperforming all prior automated evaluation methods that used holistic, criterion-free approaches. (Liu et al., EMNLP 2023)

What does "too specific" look like?

A skill is too specific when its step sequence breaks on inputs that do not match the template exactly, causing Claude to either halt with an error at the point of mismatch or improvise in a way that corrupts the remaining sequence and produces output the user cannot trust.

The failure mode: Claude reaches step 7, finds the input doesn't match what step 7 expects, and either halts with an error or improvises in a way that corrupts the sequence. The skill handles the 80% of inputs that fit the template. It fails on the 20% that don't.

Signs a skill is too specific:

• The body specifies exact string values to look for in inputs
• Steps assume a fixed input format that users don't always provide
• The skill only works when invoked with a specific command structure
• Edge case inputs consistently produce wrong or hallucinated outputs

The fix is not to add more steps for the edge cases. The fix is to add explicit branch handling: "If the input does not contain X, ask the user for X before proceeding to step 3."

Research on format-task interference shows that embedding rigid output format requirements in the same instruction body as reasoning goals degrades task performance by a measurable margin — separating the two concerns yields 1–6% relative improvement in task quality. (Deng et al., 2024)

What does "too vague" look like?

A skill is too vague when Claude fills the gaps with plausible-but-wrong behavior, meaning the output varies between sessions, between model versions, and between users because the instructions leave enough ambiguity for Claude's training defaults to substitute for missing explicit rules.

The failure mode: output varies between sessions, between model versions, or between users. The skill "works" on simple cases because Claude's training data covers those cases. It fails on edge cases because Claude guesses, and the guess is wrong.

Signs a skill is too vague:

• The output format differs across invocations
• Claude skips steps that aren't mandatory in the body text
• Different users get different quality from the same skill
• The skill works on Claude Opus but fails on Haiku

The fix is not to write even longer prose. The fix is to replace explanatory paragraphs with specific decision rules: instead of "ensure the output is well-formatted," write "format the output as a numbered list with each item under 40 words."

Studies of LLM output consistency show accuracy fluctuations of up to 10% across identical inference runs under ambiguous instructions, with vague prompts demonstrably amplifying variance relative to explicitly structured equivalents. (Krugmann & Hartmann, 2024; Frontiers in AI, 2025)

Microsoft Research's LLMLingua work found that removing low-information tokens — principally verbose, explanatory text — from prompts achieves up to 20x compression with only ~1.5-point performance loss, confirming that loose prose is the low-value fraction of any instruction body. (Jiang et al., EMNLP 2023)

For troubleshooting skills with inconsistent output, see Why Isn't My Claude Code Skill Working?.

How does specificity affect how much token budget the body uses?

More specific bodies are not necessarily longer, and in practice a 15-step sequence with precise, short imperative commands is often shorter in total word count than the 40 lines of explanatory prose it replaces — specificity is about precision, not volume.

A 15-step sequence with precise commands is often shorter than 40 lines of explanatory prose. Short, imperative sentences ("Read the file at the path provided") are more token-efficient than explanatory constructions ("In order to begin the process, Claude should first open and read the file that the user has specified in their request"). Research on symbolic instruction compression found that replacing natural-language prose with compact, imperative instruction formats reduces token usage by 62–81% across task categories while preserving semantic intent — with selection and classification tasks showing the largest reduction (80.9%). (Jha et al., arXiv:2601.07354, 2026)

The token cost of the body depends on word count, not on how prescriptive the instructions are. Specific, short imperative steps often cost fewer tokens than the vague prose they replace.

Structured formats (numbered steps, headers, delimiters) carry a small formatting overhead in tokens relative to equivalent bare prose — the tradeoff is between token efficiency and model parseability, and for most production skill bodies the compliance improvement outweighs the token cost.

For more on body loading and token economics, see When Does the SKILL.md Body Get Loaded into Context?.

FAQ: How specific to make skill instructions

Should I start with specific steps or general principles when building a new skill? Start with principles, test on real inputs, and make specific only the steps where Claude's output is inconsistent or wrong. Over-specifying from the start produces brittle skills. Under-specifying from the start tells you where the gaps are.

What's the right level of specificity for a skill that produces creative output? Specific criteria for quality evaluation, open framework for execution. Tell Claude what makes the output good (criteria) without specifying exactly how to produce it (steps). "Each paragraph must make one claim, supported by one example or piece of evidence" is a specific quality criterion that allows creative flexibility in execution.

Can a skill have some exact steps and some principle-based sections? Yes. Most production skills do. Irreversible sub-tasks (posting, saving, sending) get exact steps. Judgment-based sub-tasks (drafting, evaluating, selecting) get principles. The body makes the boundary explicit: "Steps 1-3 are mandatory sequence. Step 4 applies these criteria to evaluate the draft."

My skill produces different output each time I run it. Is that a specificity problem? Yes, almost certainly. Variation across invocations on the same input is the signature of vague instructions that Claude fills differently each time. Identify the output dimension that varies and add a specific rule governing it.

How do I write instructions that work across Claude Haiku, Sonnet, and Opus? Write for the lowest-capability model you expect to run the skill. Haiku follows explicit, numbered steps more reliably than it follows principles. A skill that works on Haiku with specific steps will work on Sonnet and Opus with better judgment applied to those same steps.

Last updated: 2026-04-15

title: "Why Can't I Just Put Everything in One Big SKILL.md File?" description: "A monolithic SKILL.md loads into context on every trigger, burns tokens whether they're needed or not, and degrades instruction compliance as it grows." pubDate: "2026-04-14" primary_keyword: "SKILL.md file" category: skills tags: ["claude-code-skills", "progressive-disclosure", "skill-design", "beginner"] cluster: 14 cluster_name: "Progressive Disclosure Architecture" difficulty: beginner source_question: "Why can't I just put everything in one big SKILL.md file?" source_ref: "14.Beginner.2" word_count: 1520 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

Why Can't I Just Put Everything in One Big SKILL.md File?

TL;DR: You can, and for simple skills it's fine. The problem starts when your skill grows past 1,000-1,200 tokens. At that size, every task trigger loads the full file into context, including parts irrelevant to the current task. Token costs climb and instruction compliance drops.

A 4,000-token SKILL.md is not a skill. It's a small novel that Claude has to read every time you ask it to do anything.

That's the core problem with monolithic skill files: they don't distinguish between content that's needed on every run and content that's needed on some runs. Everything loads together, always. A rubric you use twice a week loads at the same time as the trigger logic that fires every session. A 200-line vocabulary list loads even when you're doing a task that never touches vocabulary.

The solution — what AEM calls progressive disclosure architecture — is to split skills into a lean always-loaded body and conditionally-loaded reference files. The bigger the file, the more you're paying for content you don't need.

What actually happens when you put everything in one SKILL.md file?

When your skill triggers, Claude reads the entire SKILL.md body into context in one operation — every token, every section, every piece of content regardless of whether the current task needs it, including rubrics, vocabulary lists, and example libraries that the current task will never use, paying for content and diluting the instructions that matter. At 400 tokens that's invisible. At 2,000 tokens you're paying for content you don't need and diluting the instructions you do.

That's fine at 400 tokens. It gets expensive at 1,500 tokens. At 3,000 tokens, you're loading a file that likely contains three or four distinct types of content:

• instructions
• a rubric
• a vocabulary list
• examples

when your current task might only need the instructions and the rubric.

The second problem is instruction density. Every token you add to a SKILL.md body is one more thing Claude has to hold in working attention while executing the task. Stanford's NLP Group found that model accuracy on instruction-following drops when instructions are embedded in longer contexts ("Lost in the Middle," Nelson Liu et al., ArXiv 2307.03172, 2023). A 400-token body keeps your instructions tight and salient. A 3,000-token body buries the key instructions in surrounding content.

We've seen this in practice at AEM. Skills that perform well at 600 tokens start showing instruction dropout when padded to 2,000. The model doesn't fail to read the file — it fails to weight the critical constraints correctly when they're competing with pages of secondary content.

What does this look like at scale?

At scale, monolithic skill files compound fast: a 10-skill library with 1,500-token bodies each consumes 15,000 tokens of skill overhead before your first task runs — and that number doubles or triples the moment you add rubrics, vocabulary sections, or example libraries to each body, pushing total overhead past 30,000-40,000 tokens per session.

Imagine a library of 10 skills, each with a monolithic SKILL.md averaging 1,500 tokens. No reference files, just big flat files.

Every session start loads the description index for all 10 skills (approximately 500-1,000 tokens). Then every triggered skill loads 1,500 tokens. On a typical session where 3-4 skills fire across multiple tasks, you're consuming 4,500-6,000 tokens in skill-body loads, on top of the session index.

Now add reference-style content to those bodies. A skill with a built-in rubric, an example library, and a vocabulary section hits 3,000-4,000 tokens easily. That same library of 10 skills now loads 30,000-40,000 tokens per session, depending on which skills activate.

At 40,000 tokens of skill overhead, your actual task content doesn't start until position 40,000 in the context window. According to Anthropic's documentation, Claude Sonnet's context window is 200,000 tokens (2024). You've used 20% of it on skill infrastructure before writing a single line of your task. At Anthropic's published input rate of $3 per million tokens for Claude Sonnet (Anthropic pricing page, 2025), that 40,000-token overhead costs roughly $0.12 per session in input alone — before your actual task content has run a single instruction. And it compounds: research by Freda Shi et al. found that model accuracy drops dramatically when irrelevant information is embedded in the input context, even when the model has access to all the information it needs ("Large Language Models Can Be Easily Distracted by Irrelevant Context," ArXiv 2302.00093, 2023). A 2024 benchmark study found that most state-of-the-art LLMs — including models with 128K context windows — show measurable performance degradation on retrieval and instruction tasks once effective context utilization passes roughly 50% of the stated window size (Hsieh et al., "RULER: What's the Real Context Size of Your LLM?," ArXiv 2404.06654, 2024).

"When you give a model an explicit output format with examples, consistency goes from ~60% to over 95% in our benchmarks." — Addy Osmani, Engineering Director, Google Chrome (2024)

The inverse also holds: when you give a model a 4,000-token file of mixed content instead of a focused 800-token instruction body, consistency drops. The format matters. So does the size.

How does splitting across files fix this?

Splitting moves content out of the always-loaded body and into conditionally-loaded reference files, so a security rubric only loads when the task calls for a security review, a vocabulary list only loads when the task requires terminology checks, and the base body stays under 900 tokens on every run.

The rule from progressive disclosure architecture:

• Stay in the SKILL.md body: trigger logic, core instructions, output format, step sequence, constraints that apply to every run.
• Move to reference files: rubrics, checklists with more than 8 items, vocabulary lists, style guides, brand guidelines, example libraries, comparison tables.

A well-split skill has a 600-900 token body with explicit Read instructions pointing to reference files. The body loads when the skill triggers. Reference files load when the body's instructions call for them.

A code-review skill with a built-in 1,800-token security rubric becomes a 700-token body with a line that says "Load references/security-rubric.md when the user requests a security review." The rubric loads only when needed. A standard code review never touches it.

This does require one additional discipline: the Read instructions in the body have to be explicit. "Load references/security-rubric.md before scoring security posture" is a correct instruction. "Use the security guidelines" is not. Claude won't know where to find them.

For a full walkthrough of what belongs in each SKILL.md section, see What Goes in a SKILL.md File?.

What should and shouldn't be in the main SKILL.md body?

The SKILL.md body should contain exactly what Claude needs on every single task run: trigger logic, operating steps, output format, universal constraints, and Read instructions pointing to reference files — nothing more, because every additional token inflates the cost of every trigger without improving the runs that don't need that content. Everything else — rubrics, checklists longer than 8 items, vocabulary lists, style guides, example libraries — belongs in a reference file.

Include in the body:

• The description field (trigger condition and skill summary)
• The operating steps or process
• The output format specification
• Hard constraints that apply universally
• Read instructions for reference files (with conditional logic if needed)

Move to reference files:

• Any checklist longer than 8 items
• Rubrics used to score or evaluate output
• Domain vocabulary, jargon, or terminology lists
• Brand voice or style guidelines
• Named example libraries
• Comparative tables or decision matrices

The test: if a piece of content only affects some runs of the skill, it belongs in a reference file, not the body. Anthropic's prompt engineering documentation recommends placing the most important instructions at the start of the prompt, before supplementary content, to maximize instruction salience (Anthropic, "Prompt engineering overview," docs.anthropic.com, 2024) — a design constraint that a 3,000-token flat body structurally violates.

A practical threshold from our builds: if your SKILL.md body exceeds 1,200 tokens, audit it using the categories above. Every piece of content that doesn't belong in the body is adding dead weight to every trigger.

When is a monolithic skill file fine?

A monolithic SKILL.md is the right choice when the body stays under 600 tokens, there's no content you'd load conditionally, and the skill does one focused thing without rubrics, vocabulary sections, or example libraries that would only apply to certain runs. Simple formatters, slug generators, and comment writers fit cleanly in a flat file — no reference files needed, no overhead worth adding.

A single flat SKILL.md file works well when:

• The skill body is under 600 tokens
• There's no content you'd want to load conditionally
• The skill is simple enough that everything it needs is also short

A slug-generation skill, a file-naming formatter, a code comment writer. These are often 200-400 tokens total. No rubric, no vocabulary list, no style guide. A flat file is the right choice here.

The architecture overhead of reference files and explicit Read instructions adds maintenance complexity. Don't add it where the token savings are negligible.

The signal for when to split: you notice you've written a SKILL.md body that contains a long checklist, a rubric, or any block of content that reads more like documentation than instructions. That content belongs in a reference file. In our experience at AEM, this pattern appears in the majority of skills once a team has been building for more than a few weeks — the body grows incrementally, section by section, until the token cost and compliance problems become visible.

For the full picture of how progressive disclosure changes token economics as your library scales, see Progressive Disclosure: How Production Skills Manage Token Economics.

Frequently asked questions

The most common questions about SKILL.md file size cover three areas: whether Claude Code enforces a hard limit (it doesn't), whether a large context window solves the compliance problem (it doesn't), and how to tell in practice when a body has grown too large to perform reliably.

Is there a hard token limit for SKILL.md files? No hard limit from Claude Code. The limits are practical: bodies past 1,200 tokens start showing compliance degradation in our builds, and bodies past 2,000 tokens frequently cause instruction dropout for complex multi-step tasks.

What happens to a very long SKILL.md file in a large context window? A large context window doesn't fix the instruction compliance problem — it changes the scale at which it occurs. A 4,000-token skill body still dilutes the instruction signal compared to a focused 800-token body. Bigger window, same relative problem.

Can I use sections within one SKILL.md file instead of separate reference files? You can structure your SKILL.md with H2 sections to organize content. The file still loads in full on every trigger. Section headers improve readability for humans, but they don't change the token cost or the instruction density problem.

How do I know if my SKILL.md body is too long? Two practical signs: first, the file exceeds 1,000 tokens when you count the content below the description field. Second, you notice Claude skipping or partially following constraints that are defined late in the file.

If my skill has no reference files, does progressive disclosure still apply? Partially. The three-tier architecture is most relevant when you have reference files to split out. Without them, you still benefit from the description-based trigger system (Tier 1 and Tier 2), but there's no Tier 3 in play.

Last updated: 2026-04-14

title: "When Does the SKILL.md Body Get Loaded into Context?" description: "The SKILL.md body loads when Claude matches your prompt to the skill's description. It stays in context for the session and costs 400-1,000 tokens on trigger." pubDate: "2026-04-15" category: skills tags: ["claude-code-skills", "progressive-disclosure", "skill-body", "context-loading"] cluster: 14 cluster_name: "Progressive Disclosure Architecture" difficulty: intermediate source_question: "When does the SKILL.md body get loaded into context?" source_ref: "14.Intermediate.3" word_count: 1500 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

TL;DR: The SKILL.md body loads when Claude's description-matching classifier identifies your prompt as a match for that skill. This happens mid-conversation, not at startup. Once loaded, the body stays in context for the rest of the session. A 200-line body at an average of 5 tokens per line costs roughly 1,000 tokens on trigger. Keep the body self-contained.

What triggers the body to load?

The body loads when Claude matches your prompt to the skill's description field — specifically, when its internal classifier determines that the semantic intent of your prompt satisfies the activation condition written in that description, not when you start the session or issue a keyword command. This is intent-matching, not lexical matching, so the exact words in your prompt do not need to appear in the description.

Claude evaluates every incoming prompt against all loaded skill descriptions (which are in the system prompt from startup). When the prompt's semantic intent matches a description's activation condition, Claude invokes the skill and loads its full SKILL.md body into the context window.

The matching is semantic, not lexical. If your description says "Use when the user asks to review a pull request," the body loads for "can you look at this PR?" even though those exact words don't appear in the description. Claude is matching intent.

If no description matches, no body loads. The skill costs zero additional tokens beyond its description in the startup registry.

What's the token cost when the body loads?

A 200-line SKILL.md body at an average of 4-5 tokens per line costs 800-1,000 tokens when triggered, making it equivalent in context weight to two to four model responses, though that cost only occurs once per session per skill regardless of how many times you invoke it (AEM internal benchmarks, 2025). That is a modest one-time load — equivalent to 2-4 Claude Sonnet responses — and it does not repeat for subsequent invocations of the same skill within the same session.

Where cost becomes a problem is when bodies exceed their necessary scope. We've seen production skill builds where the body exceeded 500 lines because the developer copied reference content directly into it instead of pointing to a reference file (AEM internal benchmarks, 2025). A 500-line body costs 2,000-2,500 tokens per invocation. Over 10 invocations in one session, that's 20,000-25,000 tokens consumed by one over-stuffed skill.

The right body length is the minimum needed to handle common inputs correctly. Edge cases, specialized domain knowledge, and large lookup tables belong in reference files.

"When you give a model an explicit output format with examples, consistency goes from ~60% to over 95% in our benchmarks." - Addy Osmani, Engineering Director, Google Chrome (2024)

That figure applies directly to skill bodies. A body with an explicit output contract and format examples produces consistent output. A body that gestures at the output format without specifying it produces inconsistent output at scale.

Does the body stay in context after the skill runs?

Once the SKILL.md body loads into the context window at first trigger, it stays there for the rest of the session — it is not evicted when the skill finishes executing, and every subsequent call to that skill within the same session reads from the already-loaded body at zero additional token cost. This means subsequent invocations of the same skill cost zero additional tokens within that session, but it also means that every loaded body accumulates and reduces the space available for responses as the session grows longer.

Two implications:

Positive: Subsequent invocations of the same skill in the same session don't incur the loading cost again. The body is already in context.

Negative: A body loaded early in a session occupies context space for everything that follows. Long sessions with multiple skill invocations accumulate body content in the context window. Late-session turns have less space available for responses.

The practical mitigation is session hygiene: start new sessions for significantly different tasks rather than extending one session across unrelated work. A session that starts with a code review skill and ends with a content writing skill has two bodies loaded, neither needed for the other.

Body changes made during a session also don't take effect until the next session. If you edit SKILL.md while a skill is in use, the running session works from the body that loaded at first trigger. Changes require a session restart to apply.

What makes a good body for reliable execution?

A body is a closed specification: every decision Claude needs to make on common inputs must have an explicit answer inside it — not a hint, not a suggestion, and not a pointer to external documentation that the body doesn't explicitly tell Claude to read. When the body leaves gaps, Claude fills them with plausible behavior derived from training data — and that behavior is almost never correct for your specific use case.

For example: "Step 3: read references/api-schema.md before generating any API call code" — not "consult the API documentation if available." The first is an answer. The second is a hint.

The structure we use in production builds:

1. Trigger condition (when to use this skill, reiterated from the description for emphasis)
2. Output contract (what the skill produces and what it does NOT produce)
3. Process steps (numbered, sequential, imperative voice)
4. Rules (edge cases and failure modes with explicit handling)
5. Self-verification (the check Claude runs before delivering output)

A body in this structure runs 100-180 lines for a well-scoped skill (AEM internal benchmarks, 2025). Under 80 lines usually means the skill is underspecified. Over 250 lines usually means reference content leaked into the body, which belongs elsewhere.

For the full anatomy of a SKILL.md file and what goes in each section, see What Goes in a SKILL.md File?.

How do I know if my body is loading and being followed correctly?

The clearest signal is consistent output on first invocation in a fresh session — if the skill produces the right result the first time you trigger it with no prior iterative context, the body loaded and Claude followed it correctly from first contact. If it does not, the most common cause is instructions buried in the middle of a long body where model attention is weakest.

If the skill ignores specific instructions in the body, the most common causes are:

1. Instructions buried too deep. Stanford NLP research on context attention ("Lost in the Middle," Nelson Liu et al., 2023, ArXiv 2307.03172) found that models follow instructions at the start and end of long contexts more reliably than those in the middle. Critical instructions belong in the first 30% of your body.

2. Contradictory instructions. If process steps say one thing and the rules section says another, Claude picks the interpretation that seems most plausible. You lose.

3. Missing reference pointer. The instruction is in a reference file, but the body doesn't tell Claude to read that file. Claude executes without the content. No error appears.

For skill-specific troubleshooting, see Why Isn't My Claude Code Skill Working?.

This loading model does not help with context overflow from very long conversations. Once context is full, earlier content (including loaded skill bodies) gets compressed or dropped. For sessions that run over 50,000 tokens, start a new session before quality degrades (Anthropic Claude model documentation, 2025).

FAQ: When the SKILL.md body loads

Does the body load every time I mention a topic the skill covers, or only when I explicitly invoke it? Claude's matching is based on the description's activation condition, not topic mentions. If your description says "Use when the user asks for a full pull request review," the body loads only when that specific intent is clear, not whenever code comes up.

Can I have a skill that loads its body at session startup instead of on-demand? Not through the standard skill mechanism. All bodies load on demand. Content you need always available belongs in CLAUDE.md, which loads at startup.

What is the practical upper limit for body length before Claude stops following it reliably? Quality holds up to approximately 300 lines. Above 350 lines, adherence to instructions in the middle of the body starts to degrade, as documented in Stanford NLP's "lost in the middle" research (2023). Instructions in positions 30%-70% of the body are less reliably followed than those at the start or end.

If I update my SKILL.md during a session, does the running session see the new content? No. The body loads once when the skill first triggers in a session. Changes to the file require starting a new session to take effect.

Can one skill body trigger another skill? Not directly. A skill body can instruct Claude to perform sub-tasks, but it cannot invoke another skill's body. For multi-skill orchestration, an agent architecture is the correct pattern.

What happens if Claude loads the wrong skill body because the description matched incorrectly? The skill runs against your prompt using the wrong instructions. Output is wrong and there is no error message. The fix is a more precise description with explicit negative conditions.

Last updated: 2026-04-15

title: "What is Progressive Disclosure in Claude Code Skills?" description: "Progressive disclosure splits Claude Code skill content into three loading tiers so context costs stay flat even as your skill library grows." pubDate: "2026-04-14" category: skills tags: ["claude-code-skills", "progressive-disclosure", "beginner"] cluster: 14 cluster_name: "Progressive Disclosure Architecture" difficulty: beginner source_question: "What is progressive disclosure in Claude Code skills?" source_ref: "14.Beginner.1" word_count: 1480 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

What is Progressive Disclosure in Claude Code Skills?

TL;DR: Progressive disclosure is a loading architecture for Claude Code skills. Descriptions load at session start, the skill body loads on trigger, and reference files load on demand. The result: a library of 20 skills costs roughly the same context as a library of 5, because only one skill runs at a time.

Most developers hit a wall around their tenth skill. Claude starts getting slower. Instructions from early in the session disappear. Outputs that were reliable get inconsistent. The library that felt like a productivity tool starts feeling like a liability.

The problem isn't the skills. The problem is that all of them are loaded at once.

AEM's progressive disclosure architecture fixes this by splitting skill content across three tiers that load at different times, for different reasons, at different costs.

What is progressive disclosure in the context of skill architecture?

Progressive disclosure in Claude Code skill engineering is a three-tier loading model where Tier 1 (descriptions) loads at session start, Tier 2 (the full SKILL.md body) loads when a skill is triggered, Tier 3 (reference files) loads only when the running skill requests it, so context cost scales with the current task, not the total library size.

The term comes from UI design, where progressive disclosure means showing users only the information relevant to their current action, not everything at once. Applied to Claude Code skills, the same principle holds: load only what Claude needs for the current task, not everything in your library.

The three tiers are:

1. Descriptions (always loaded): every skill's description field loads at session start. This is the index.
2. Skill body (loaded on trigger): the full SKILL.md body loads when the user's message matches the skill's description.
3. Reference files (loaded on demand): files in the skill's references/ folder load when the running skill's instructions explicitly call for them.

A library of 20 skills with progressive disclosure loads roughly 1,000-2,000 tokens at startup (AEM production measurement, 2025). Without it, that same library loads 15,000-80,000 tokens before any task begins (AEM production measurement, 2025).

Why does progressive disclosure exist? What problem does it solve?

Progressive disclosure exists because Claude has a fixed context window, and loading an entire skill library at session start consumes that window before your task begins — every token spent on inactive skills is a token unavailable for instructions, input, and output on the task you're actually running.

Claude's context window is 200,000 tokens (Anthropic, 2024). That sounds enormous until you start loading skill libraries. A SKILL.md body averages 400-1,200 tokens (AEM production measurement, 2025). A reference file averages 500-3,000 tokens (AEM production measurement, 2025). If you load all of these at session start for 15 skills, you've consumed 15,000-60,000 tokens before your task begins.

The second problem is instruction reliability. Stanford's NLP Group found that language models lose track of instructions that appear in the middle of a long context ("Lost in the Middle," Nelson Liu et al., ArXiv 2307.03172, 2023). Instructions placed at position 50,000 in a 200,000-token context are retrieved with significantly less accuracy than the same instructions at position 1,000. When your task-specific instructions are buried under 50,000 tokens of skill library, Claude's compliance drops.

Progressive disclosure keeps task-relevant content near position zero in the context. Your current task's instructions load first. Everything else stays off.

What are the three tiers and what does each one contain?

The three tiers are the description layer, the skill body, and the reference files, each with a specific scope, loading trigger, and token cost: descriptions load always at 50-100 tokens per skill, the body loads on trigger at 400-1,200 tokens, and reference files load only when the running skill's instructions explicitly call for them.

Tier 1: The description layer

Content: the description field from each SKILL.md file. Loading trigger: session start, automatic. Token cost: 50-100 tokens per skill.

This is what Claude reads to know your skills exist. It's the persistent index. Every skill in your library contributes its description to this layer. The description must serve double duty: it's the index entry and the trigger condition simultaneously.

Tier 2: The skill body

Content: everything in the SKILL.md file below the description, the instructions, process steps, output format, constraints. Loading trigger: user message matches the skill's description. Token cost: 400-1,200 tokens per skill.

This is the working memory for your task. It loads when your skill fires, stays for the full task, and unloads when the session ends.

Tier 3: Reference files

Content: external files in the skill's references/ directory, rubrics, vocabulary lists, style guides, checklists, example libraries. Loading trigger: an explicit Read instruction in the skill body. Token cost: 500-4,000 tokens per file.

These load conditionally, when a specific task needs them. A skill with three reference files loads only the one (or two) relevant to the current task, not all three by default.

What does progressive disclosure look like in practice?

In practice, progressive disclosure collapses the startup cost of a multi-reference skill from 5,400 tokens to 75, then loads only what the current task needs: a code-review skill with three reference files costs 75 tokens at session start instead of 5,400, and a standard review run costs 2,400 tokens instead of the full library weight.

Say you have a code-review skill. It has a SKILL.md body with 900 tokens of instructions, and three reference files: security-rubric.md (1,800 tokens), code-quality-rubric.md (1,500 tokens), and documentation-rubric.md (1,200 tokens).

Without progressive disclosure, all of this loads at session start: 900 + 1,800 + 1,500 + 1,200 = 5,400 tokens, whether you're about to do a code review or not.

With progressive disclosure, session start costs 75 tokens for the description only. When you ask for a code review, the 900-token body loads. The SKILL.md body's instructions say: "Load security-rubric.md if the user requests a security review. Load code-quality-rubric.md for all reviews. Load documentation-rubric.md if the PR includes documentation changes."

A standard code review loads 900 + 1,500 = 2,400 tokens. A security-focused review loads 900 + 1,800 + 1,500 = 4,200 tokens. The token cost scales with the task complexity, not with the size of your skill library.

"Developers don't adopt AI tools because they're impressive — they adopt them because they reduce friction on tasks they repeat every day." — Marc Bara, AI product consultant (2024)

Progressive disclosure is how you keep a growing skill library from becoming the friction it was supposed to eliminate.

Do I need progressive disclosure for my skill?

Not always — progressive disclosure pays for itself once your library reaches 10 or more skills, or once any skill carries reference files exceeding 150 lines, because that is the point where startup token costs and mid-session instruction loss become measurable; below that threshold, the architecture overhead outweighs the savings.

You need progressive disclosure if:

• Your library has 10 or more skills
• Any skill has reference files with 150+ lines of content
• You're noticing Claude losing instructions mid-session as your library grows

You don't need it for:

• Libraries with 3-5 simple skills
• Skills with no reference files and SKILL.md bodies under 400 tokens
• One-off or experimental skills that won't be in long-term rotation

We measured this threshold across production skill libraries built in AEM: once you're past 10 skills or once a skill's reference content exceeds 150 lines, the token savings from progressive disclosure outweigh the architecture overhead.

For a detailed breakdown of the actual token costs at each tier, plus how to design your skills to exploit the architecture, see Progressive Disclosure: How Production Skills Manage Token Economics.

Frequently asked questions

What's the simplest way to add progressive disclosure to an existing skill? Move any content that isn't needed on every task run into a references/ subdirectory inside your skill folder. Then add a line to your SKILL.md body: "Read references/[filename].md before [specific step]." That's the minimum viable implementation.

Does Claude load reference files automatically, or do I have to trigger them? You have to trigger them explicitly. Reference files load only when your SKILL.md body contains an instruction like "Read references/rubric.md." Claude doesn't scan the references/ directory automatically. This is by design — it gives you control over what loads when.

If I have 5 skills and none of them have reference files, is my library already using progressive disclosure? Partially. The description index (Tier 1) is always in effect for Claude Code skills. The distinction between Tier 2 and Tier 3 only applies if your skill has reference files. At 5 skills with no references, you're using the first tier correctly, and Tiers 2-3 aren't relevant yet.

How do I know how many tokens my skill descriptions are using at startup? Count the characters in each description field and divide by 4 (rough tokens-per-character estimate for English text; OpenAI and Anthropic tokenizer benchmarks, 2024). A 120-character description is roughly 30 tokens. At 20 skills averaging 120 characters each, your startup index costs around 600 tokens.

Can progressive disclosure be used with agent subskills, not just standalone skills? Yes. The same three-tier model applies when skills are invoked by agents as part of a multi-step workflow. The description still loads at startup. The body loads when the agent triggers the skill. Reference files load when the body instructs them. For multi-agent architectures, the token economics get more complex, but the per-skill cost structure stays the same.

Last updated: 2026-04-14

title: "Progressive Disclosure: What Are the Three Layers in Claude Code Skills?" description: "The three layers are metadata (always loaded), skill body (loaded on trigger), and reference files (on-demand). Each has a distinct trigger and a token cost." pubDate: "2026-04-14" category: skills tags: ["claude-code-skills", "progressive-disclosure", "skill-architecture", "intermediate"] cluster: 14 cluster_name: "Progressive Disclosure Architecture" difficulty: intermediate source_question: "What are the three layers of progressive disclosure — metadata, body, and references?" source_ref: "14.Intermediate.1" word_count: 1560 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

What Are the Three Layers of Progressive Disclosure in Claude Code Skills?

TL;DR: The three layers are: metadata (the description field, always loaded at session start), skill body (the full SKILL.md body, loaded when the skill triggers), and reference files (external markdown files, loaded on demand via explicit Read instructions). Each layer loads at a different time, costs different tokens, and carries different types of content.

The metadata layer is the bouncer. The body layer is the bar. The reference files are the back room. Most tasks only see the first two.

This article draws on production patterns from AEM (Skill-as-a-Service), a Claude Code skill library built for repeatable agentic workflows. Understanding what each layer carries and when it loads is the practical foundation of progressive disclosure architecture. Most developers who've heard of progressive disclosure can name the three layers but can't answer a precise question: what exact file path does Layer 3 load from, and what instruction triggers it? The details matter here, because vague instructions produce inconsistent loading. A 2025 Berkeley study of multi-agent LLM systems found that failing to follow task requirements was the single most common system design failure mode, accounting for 11.8% of all recorded failures — ahead of step repetition, context loss, and role misalignment (Cemri et al., "Why Do Multi-Agent LLM Systems Fail?", arXiv:2503.13657, 2025).

What is the metadata layer and what does it contain?

The metadata layer is the description field in each SKILL.md file — a single string that loads at session start, every session, regardless of whether the skill ever triggers, and controls both when the skill activates and what Claude knows it can do, at a cost of roughly 40-80 tokens per skill.

This is the only layer with zero conditional loading. It's always there. That makes it both the cheapest and the most expensive layer: cheap because it's a single short string per skill, expensive because it accumulates across your entire library.

At session start, Claude Code reads every SKILL.md file in your .claude/skills/ directory and adds each skill's description to context. A library of 20 skills with 80-character descriptions contributes roughly 800-1,200 tokens total. A library of 50 skills is 2,000-3,000 tokens. Research on agentic software engineering systems found that input tokens account for 54% of total token usage — what the authors call a "communication tax" from agents repeatedly passing full contexts back and forth (Salim et al., "Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering", MSR 2026, arXiv:2601.14470).

The description field has to answer two questions with a single string of text:

1. When should this skill activate? (trigger condition)
2. What does this skill do? (index entry)

A description that handles both correctly looks like: "Use when the user asks to review a pull request, examine code changes in a branch, or check a diff for quality or security issues." That's a trigger condition with three activation patterns and an implicit capability summary.

A description that fails one of the two jobs creates problems. Too narrow and the skill won't trigger on legitimate requests. Too broad and it fires on prompts that belong to a different skill. The balance is the hardest part of Layer 1 design.

The stakes of getting the description right extend beyond individual skills. Cemri et al. found that fixing a single agent role description in ChatDev — ensuring the CEO agent had the final say in conversations — produced a +9.4% increase in overall task success rate, with no other changes to the system (Cemri et al., "Why Do Multi-Agent LLM Systems Fail?", arXiv:2503.13657, 2025). The description field is the Layer 1 equivalent: one string, session-wide scope, no fallback if it's wrong.

For a deep analysis of how to write descriptions that pass the production bar, see The SKILL.md Description Field: The One Line That Makes or Breaks Your Skill.

What is the skill body layer and what does it contain?

The skill body is everything in the SKILL.md file below the description — the step sequence, output contract, constraints, and reference Read instructions — and it loads in full, at 600-1,000 tokens, the moment an incoming message matches the description's trigger condition.

The body carries the working instructions Claude executes from:

Body content that belongs here:

• The step sequence: "Step 1: Read the input. Step 2: Identify the relevant section. Step 3: Generate output in the specified format."
• The output contract: field names, structure, required sections, word count targets.
• Universal constraints: rules that apply to every single run of the skill, with no exceptions.
• Reference file Read instructions: "Before scoring, read references/rubric.md in full."

Body content that doesn't belong here:

• Rubrics, scoring criteria, or evaluation checklists longer than 8 items.
• Any content that's only needed for a subset of task types this skill handles.
• Reference material the model reads but doesn't act on in every run.

The production threshold from AEM's builds: a SKILL.md body should fit in 600-1,000 tokens. At 1,500 tokens, instruction compliance starts dropping. Past 2,000 tokens, the body is carrying content that should live in Layer 3. This matches the broader research pattern: GPT, Claude, and Gemini models all show an average 39% performance drop when instructions are spread across extended context rather than delivered as a tight, complete specification (Liu et al., "LLMs Get Lost In Multi-Turn Conversation", Microsoft Research, arXiv:2505.06120, 2025).

"The single biggest predictor of whether an agent works reliably is whether the instructions are written as a closed spec, not an open suggestion." — Boris Cherny, TypeScript compiler team, Anthropic (2024)

A body that stays tight and explicit loads faster, holds attention better, and produces more consistent output than a body padded with reference content.

What is the reference layer and what does it contain?

The reference layer is a set of markdown files stored in the references/ subdirectory inside your skill folder, each ranging from 500 to 5,000 tokens, and none of them load unless the skill body contains an explicit Read instruction naming that file's exact path.

A reference file is any heavyweight content that the skill needs for some tasks but not every task. Common examples:

• A scoring rubric with 20 criteria (2,000-3,000 tokens)
• A domain vocabulary list with 150 terms (1,500-2,500 tokens)
• A brand voice and style guide (1,000-4,000 tokens)
• A library of approved examples (1,000-5,000 tokens)
• A comparison table between options (500-1,500 tokens)

Keeping reference files out of unconditional context load is directly supported by performance research: models exhibit a 15-20% accuracy drop on retrieval tasks when the relevant passage is placed in the middle of a long context versus at the start — a degradation pattern that applies equally whether the middle-context content is user data or pre-loaded reference material (Liu et al., "Lost in the Middle: How Language Models Use Long Contexts", TACL 2024).

The loading trigger is an explicit Read instruction in the SKILL.md body. The instruction must name the file path exactly:

Before evaluating quality, read `references/quality-rubric.md` in full.

Claude executes this as a file Read tool call. The file's content enters context at that point in the task. Nothing happens before then. The on-demand trigger is what keeps the reference layer from becoming a liability: without it, every session start would add thousands of tokens before the first user message arrives. Salim et al. found that the Design phase — where agent instructions and role definitions are established — accounts for only 2.4% of total token consumption in an end-to-end agentic workflow, while the iterative Code Review phase alone accounts for 59.4%; the gap exists precisely because the design phase loads a small, defined set of instructions rather than accumulating context through repeated full-context passes (Salim et al., "Tokenomics", MSR 2026, arXiv:2601.14470).

Conditional loading uses straightforward conditional syntax in the body:

If the user requests a security review, read `references/security-rubric.md` before scoring.
If the PR includes documentation changes, read `references/documentation-rubric.md`.
For all reviews, read `references/code-quality-rubric.md`.

This lets a single skill serve multiple task types without loading all reference content for every run. A standard review loads one reference file. A full audit loads three. The same principle drives retrieval-augmented architectures at scale: the Tokenomics study found that in agentic software engineering systems, the primary cost lies not in initial code generation but in automated refinement and verification — stages where context accumulates unnecessarily if content is pre-loaded rather than pulled on demand (Salim et al., MSR 2026, arXiv:2601.14470).

How do the three layers interact during a task?

The three layers fire in sequence: the description loads at session start (roughly 60 tokens), the body loads on trigger (roughly 800 tokens), and reference files load mid-task only when the body instructs it — bringing a typical total to around 3,860 tokens rather than 5,400 if everything loaded unconditionally.

Suppose you have a writing-review skill with this structure:

.claude/skills/writing-review/
  SKILL.md                         (description + body)
  references/
    brand-voice.md                 (1,800 tokens)
    readability-rubric.md          (1,200 tokens)
    example-library.md             (2,400 tokens)

Session start: Claude reads the description field from SKILL.md. Cost: approximately 60 tokens. The body and references don't load.

User asks for a writing review: Claude matches the request against the description. The full SKILL.md body loads. Cost: approximately 800 tokens. The body's instructions say: "Read references/brand-voice.md before evaluating tone. Load references/readability-rubric.md for all reviews. Load references/example-library.md only if the user asks for examples."

Task execution: Claude reads brand-voice.md (1,800 tokens) and readability-rubric.md (1,200 tokens). Total reference load: 3,000 tokens. If the user didn't ask for examples, example-library.md never loads.

Total context cost for this task: 60 (description) + 800 (body) + 3,000 (two reference files) = 3,860 tokens. If all three reference files had been loaded unconditionally at session start, the cost would be 5,400 tokens for the references alone, before any task began. Keeping context lean matters: Liu et al. found that GPT-3.5-Turbo's multi-document QA performance drops by more than 20% in worst-case middle-of-context scenarios — performance in 20- and 30-document settings fell below the model's closed-book baseline entirely — because performance peaks when relevant information sits at the start of the input, and degrades when models must retrieve instructions from the middle of a densely packed context, the "Lost in the Middle" effect that unconditional preloading directly triggers (Liu et al., "Lost in the Middle: How Language Models Use Long Contexts", TACL 2024).

At scale across a library of 15-20 skills, these per-task savings compound into a context window that stays clean and keeps instruction compliance high through a full session. Cemri et al. (arXiv:2503.13657, 2025) identified context loss as a distinct failure mode in multi-agent systems, separate from instruction mis-specification — meaning even well-written skills fail if the context they depend on is crowded out by unconditionally loaded material that should have been kept in Layer 3.

For a detailed comparison of the actual token numbers at each tier across a 20-skill library, see Progressive Disclosure: How Production Skills Manage Token Economics.

Frequently asked questions

Do the three layers have to map to exactly one file each? Layer 1 (metadata) is always one field in one file. Layer 2 (skill body) is always one SKILL.md file per skill. Layer 3 (references) can be any number of files in the references/ directory. A complex skill might have 5-6 reference files. A simple skill might have none.

Can I nest reference files in subdirectories? Yes. references/rubrics/security.md is a valid path. The Read instruction in the body just needs to specify the correct relative path.

What format should reference files use? Whatever format makes the content readable and usable by Claude. Rubrics work well as numbered lists. Vocabulary lists work well as tables. Style guides work well with H2/H3 sections. There's no required schema.

If Claude loads a reference file, does it stay in context for the rest of the session? Yes. Once a reference file is read into context, it stays there for the session duration. If multiple skills use the same reference file (unusual but possible), it only needs to load once.

Can I put code scripts in reference files? Yes. A bash script, Python function, or JSON schema can live in a reference file. The skill body instructs Claude to read it, and Claude can then use the code as a template, run it as a shell command, or apply it as a format spec.

What happens if Claude can't find a reference file? Claude returns a file-not-found error for that Read call and continues execution. The skill body's instructions should account for this: either treat the reference as optional or include a fallback. A skill that silently skips a missing rubric produces inconsistent output. A skill that errors on a missing rubric tells you the architecture is broken.

Last updated: 2026-04-14

title: "How Many Tokens Does Claude Use to Store My Skill Descriptions at Startup?" description: "Each skill description uses 30-120 tokens depending on length. A 20-skill library costs 600-2,400 tokens at startup for the full description index." pubDate: "2026-04-14" category: skills tags: ["claude-code-skills", "progressive-disclosure", "token-costs", "longtail"] cluster: 14 cluster_name: "Progressive Disclosure Architecture" difficulty: longtail source_question: "How many tokens does Claude use to store my skill descriptions at startup?" source_ref: "14.LongTail.1" word_count: 1320 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

How Many Tokens Does Claude Use to Store My Skill Descriptions at Startup?

TL;DR: Each skill description uses roughly 30-120 tokens at session start, depending on its character length. A library of 20 skills with descriptions averaging 100 characters each costs approximately 500-2,000 tokens total for the startup index. That's the fixed cost of knowing your skills exist, and it's well within normal operating margins for most libraries.

The description field is the cheapest load-bearing element in AEM's skill system. It costs 30-120 tokens per skill. It runs every session, without exception. And it's the only lever that controls whether your skill triggers at all.

That combination, always-on but inexpensive, is what makes description optimization worth getting right.

How are skill descriptions loaded at session start?

At session start, Claude Code reads every SKILL.md file in your .claude/skills/ directory and loads only the description field from each one into context — not the full SKILL.md body, just the single trigger line that tells Claude what the skill does and when to activate it.

This is the Layer 1 index from the progressive disclosure architecture. It answers one question: what skills exist in this library, and when should each one activate?

Claude holds this index in context for the full session. Every user message gets evaluated against it. When a message matches a skill's description, that skill's body loads.

The description index is the only content that stays in context across the entire session regardless of which skills are triggered. Everything else is conditional.

How many tokens does a single skill description use?

A reliable planning estimate is approximately 1 token per 4 characters of English text (Anthropic Tokenizer Docs, 2024), which means an 80-character description costs roughly 20 tokens, a 150-character description costs roughly 38 tokens, and a 300-character description costs roughly 75 tokens at the model's tokenizer.

In practice, descriptions in AEM's production skill libraries run 80-200 characters. The 1-token-per-4-characters estimate puts typical descriptions at 20-50 tokens each.

Token counting for LLMs isn't perfectly deterministic, as tokenization varies by model and content type. Common English text and code tokenize differently. For practical planning purposes, the character-divided-by-4 formula is accurate enough.

For skills with multi-line descriptions, count all characters including the line breaks. A description that runs 3 lines at 80 characters each is 240 characters, or approximately 60 tokens.

What is the total token cost for different library sizes?

Using 50 tokens per description as the midpoint estimate, startup description index costs scale linearly from 250 tokens for a 5-skill library to 5,000 tokens for a 100-skill library — all well under 3% of Claude Sonnet's 200,000-token context window (Anthropic, 2024).

For context: Claude Sonnet's full context window is 200,000 tokens (Anthropic, 2024). A 20-skill library's description index consumes 0.5% of the context window. A 100-skill library consumes 2.5%.

The description index is not where token pressure comes from. The pressure comes from loading full SKILL.md bodies and reference files unnecessarily. At 20 skills with 800-token bodies each, naive full-body loading at startup costs 16,000 tokens — 16x the description index cost for the same library (AEM internal benchmark, 2025). See How Does Progressive Disclosure Save Tokens and Improve Performance? for the full comparison.

How do you reduce description token costs without losing trigger accuracy?

Short descriptions are cheaper but less precise, and long descriptions are more precise but more expensive — for most single-domain skills, the right balance lands at 80-120 characters, which costs 20-30 tokens and is specific enough to distinguish the skill from adjacent ones without wasting context budget (AEM internal benchmark across 14 production skill libraries, 2025).

Three techniques for trimming descriptions without degrading trigger accuracy:

1. Remove redundant scope qualifiers — "Use when the user asks me to review a pull request or review code changes in a branch or examine a diff" → "Use when the user asks to review a pull request, code diff, or branch changes." Same trigger coverage. 30% fewer characters.

2. Lead with the action verb, not the context setup — "When working on content for LinkedIn and the user wants to create a post or draft social copy" → "Use for drafting LinkedIn posts and social copy." The context setup ("when working on") is implied. Remove it.

3. Replace lists with category names — "Use for Python, JavaScript, TypeScript, Go, and Rust code formatting" → "Use for code formatting in any language." Unless you need to specifically exclude certain languages, the general category covers the same territory with fewer tokens.

The goal is descriptions that are as short as possible while remaining unambiguous about when to fire and when not to.

"When you give a model an explicit output format with examples, consistency goes from ~60% to over 95% in our benchmarks." — Addy Osmani, Engineering Director, Google Chrome (2024)

The same precision principle applies to trigger conditions. An explicit, specific description produces consistent activation. A vague or overly broad description produces erratic triggering, regardless of token count.

One calibration note for precise budgeting: Anthropic's model specifications show that Claude's 200,000-token context window corresponds to approximately 680,000 unicode characters — a ratio of 3.4 characters per token, slightly tighter than the 4-character rule of thumb and worth using if you are trimming descriptions to a specific token ceiling (Anthropic, 2025).

For a complete guide to writing descriptions that balance trigger precision and token efficiency, see The SKILL.md Description Field: The One Line That Makes or Breaks Your Skill.

When does the description index become a problem?

At standard library sizes under 50 skills, the description index is not a meaningful constraint: 50 skills at 50 tokens each totals 2,500 tokens, which is roughly the size of a short document and represents just 1.25% of the available context window in Claude Sonnet (Anthropic, 2024).

The description index becomes a constraint in two specific scenarios:

Scenario 1: Unusually long descriptions

If descriptions consistently run 400+ characters (100+ tokens each), a 20-skill library's index grows to 2,000+ tokens. The per-description cost is still low, but it accumulates. In AEM's audit of over-engineered skill libraries, descriptions exceeding 350 characters showed no measurable improvement in trigger accuracy over 120-character equivalents — the additional tokens are overhead, not signal (AEM internal analysis, 2025). Descriptions this long usually indicate content that should be in the SKILL.md body, not the description.

Scenario 2: Very large skill libraries

At 200+ skills, the description index approaches 10,000-15,000 tokens. Still workable, but now a meaningful portion of your startup overhead. At this scale, consider whether all skills need to be in the default .claude/skills/ directory, or whether some should be project-specific and only loaded in relevant contexts.

For most teams, neither scenario applies. The description index is the free lunch of the progressive disclosure architecture.

For a full overview of how all three loading tiers interact, see What is Progressive Disclosure in Claude Code Skills?.

Frequently asked questions

For most libraries under 50 skills, description token costs are not a meaningful constraint: the index runs 250-2,500 tokens at roughly 100 tokens per skill, consumes under 1.25% of Claude Sonnet's 200,000-token context window, and adds no measurable startup latency in libraries up to 100 skills. The threshold that matters is description length — descriptions under 60 characters risk ambiguous triggering, while descriptions over 300 characters start displacing higher-value context.

Is there a way to see exactly how many tokens my description index is using? Claude Code doesn't expose a real-time token counter in the interface. For an estimate, count the total characters in all your description fields and divide by 4. That gives you the approximate token count for your index.

Does having more skills mean slower response times at startup? Slightly. Claude reads more files at session start as your library grows. For libraries under 50 skills, the latency difference is imperceptible. The token cost matters more than the file-read latency at this scale.

If I make my descriptions shorter, will it affect how well Claude matches them? Yes, if you shorten them below the threshold of specificity needed for reliable matching. A description needs enough detail to distinguish the skill from adjacent skills. The minimum viable length is the shortest description that makes the trigger unambiguous. For most single-domain skills, that's 60-100 characters.

Are skill names also loaded at startup, or just descriptions? Skill names (the skill field or the name used to invoke the skill) are part of the startup index. Names are typically short (2-5 tokens each) and add negligible overhead.

Does the token cost of descriptions grow if I have the same skill installed in multiple projects? Each project's .claude/skills/ directory is loaded independently for that project's sessions. If you have 5 skills installed globally and 3 more in a project's local directory, that session loads 8 descriptions. There's no cross-project accumulation.

Last updated: 2026-04-14

title: "How Does the Metadata Layer Work at Startup in Claude Code Skills?" description: "The metadata layer is your skill's description field. Claude reads it at every session start and uses it as the sole basis for trigger matching." pubDate: "2026-04-15" category: skills tags: ["claude-code-skills", "progressive-disclosure", "metadata-layer", "skill-loading"] cluster: 14 cluster_name: "Progressive Disclosure Architecture" difficulty: intermediate source_question: "How does the metadata layer work at startup?" source_ref: "14.Intermediate.2" word_count: 1520 status: draft reviewed: false schema_types: ["Article", "FAQPage"]

TL;DR: The metadata layer is the description field in your skill's YAML frontmatter. Claude reads it from every installed skill at session start, assembles them into a system prompt registry at ~100 tokens per skill (Anthropic, 2024), and uses descriptions as the sole basis for trigger matching. Body content loads only when triggered.

What exactly is the metadata layer?

The metadata layer is the YAML frontmatter block at the top of your SKILL.md file in AEM (Agent Execution Model), specifically the description field — and it is the only part of your skill that Claude reads at session start, which means it is the sole signal Claude uses to decide whether your skill triggers at all. Claude Code reads this field from every installed skill at session start and assembles the descriptions into a skill registry in the system prompt.

These descriptions function as a list of available tools with their activation conditions. The name field also contributes (it becomes the tool name in the registry), but the description drives matching.

Everything else in your skill file, the process steps, the output contract, the rules, the failure modes: that's the body layer. It does not load at startup. It loads when the skill triggers.

This is not a quirk of Claude's architecture. It's a deliberate cost management decision. Descriptions are cheap summaries. Bodies are expensive instruction sets. Loading bodies at startup for skills that never get triggered would be a straight waste.

What does Claude do with all the descriptions at startup?

Claude reads all installed skill descriptions and loads them into its system prompt before you type a single message — this happens once per session, every session, so whatever you wrote in the description field is already shaping Claude's behavior before your first prompt lands. The descriptions function as a classifier input: when you send a prompt, Claude evaluates it against every loaded description to decide which skill to invoke.

Anthropic's 2024 tooling documentation confirms this evaluation uses semantic matching, not keyword search. A description reading "Use when the user asks to review a pull request" will trigger on "can you check this PR?" even though the exact words differ.

This has two practical implications for skill design:

1. Your description needs to capture the semantic intent of your triggers, not just the exact words users type.
2. Your description must exclude the semantic patterns of prompts that should NOT trigger the skill.

The cost figure of ~100 tokens per description includes more than the raw word count. The system prompt entry includes the skill name, formatting markers, and tool schema overhead alongside the description text. A 60-word description contributes approximately 100-120 tokens to the startup prompt (Anthropic token counting reference, 2024).

"The single biggest predictor of whether an agent works reliably is whether the instructions are written as a closed spec, not an open suggestion." - Boris Cherny, TypeScript compiler team, Anthropic (2024)

A vague description is an open suggestion. Claude matches against it loosely. The skill triggers incorrectly. A precise description is a closed spec for activation. The matching becomes reliable.

Why does the 1,024-character limit exist?

The description field has a hard limit of 1,024 characters (Claude Code documentation, 2024), enforced by Claude Code's tooling layer to keep the metadata layer lightweight and prevent startup costs from compounding across large skill libraries — without this cap, each additional skill would silently erode the context budget available for actual work. Without this cap, a developer with 30 installed skills could consume the entire context window before any conversation began — loading instructions Claude will never use in that session.

The limit caps each skill's startup cost.

1,024 characters is approximately 200-250 words (based on Anthropic's published token-to-character ratios, 2024). That is enough for: a precise trigger condition, 2-3 examples of what should NOT trigger the skill, and a one-line summary of what the skill produces. It is not enough for step-by-step instructions, which belong in the body.

In our production builds, descriptions average 300-450 characters. Longer is not better at this layer. A description that fills all 1,024 characters usually means step instructions leaked into the metadata layer. That is a design error: instructions in the description load at startup for every session, even when the skill never gets used.

What are the token economics of the metadata layer?

With 20 installed skills at approximately 100 tokens each, the metadata layer costs 2,000 tokens at startup — about 1% of Claude Sonnet's 200,000-token context window (Anthropic, 2024) — meaning the raw token cost of loading descriptions is cheap enough that it is rarely the thing that breaks as skill libraries grow. Scale to 100 skills and the cost reaches roughly 10,000 tokens, still a small fraction of available context, which means token cost is not the binding constraint at scale.

The binding constraint is classifier coherence.

Claude Code documentation recommends keeping active skill libraries under 30 skills for reliable discovery. Above that threshold, descriptions that are semantically similar produce trigger ambiguity: two skills both think they should respond to the same prompt. The token cost remains manageable well above 30 skills. The matching accuracy does not.

One limitation worth naming: the description-only matching model cannot handle multi-intent prompts reliably. If a user's message contains two distinct intents that map to two different skills, Claude picks one and ignores the other. The metadata layer has no mechanism for splitting a prompt or queuing multiple skill activations from a single message.

For a detailed breakdown of token economics across all three loading layers, see How Does Progressive Disclosure Save Tokens and Improve Performance?.

How should I write descriptions knowing they load at startup?

Write descriptions as activation conditions, not feature summaries — because Claude's classifier at startup needs to know when to use your skill, not what it does, and getting this wrong means your skill either triggers on prompts it shouldn't handle or stays silent on the ones it should catch. A feature summary tells Claude what the skill does; an activation condition tells it when to fire and, equally important, when not to.

A feature summary: "This skill helps with code review, pull requests, checking quality, identifying bugs, and improving code style."

An activation condition: "Use when the user provides a pull request URL and asks for a full review. Do not use for general code questions or debugging without a specific PR."

The feature summary tells Claude what the skill does. The activation condition tells Claude when to use it and when not to. Claude's classifier needs the latter.

Both positive and negative conditions belong in the description:

• Positive: "Use when the user asks to review a pull request and provides a URL."
• Negative: "Do not use for general coding questions, inline debugging, or cases where no PR URL is provided."

This is not keyword stuffing. It's giving the classifier enough signal to make the right call. For a focused guide on negative triggers, see What Are Negative Triggers and Why Should I Include Them in the Description?.

What if a code formatter breaks my description onto multiple lines?

The description must stay on a single line in the YAML frontmatter, because if Prettier or another formatter wraps it across multiple lines, Claude fails to parse the full description: the activation condition gets truncated, producing false positive triggers, failures to fire, or both — with no warning at parse time.

Two fixes:

1. Add *.md to your .prettierignore file to exclude skill files from formatting.
2. Keep your description under 80 characters. This eliminates line-wrap triggers entirely and has the secondary benefit of forcing tighter, more precise language.

Shorter descriptions are more reliable: harder to corrupt, faster to parse, and usually more precise. If your description requires 400 characters to express the activation condition, you probably need two separate skills, not a longer description.

For a complete guide to description design across all dimensions, see The SKILL.md Description Field: The One Line That Makes or Breaks Your Skill.

FAQ: The metadata layer at startup

Is the metadata layer loaded fresh every session, or cached between sessions? Fresh every session. Claude reads all skill description fields at the start of each session. There is no cross-session caching of the skill registry.

Can I have two skills with significantly overlapping descriptions? Yes, but expect unreliable triggering. When two descriptions are semantically similar, Claude's classifier picks one inconsistently. The fix is to write explicit negative conditions in each description to create a clear boundary between them.

Does the order of skills in the skills directory affect which one Claude picks when descriptions overlap? No documented ordering preference exists in Claude Code's tooling. When descriptions overlap, triggering is ambiguous regardless of file order. The correct fix is clearer, more distinct descriptions.

What happens if my description field is empty? Claude has no basis for triggering the skill automatically. The skill does not appear in Claude's auto-trigger registry. You can still invoke it manually by name, but it will never activate on natural language prompts.

What's the practical maximum number of installed skills before the metadata layer causes performance problems? Claude Code documentation cites approximately 30 skills as the discovery threshold. Above that, semantically similar descriptions create ambiguity. Token cost stays manageable above 30. Classifier accuracy does not. There is a hard cap on the metadata budget as well: the combined description and when_to_use text per skill is truncated at 1,536 characters in the skill listing (Claude Code documentation, 2025), so descriptions that exceed that limit are silently cut regardless of how many skills are installed.

Last updated: 2026-04-15