"It worked yesterday" tells you nothing about what broke. True non-determinism in skill behavior is rare. Claude's temperature is non-zero, so identical prompts in identical contexts don't always produce identical outputs, but the variation should be in output quality and style, not in whether the skill follows its instructions. When a skill works correctly in 60% of sessions and fails in 40%, the cause is almost always environmental, not random.
TL;DR: Non-deterministic skill behavior is rarely caused by model randomness. The most common causes are context bleed from prior session content, session-state variability (which tools loaded, what appeared in prior messages), and ambiguous instructions that Claude interprets differently depending on what it read recently. Test in a stripped-down fresh session. If the fresh session passes consistently, the problem is context. If not, the problem is the skill instructions.
What's the Difference Between True Non-Determinism and Environmental Variation?
True non-determinism: Claude uses sampling with a non-zero temperature. The same prompt in the same context produces slightly different outputs each time. This affects output quality, word choice, and style. Not whether steps are followed.
Environmental variation: The session state changes between runs. Different amounts of prior context, different tools loaded, different skills in the library, different conversation history. This produces behavior that looks random but isn't. The skill fails consistently in sessions with characteristic X and passes in sessions without it.
Most "non-deterministic" skill failures are environmental variation. The skill works in a test session because the test session doesn't have the contaminating variable. It fails in production because production sessions always carry that variable.
The diagnostic question: can you reproduce the failure in a controlled environment? If you can reproduce it by replicating the contaminating condition, it's environmental. If you can't reproduce it even in a controlled environment with the same input, it's genuine sampling variation.
How Do You Isolate Environmental Variables?
Three environmental variables cause most non-deterministic failures:
1. Prior conversation content (context bleed). A production session that started three hours ago has accumulated 50 messages of context. The skill executes inside a context window that includes all of that. Earlier messages prime Claude to interpret the current prompt differently.
Test: Run the failing skill in a completely fresh session with only the skill trigger as the first message. If it passes, the prior context is the contaminating variable. Nelson Liu et al. at Stanford found that instructions placed in the middle of long contexts are attended to at significantly lower rates than instructions at the start or end (ArXiv 2307.03172, 2023). A skill that works at session start will fail at message 40 if its instructions land in the middle of a dense context window.
2. Tool availability. If your skill references MCP tools, the set of available tools differs between sessions when servers restart, auth expires, or configs change. Test in a session where you've verified the expected tools are active before running the skill.
3. Ambiguous instructions that produce different behavior depending on recent context. An instruction like "Write in the established style" produces different output if earlier in the session Claude was working in a formal context vs a casual one. "Established" is relative to what's in context. Replace relative instructions with absolute ones: "Write in first-person, under 200 words, using the sentence rhythm from the examples in assets/style-examples.md."
How Do You Build a Reliable Reproduction Case?
If you can't reproduce the failure, you can't verify the fix. Building a reliable reproduction case requires capturing the environmental state:
Recreate the context state. Before running the skill in your test session, paste in a representative sample of the prior conversation that was present when the failure occurred. 20-30 messages is enough to recreate the context effect.
Use the exact prompt. Paraphrasing changes the semantic content. Use the exact words that caused the failure.
Verify the skill library is identical. If the failure occurred in a project with 18 skills loaded, test with 18 skills. Remove any skills added since the failure occurred.
Test five times. If the failure is environmental, it reproduces consistently once you've captured the conditions. If it only reproduces 1 in 5 times even with the captured conditions, there's a sampling-variation component.
In commission builds, 80% of "can't reproduce" failures become reproducible once the context state is captured. The other 20% are genuine sampling variation, and those require a different fix.
How Do You Fix Non-Deterministic Failures?
Fix for context bleed: Add an explicit instruction that overrides prior context. "Ignore the style conventions used earlier in this session. Apply only the rules in assets/style-rules.md." Explicit override instructions outperform hoping the skill's instructions dominate the existing context.
Fix for ambiguous instructions: Replace every relative instruction with an absolute one. "Match the tone of the previous output" becomes "Use the exact tone parameters listed in references/brand-voice.md: second-person, present tense, 9-14 word average sentence length." Ambiguous instructions are the root cause of most sampling variation. The model has more interpretive freedom, so it uses it.
Fix for genuine sampling variation: Use the verifier pattern. Have a second pass check the output against specific assertions before returning it to the user. "After generating the output, verify it contains at least one example, at least three steps, and no exclamation marks. If any assertion fails, regenerate the specific section." For more on this approach, see What Is the Verifier Pattern in Claude Code Skills?.
"When you give a model an explicit output format with examples, consistency goes from around 60% to over 95% in our benchmarks." — Addy Osmani, Engineering Director, Google Chrome (2024)
Fix for tool availability: Add an explicit tool check at the start of the skill. "Before executing, verify [ToolName] is available. If unavailable, stop and inform the user which tool is missing before proceeding."
What Are the Signs That a Failure Is Truly Non-Deterministic vs Environmental?
Signs it's truly non-deterministic: it fails in fresh, minimal sessions; the output format or a specific claim changes between runs with identical inputs in identical conditions; the failure is about output quality, not about steps being skipped or instructions ignored.
Signs it's environmental: it fails in production sessions but passes in fresh ones; failure correlates with session length, topic, or prior tool usage; adding a "reset context" instruction stops the failure.
If the failure is truly non-deterministic, the fix is specificity: more concrete instructions, explicit examples, and verifier checks. If it's environmental, isolate the variable and address it directly.
For related debugging approaches on session-based failures, see Why Does My Skill Work in One Session but Fail in Another?.
Frequently Asked Questions
What temperature does Claude Code use and how does that affect skill output? Claude Code's temperature settings are managed by Anthropic and aren't configurable by users. The default temperature produces some variation in output style and wording between identical runs. It's enough to affect creative choices but not enough to cause a skill to skip steps or violate explicit instructions. If steps are being skipped, the cause is instruction clarity or context interference, not temperature.
Can I make my skill 100% deterministic? Not completely. The model uses sampling, so some variation is inherent. But you can make it highly reliable. Explicit output formats, concrete examples as reference points, verifier checks on specific assertions, and absolute rather than relative instructions bring consistency above 95% for structured output skills in our builds.
How do I test for context bleed specifically? Test the skill at three points in a realistic session: as the first message, at message 10, and at message 30. If the skill passes at message 1 and fails at messages 10 and 30, context bleed is the cause. The failure threshold tells you how much context accumulation the skill can handle before instructions are crowded out.
My skill fails more often on complex inputs than on simple ones. Is that non-determinism? No, that's instruction scope. The skill was designed for simple cases and the instructions don't generalize to complex ones. Test with systematically increasing input complexity: 3 simple, 3 medium, 3 complex. If it fails consistently at a specific complexity level, the instructions need to handle that complexity explicitly.
Should I add a "check your work" instruction to fix non-deterministic failures? A general "check your work" instruction adds no reliability. It's too vague. A specific verifier instruction works: "After generating the output, verify it meets each condition in this list: [specific conditions]. If any condition fails, fix the section that fails." The specificity of the check is what makes it effective.
How do I know when to give up debugging and commission a rebuild? If you've spent more than 3-4 hours debugging a skill and still can't isolate the root cause, the skill's architecture is the problem, not a specific instruction. Skills built without output contracts or evals are structurally difficult to debug because you have no defined standard to test against. A rebuild with evals-first development is faster than continued debugging.
Last updated: 2026-04-23