A skill that reads well on the page can still fire for the wrong things, miss the right ones, or produce silent garbage. Five minutes of real testing now saves an hour of mystery debugging later.
A skill is a small system with two ways to fail: it doesn't trigger when you want it to, or it triggers and then does something wrong. Testing is how you find both kinds of failure before a real user does.
This chapter is the loop: write a draft, prod it with realistic prompts, fix what's broken, repeat until 8 out of 10 reasonable invocations land. Then ship. The rest of the work is real-usage feedback, not imagined edge cases.
You finish a SKILL.md. The description reads well. The body is tidy. You're proud of it. None of that means it works.
A description can read perfectly to you and still never fire โ because the words you used aren't the words your users say. Or it can fire on prompts that have nothing to do with your skill โ because one trigger phrase is too broad. The only way to know is to try it, the same way a user would.
The good news: skill testing is cheap. You don't need a test framework. You need five minutes, a fresh session, and a willingness to be wrong about your own writing.
The simplest test there is: in a fresh session, type five variations of what a real user would say. Note which trigger your skill and which don't. That's it. That's the whole test.
Five variations means actually different phrasings, not five rewordings of the same sentence. For a transcript skill:
youtu.be/abc"If the skill fires on 4 or 5, you're probably fine. If it fires on 2, your description is missing the natural words. If it fires on all 5 plus three unrelated prompts you didn't even type โ you wrote it too broadly.
Keep a small tests.md file next to SKILL.md. List the prompts you ran and whether they worked. Six months later when you tweak the description and break something, the file tells you exactly what to retry. The whole file is maybe twenty lines. You'll never regret writing it.
Triggering is half the battle. The skill firing for the right prompt doesn't mean it did the right thing once it fired.
Output testing is running the skill end-to-end on a real input and comparing the actual result to what you expected:
The trap is reading the result and going "looks good" because the shape is right. A transcript-shaped blob isn't a transcript. A deploy-shaped success message isn't a deploy. Open the artifact. Verify the thing actually happened.
Edge cases are inputs the skill doesn't really handle. The point isn't to make every edge case work โ it's to find out how the skill fails. Loud failure is good; silent failure is dangerous.
Throw weird stuff at it. Empty inputs. Malformed inputs. Conflicting arguments. Inputs from an adjacent domain. Watch what happens.
If the skill is going to fail โ and it is โ you want it to fail visibly, with a clear "I can't do X because Y." Failing loudly is much better than failing quietly. Quiet failure is the kind of bug that takes weeks to find because nobody knows it happened.
For anything more than the simplest skill, you want a record of what it actually did. Have the skill write a short log line to a file or to stdout each time it runs: which inputs it saw, which branch of logic it took, what it produced. Audit the file periodically. Spot the patterns you didn't predict.
Without observability, the skill is opaque. You ask "did it work last Tuesday?" and the answer is a shrug. With a log file, the answer is one grep away.
It doesn't need to be elaborate. A single line per run:
# ~/.claude/skills/youtube-transcript/logs/runs.log 2026-05-28 14:02:13 url=youtu.be/abc format=plain result=ok lines=412 2026-05-28 14:09:01 url=youtu.be/xyz format=timestamped result=no-captions 2026-05-29 09:14:33 url=vimeo.com/123 format=plain result=wrong-host action=declined
When trigger testing turns up misses, the fix is usually in the description (the trigger field โ see Ch.2). Three moves cover most of it:
The A/B widget below makes the loop concrete: keep your current description in version A, draft a candidate fix in version B, run the same ten prompts against both, and see which actually scores better. Sometimes your "better" version is worse than what you had.
If you care about a skill staying healthy over time, keep a CHANGELOG.md next to SKILL.md. One line per change: "2026-05-12: added 'captions' to trigger phrases โ was missing this." When the skill breaks six months from now, the changelog is the time machine.
Iteration is a trap. There's always another phrasing to add, another edge case to guard, another sentence to tighten. The trick is knowing when "better" stops being worth it.
The rule of thumb that works: when 8 out of 10 reasonable invocations work, ship. Not 10/10 โ that's perfectionism. Not 5/10 โ that's broken. Eight is the number where the skill is genuinely useful and the remaining two failures are either rare or already legible enough that you'll spot them and adjust.
Perfect is the enemy of useful. Ship the skill at 8/10, watch how it gets used in the real world, then iterate on the actual failures โ not the ones you imagined sitting at your desk. Real usage finds bugs you wouldn't have invented.
The two pieces of glue that turn one-off testing into a repeatable habit. The first is a checklist. The second is a tiny loop. Neither is fancy. Both pay off the third time you touch the skill.
tests.md file next to SKILL.md is the cheapest QA you'll ever do.