Stop Putting Best Practices in Skills

Summary of Multi-Turn Evaluation Results

The multi-turn evaluation suite, designed to simulate real-world development scenarios across various stages (TDD, debugging, planning, refactoring), yielded the following results:

Configurations:

1. Superpowers: SessionStart hook + skills.
2. Plain Skills: Same skills installed without any hooks or hints.
3. CLAUDE.md: Equivalent guidelines written as static rules, always in context.
4. CLAUDE.md + Hint: One-liner hint in CLAUDE.md saying "invoke the relevant skill before coding" + skills installed.

Scenarios:

1. TDD – Email Validator
2. Debugging – Broken LRU Cache
3. Planning – Rate Limiter
4. Refactoring – Express Middleware
5. Mixed – HTTP Client Retry

Key Findings:

• Superpowers: The model consistently invoked skills across all scenarios, leading to a high pass rate (70-90%).
• Plain Skills: The model rarely or never opened the skill drawer on its own, resulting in very low success rates.
• CLAUDE.md: With static rules always in context, the model performed significantly

Read the full article at DEV Community

---

Want to create content about this topic? Use Nemati AI tools to generate articles, social posts, and more.