skill-optimizer – Data & Science skill for AI agents

When to Use This Skill

• Use when skills are not triggering as expected or seem broken
• Use when you want to audit and improve your skill library's quality
• Use when you want to understand which skills are underperforming or wasting context tokens

Rules

• Read-only: never modify skill files. Only output report.
• All 8 dimensions: do not skip any. If data is insufficient, report "N/A — insufficient session data" rather than omitting.
• Quantify: "you had 12 research tasks last week but the skill never triggered" beats "you often do research".
• Suggest, don't prescribe: give specific wording suggestions for description improvements, but frame as suggestions.
• Show evidence: for undertrigger claims, quote the actual user message that should have triggered the skill.
• Evidence-based suggestions: when suggesting description rewrites, cite the specific research finding that motivates the change (e.g., "front-load trigger keywords — MCP study shows 3.6x selection rate improvement").

Overview

Analyze skills using historical session data + static quality checks, output a diagnostic report with P0/P1/P2 prioritized fixes. Scores each skill on a 5-point composite scale across 8 dimensions.

CSO (Claude/Agent Search Optimization) = writing skill descriptions so agents select the right skill at the right time. This skill checks for CSO violations.

Usage

• /optimize-skill → scan all skills
• /optimize-skill my-skill → single skill
• /optimize-skill skill-a skill-b → multiple specified skills

Data Sources

Auto-detect the current agent platform and scan the corresponding paths:

| Source | Claude Code | Codex | Shared |

|--------|------------|-------|--------|

| Session transcripts | ~/.claude/projects/**/*.jsonl | ~/.codex/sessions/**/*.jsonl | — |

| Skill files | ~/.claude/skills/*/SKILL.md | ~/.codex/skills/*/SKILL.md | ~/.agents/skills/*/SKILL.md |

Platform detection: Check which directories exist. Use only user-authorized platforms, date ranges and transcripts. Directory existence does not authorize reading unrelated chat histories. Redact private messages and secrets from reports; prefer minimal excerpts and local analysis.

Workflow

Identify target skills
        ↓
Collect session data (python3 scripts scan JSONL transcripts)
        ↓
Run 8 analysis dimensions
        ↓
Compute composite scores
        ↓
Output report with P0/P1/P2

Step 1: Identify Target Skills

Scan skill directories in order: ~/.claude/skills/, ~/.codex/skills/, ~/.agents/skills/. Identify copies by resolved path and content hash; the same name in multiple locations may contain different versions and must not be merged silently. For each, read SKILL.md and extract:

• name, description (from YAML frontmatter)
• trigger keywords (from description field)
• defined workflow steps (Step 1/2/3... or ### sections under Workflow)
• word count

If user specified skill names, filter to only those.

Step 2: Collect Session Data

Use python3 scripts via Bash to scan session JSONL files. Extract:

Claude Code sessions (~/.claude/projects/**/*.jsonl):

• Skill tool_use calls (which skills were invoked)
• User messages (full text)
• Assistant messages after skill invocation (for workflow tracking)
• User messages after skill invocation (for reaction analysis)

Codex sessions (~/.codex/sessions/**/*.jsonl):

• session_meta events → extract base_instructions for skill loading evidence
• response_item events → assistant outputs (workflow tracking)
• event_msg events → tool execution and skill-related events
• User messages from turn_context events (for reaction analysis)

Note: Codex injects skills via context rather than explicit Skill tool calls. Skill loading (present in base_instructions) does NOT equal active invocation. To detect actual use, search for skill-specific workflow markers (step headers, output formats) in response_item content within that session. Workflow markers are indirect evidence of possible use, not proof of invocation; report confirmed calls separately from inferred matches and record uncertainty. Transcript schemas differ by client version; inspect actual event shapes before extracting messages.

Aggregated:

• Per-skill: invocation count, trigger keyword match count
• Per-skill: user reaction sentiment after invocation
• Per-skill: workflow step completion markers

Step 3: Run 8 Analysis Dimensions

You MUST run ALL 8 dimensions. The baseline behavior without this skill is to skip dimensions 4.2, 4.3, 4.5b, and 4.8. These are the most valuable dimensions — do not skip them.

4.1 Trigger Rate

Count how many times each skill was actually invoked vs how many times its trigger keywords appeared in user messages.

Claude Code: count Skill tool_use calls in transcripts.

Codex: count sessions where the agent produced output following the skill's workflow markers (not merely loaded in context).

Diagnose:

• Never triggered → skill may be useless or trigger words wrong
• Keywords match >> actual invocations → undertrigger problem, description needs work
• High frequency → core skill, worth optimizing

4.2 Post-Invocation User Reaction

This dimension is critical and easy to skip. Do not skip it.

After a skill is invoked in a session, read the user's next 3 messages. Classify:

• Negative: "no", "wrong", "never mind", "not what I wanted", user interrupts
• Correction: user re-describes their intent, manually overrides skill output
• Positive: "good", "ok", "continue", "nice", user follows the workflow
• Silent switch: user changes topic entirely (likely false positive trigger)

Report observed reaction categories with sample counts; silence, interruption and topic changes do not prove dissatisfaction or false triggering.

4.3 Workflow Completion Rate

This dimension is critical and easy to skip. Do not skip it.

For each skill invocation found in session data:

1. Extract the skill's defined steps from SKILL.md

2. Search the assistant messages in that session for step markers (Step N, specific output formats defined in the skill)

3. Calculate: how far did execution get?

Report: {skill-name} (N steps): avg completed Step X/N (Y%)

If a specific step is frequently where execution stops, flag it.

4.4 Static Quality Analysis

Check each SKILL.md against these 14 rules:

| Check | Pass Criteria |

|-------|--------------|

| Frontmatter format | Validate against the target host/repository schema; additional supported metadata is allowed |

| Name format | Letters, numbers, hyphens only |

| Description trigger | Starts with "Use when..." or has explicit trigger conditions |

| Description workflow leak | Description does NOT summarize the skill's workflow steps (CSO violation) |

| Description pushiness | Description actively claims scenarios where it should be used, not just passive |

| Overview section | Present |

| Rules section | Present |

| MUST/NEVER density | Count ALL-CAPS directive words; >5 per 100 words = flag |

| Word count | < 500 words (flag if over) |

| Narrative anti-pattern | No "In session X, we found..." storytelling |

| YAML quoting safety | description containing : must be wrapped in double quotes |

| Critical info position | Core trigger conditions and primary actions must be in the first 20% of SKILL.md |

| Description 250-char check | Primary trigger keywords must appear within the first 250 characters of description |

| Trigger condition count | ≤ 2 trigger conditions in description is ideal |

4.5a False Positive Rate (Overtrigger)

Skill was invoked but user immediately rejected or ignored it.

4.5b Undertrigger Detection

This is the highest-value dimension. For each skill, extract its capability keywords (not just trigger keywords — what the skill CAN do). Then scan user messages for tasks that match those capabilities but where the skill was NOT invoked.

Report: which user messages SHOULD have triggered the skill but didn't, and suggest description improvements.

Compounding Risk Assessment:

For skills with chronic undertriggering (0 triggers across 5+ sessions where relevant tasks appeared), flag as "compounding risk" — undertriggered skills cannot self-improve through usage feedback, causing the gap to widen over time. Recommend immediate description rewrite as P0.

4.6 Cross-Skill Conflicts

Compare all skill pairs:

• Trigger keyword overlap (same keywords in two descriptions)
• Workflow overlap (two skills teach similar processes)
• Contradictory guidance

4.7 Environment Consistency

For each skill, extract referenced:

• File paths → check if they exist (test -e)
• CLI tools → check if installed (which)
• Directories → check if they exist

Flag any broken references.

4.8 Token Economics

This dimension is critical and easy to skip. Do not skip it.

For each skill:

• Word count (from St

… (full content is in the download package)