CockpitCockpit
← Back to blog

From chat to skill: turn yesterday’s 28-minute debug into today’s /command

Published April 30, 2026 · 6 min read

Every productive Claude Code session ends with dark knowledge that dies with the conversation. Cockpit’s Skills feature lets the agent crystallize a chat into a SKILL.md — saved to *your* knowledge base (not a Cockpit-owned folder), then registered as a slash command. Your notes stay where they live; Cockpit just holds the pointer.

Yesterday I spent 28 minutes walking Claude through our OAuth refresh-token flow. Token endpoint, leeway window, two custom claims, the one staging-only quirk. Bug found, fixed, shipped.

This morning, almost the same problem in a sister service. I open a new chat. The agent has no memory of yesterday. 28 minutes again.

This isn't an Anthropic limitation. Stateless agents are the right default — you don't want yesterday's wrong assumption haunting tomorrow's session. The fix isn't "give the AI memory". The fix is give yourself memory, in a place the agent will read again.

That place is a Skill.

What a Skill actually is — and where it should live

A Skill is one Markdown file. Nothing more. Cockpit only cares about two things: the file's content (which becomes the system prompt for /skill-name) and an absolute path that points at it.

The path is the design choice that matters most. Anthropic's reference convention puts skills under ~/.claude/skills/. Cockpit deliberately does not do that. We don't scan your home directory. We don't reserve a folder name. Your skills can live anywhere you want them to live:

  • ~/Notes/Skills/oauth-debug/SKILL.md — alongside your personal notes
  • ~/Documents/team-playbooks/oauth-debug/SKILL.md — in a synced folder
  • ~/Work/our-handbook/skills/oauth-debug/SKILL.md — inside a git'd team repo
  • ~/Obsidian/Vault/Skills/oauth-debug/SKILL.md — inside your Obsidian vault
  • ./.claude/skills/oauth-debug/SKILL.md — Anthropic-style, if you prefer

Each skill is its own folder. The Markdown file is named SKILL.md (Anthropic convention) and the folder name becomes the slash command. Why a folder per skill? Because skills frequently grow companion files — example transcripts, a reference cheat-sheet, a small Python helper the agent calls — and a folder gives them somewhere natural to live.

Why this matters: your knowledge already lives somewhere. You have a vault, a notes app, a docs repo, a "Skills" folder you've curated for years. Forcing skills into ~/.claude/skills/ would be Cockpit picking a fight with your existing system. We register pointers instead.

The two-step flow

Step 1. Write the SKILL.md in your knowledge base. Whatever path you want.

Step 2. In Cockpit's Skills sidebar, click + Add Skill and paste the absolute path. Cockpit stores it in ~/.cockpit/skills.json (one file, one list of pointers, easy to back up).

That's it. The skill is now in your slash autocomplete. You can edit the source file directly in your editor of choice — Cockpit watches it, picks up changes on save, no re-import.

What Cockpit then gives you on top:

  • Skills sidebar: lists every registered skill with name, description, icon, source path, last-used time
  • Slash autocomplete: / in chat shows your skills mixed with built-ins like /qa, /fx, /review
  • Validity checks: red "Invalid" badge if the source file disappears (you renamed it, moved the vault) — fix the path, skill comes back
  • Preview: click any skill to render the markdown full-page, or view raw source

The crystallize loop

The trick isn't writing skills by hand. The trick is letting the conversation that just succeeded write the skill for you.

Three messages at the end of every productive session:

Message 1 — distill

Summarize what we just figured out about the OAuth refresh flow. Be concrete: file paths, the actual root cause, what we'd check next time.

The agent now has the conversation in its working set. It knows what mattered.

Message 2 — encode

Now write that as a SKILL.md I can drop into my knowledge base at ~/Notes/Skills/oauth-debug/SKILL.md (or wherever you keep yours).

Shape: when invoked, it should take a problem statement, walk through the same diagnostic steps we used today, in the same order, and refuse to suggest a fix until step 4. Keep it under 40 lines.

You'll get something like:

---
name: oauth-debug
description: Diagnose OAuth refresh-token failures using our debugging playbook.
---

You diagnose OAuth refresh-token failures in our auth stack.

When the user describes a failure, follow these steps in order:

1. Confirm which environment (dev / staging / prod). Staging has a
   60-second leeway window; prod does not.
2. Read `auth/refresh.ts` and identify which JWT claims are validated.
   We require both `sub` and `tid` — never just `sub`.
3. Ask for the failing access token (or a redacted version). Decode
   the header and check `alg` — must be RS256, never HS256.
4. Check `auth/keys.ts` for the JWKS cache TTL. We've shipped 3 bugs
   in 2 years from stale JWKS.
5. Only after the above, propose a fix.

Never edit code until step 5. If the user asks earlier, restate the
remaining diagnostic steps and ask them to choose.

Message 3 — save

Save it to that path.

If you've given the agent write access to your knowledge base, it'll create the file itself. Otherwise copy-paste from the chat into your editor — same outcome. The file now lives in your notes, indexed by your existing search, version-controlled by your existing git, synced by your existing Dropbox / iCloud / Syncthing. It is not Cockpit's data.

Then register it. In Cockpit's Skills sidebar, click + Add Skill, paste the absolute path:

/Users/you/Notes/Skills/oauth-debug/SKILL.md

(One-time, takes 5 seconds.) From here on, /oauth-debug shows up in the slash autocomplete in every chat. Edit the source file from anywhere — your editor, another machine, a teammate's PR — and Cockpit picks it up on next file-system event.

Tomorrow morning

/oauth-debug Token refresh failing intermittently in staging only.

The agent enters the same posture you trained yesterday. Same checks, same order, same refusal to jump to fixes. Not because it remembers — because you wrote yesterday down, somewhere it will read again.

The 28 minutes from yesterday is now a 30-second invocation.

Three skill shapes that earn the file

Not every conversation deserves a skill. The ones that do tend to fall into three shapes:

Diagnostic skills — /oauth-debug, /db-deadlock, /cors-issue, /flaky-test. Freeze a debugging procedure. The agent gets a checklist instead of guessing.

Convention skills — /our-pr-style, /our-test-style, /our-error-handling. Freeze your team's tribal knowledge. New contributor on day one types /our-pr-style and the agent writes PRs that pass review without 4 rounds of nitpicks.

Onboarding skills — /our-stack, /our-deploy-flow, /where-does-X-live. Explain your codebase to a fresh agent. This is the highest-leverage one — every new chat in your repo starts with the right map.

If a conversation doesn't fit one of these, it's probably a one-off. Don't crystallize it.

Skills as team assets

Because skills are just markdown files at paths you choose, the team-asset story falls out for free. Pick a repo your team already trusts:

~/Work/our-handbook/skills/
├── README.md             # how to write a skill, how to register it
├── oauth-debug/
│   ├── SKILL.md
│   └── examples.md       # optional companion files the skill can reference
├── our-pr-style/
│   └── SKILL.md
└── our-deploy-flow/
    ├── SKILL.md
    └── runbook.md

Now skills are diffable, reviewable, git blame-able. The senior engineer's "always do X but never Y" stops being a Slack DM and becomes a PR with a reviewer. New hires git pull, click + Add Skill three times, and the team's tacit knowledge is in their slash menu before the end of day one.

Cockpit's LAN-shared review surface (see our previous post) makes the inner loop tighter: write a skill in chat, share the review page over LAN, teammate comments line-by-line, send their comments back to the agent as context, agent revises the skill, you commit.

The point: skills don't make a copy of your team's knowledge. They reference it. When the handbook updates, the skill updates. There's only one source of truth, and it's already where your team keeps source of truth.

Meta: a skill that writes skills

Once you have one skill, you can write a meta-skill:

/distill Read the last 50 messages in this conversation. If you
spot any pattern that repeated 3+ times, propose a SKILL.md for
it. Don't save automatically — show me the draft first.

Now your Skills sidebar fills itself, slowly, from the conversations you actually have. Cockpit ships /qa, /fx, /review, /commit as opinionated defaults — but the best skills in your sidebar a year from now will be ones you didn't write by hand.

The bigger principle

Stateless agents are correct. They reset between conversations because that's how you avoid yesterday's wrong assumption breaking tomorrow's session.

But your team isn't stateless. Your team learns. The question is whether that learning lives in three engineers' heads or in 12 reviewed Markdown files in ./.claude/skills/.

Skills are how you make that choice explicit.

npm i -g @surething/cockpit · GitHub · Try Online