Agent skills — the small bundles of instruction text plus tool wiring that let a general-purpose model perform a specific job — have quietly become one of the most useful primitives in agentic AI. A few hundred lines of well-written prompts and a couple of HTTP calls can now do work that, eighteen months ago, needed a small team of engineers and a maintenance budget.
When I'm working with client teams on internal agent deployments, one of the conversations that comes up earliest is: what should we be building skills for ourselves, and what should we adopt off the shelf? The answer is almost always "some of both" — community skills for the obvious stuff, bespoke in-house skills for the workflows specific to your business, your data, your systems.
The bespoke skills are where most of the value sits. They're also where most of the risk sits, and the kind of risk people aren't necessarily looking for.
The artefact you didn't realise you were producing
When you (or the agent helping you) build a skill against an internal system, you accumulate two distinct things:
The runtime code — what the skill actually does at invocation time. A few HTTP calls, some auth handling, output formatting. Usually small.
The development knowledge — captured payloads, observed quirks of the internal API, the precise shape of how authentication is handled, the scraped JSON from a backend that wasn't really meant to be hit directly, the screenshot from when the form validation broke, the list of endpoints you mapped while figuring out which one returned what.
The runtime code has to live in the skill. The development knowledge does not. And in my experience — particularly when the skill is being authored by an AI coding assistant rather than written by hand — there's a strong tendency for the second category to bleed into the first. The agent, dutifully trying to "document thoroughly", will paste the captured API response into the README. It will leave a comment explaining that endpoint X returns the user's full record including the salary field. It will name a variable extract_employee_pii_block() because that, technically, is what it does.
Then the skill gets committed. It gets shared with another team. It gets pushed to a repo that, six months later, somebody makes public because they thought it was just utility code. Or — the case I worry about more — it gets used as context by another agent in a different session, and the captured payload is now in some LLM provider's logs.
A short list of disciplines
Nothing in this list is novel. None of it is hard. But none of it is the default behaviour of an unsupervised coding agent either, and that's the gap.
1. Two workspaces, always
Maintain a private development workspace for each skill, separate from the published (or shared) artefact. The development workspace holds the unfiltered output of building the thing: HAR files, scraped fixtures, debug notes, "here's what the endpoint actually returns" dumps, captured tokens (rotated afterwards, obviously). The shared workspace holds only what the skill needs at runtime to do its job.
The discipline isn't about secrecy from the agent that runs the skill — that agent has to do the work. It's about not handing the map of how your internal systems actually behave to anyone who happens to clone the repo.
2. Sanitise the documentation
Skills can't hide what they do at runtime; that's a losing game and it makes maintenance worse. The discipline applies to the documentation around the code. Specifically:
No tutorial-style READMEs walking a reader through which header the auth gateway checks, or what the rate limit shape looks like, or which combinations of inputs cause the validation to fail.
No comments narrating internal defensive behaviour.
// the gateway rejects requests without X-Internal-Trace within 100ms of session creationbelongs in development notes, not in the deployed skill.Generalised function names.
solve_sso_token_exchange_with_legacy_fallback()becomesauthenticate().Redacted fixtures. If a test needs a sample payload, scrub the bits that aren't load-bearing for the test.
The heuristic: would this paragraph or comment still make sense if the internal system were anonymised? If anonymising would gut the meaning, it's too internals-specific to be in the shared artefact.
3. Describe outcomes, not internals
User-facing documentation describes what the skill produces, what it needs as input, and what its limits are. It does not read like an architecture document for the system the skill talks to. The reader should learn how to use the skill without learning how the underlying system works.
4. Treat reconnaissance notes as sensitive
The notes you accumulated while figuring out how the internal API actually behaves — undocumented endpoints, error-message taxonomies, authentication corner cases, the spreadsheet of which fields appear under which conditions — are sensitive in the same way any internal architecture document is sensitive. Kept private. Not pasted into public issues. Not committed to a branch you'll later open-source. Not included in agent session transcripts that get shared or stored externally.
5. Default to the narrow capability
Where a design choice branches between helping a specific legitimate use case and a more general capability that also makes other things easier, prefer the narrower form unless the broader one is explicitly required. A skill that fetches a record for the authenticated calling user is a different artefact from a skill that fetches arbitrary records by ID. Both are reasonable; the second one is reasonable in fewer situations, and the second one is the one that needs an explicit justification.
6. Be willing not to ship
Some skills, on reflection, shouldn't be shared past their author or their immediate team. "We built it, therefore we publish it internally" is not a default worth preserving when the shared form materially increases the blast radius of a credential leak or a misuse incident. Authors reserve the right — and accept the responsibility — to keep some skills local.
A note on AI-authored skills specifically
If you're building skills with an AI coding assistant (and you probably are — that's much of the reason this is a viable workflow), the assistant needs to know about these constraints, or it won't apply them. Left to its own defaults, an LLM trained to be a thorough documenter will document thoroughly. It will helpfully include the captured payload. It will helpfully name the function after what it does, including the bit you'd rather it not.
The fix is in the skill's own development context: a short instruction set telling the agent what categories of artefact stay in development, what gets stripped before the skill is committed, and what a "shared-form" skill looks like in this codebase. It's not foolproof. It is materially better than nothing.
The Carrot Cake angle
A lot of the work I do at Carrot Cake AI is bespoke in-house skill and plugin development for teams who are starting to deploy agents against their own systems — internal APIs, data warehouses, support tooling, the operational surfaces that don't have an off-the-shelf integration. The disciplines above are the ones I bring to that work. They're not glamorous; they're not novel; they're the difference between an agent deployment that quietly does its job and one that, six months in, becomes the subject of a slightly awkward conversation with someone in security.
If you're building this kind of capability internally and want a second pair of hands — or just a second opinion — drop me a line.