SkillWireChoose exactly one skill in Search, inspect its source, then download its ZIP. Keep that ZIP for this trial.
Popular skills
Search resultsInstall counts come from skills.sh; stars count the source repository.
| Stack | Popularity | Name | Problem solved | Category | Repository | Rating | Eval |
|---|
No matches. Try another search or filter.
Your stackSaved in this browser. Public sources only.
skillwire-skill-stack.zipZIP size checked when fetched
Up to ten public folders. The ZIP includes original bytes, file hashes and matched warning lines. SkillWire does not run downloaded files.Install into a fresh project
- Unzip the download into a temporary folder. Read
SKILL-SOURCES.json, every warning line, the pinned repository licence and any skill-specific terms. - For Codex, create
.agents/skills/inside your project. Copy each reviewed skill folder there, keeping itsSKILL.mdand supporting files together. Use the skill name as its folder name. Do not replace an existing skill without comparing it. - Start a fresh Codex session in that project and confirm the skill appears. Ask for it by name. Discovery does not prove the full instructions were read or that the skill improves results.
Downloaded scripts may run with your agent’s permissions if you later use them. Start in a disposable project with no private files or credentials. Other harnesses use different install locations; follow their official instructions.
Source warningsSee what each matched line means
Run your own paired trialOne task · same start · two outcomes
Keep this skill for your task?
Use one pinned skill and your own disposable project. SkillWire freezes its exact files, your task, grader, CLI settings and arm order before the two local runs. The same starting files go to both arms. Review the artifacts blind before you reveal which arm had the skill.
Installed skills on your computer
Run python3 lime_agent_eval.py doctor --project . > doctor.json from the downloaded trial kit, then open that file here. The CLI reports skill names and SKILL.md hashes without a model call. This browser cannot scan your files; the JSON stays in this tab.
Choose one skill in Search for this task.
Use your task, starting files, a known good result, a deliberately bad result and an independent grader. The local preflight must pass good and reject bad.
Download local trial kitUse the same downloaded ZIP with --stack skillwire-skill-stack.zip. Seal the task, grader, model, arm order and same starting files; run the two arms on your subscription.
Open the saved A/B outputs without arm names, grades or commands. Save your choice, including tie or cannot judge, before revealing the conditions.
Open blind A/B review →Reveal investigator evidence after your choice
Save a verdict in the blind reader. Then choose its blind-review.json, your saved human-review-verdict.json, and the withheld reveal-key.json. The key must match the blind bundle and the run receipts. This is a local consistency check, not proof of who judged.
Blind verdict and reveal key required before labelled evidence.
Inspect the run after reveal
Choose the complete run folder for raw CLI trace evidence, or select its six JSON receipts: plan.json, protocol.json, protocol-freeze.json, SKILL-SOURCES.json, artifact-results.json, and model-runs/records.json. Missing traces stay unverified. Files stay in this browser.
No run loaded.
A valid seal and completed CLI trace document what ran; they do not prove the model used the skill internally. One pair cannot establish general benefit. Missing arms or grader errors remain visible and are never scored as task failures. A historical run for another skill is labelled as a mismatch with your current selection.
Earlier studies, limits and detailed methods
Historical setup
Use your existing Codex or Claude Code subscription and a selected installed skill or public stack. Freeze the task and independent outcome checks, then compare the same starting files with and without those instructions. No extra API key or hosted evaluation service.
Set up a local evaluation Download CLI tools and example
Local detection does not start calls. Explicit trials use your chosen provider and subscription allowance. Unknown/API-key authentication is refused. Local trial setup freezes settings and disables unrelated customizations to keep the comparison controlled.
Judge the outputs yourself
Automated checks can miss usefulness, clarity and mistakes that matter to you. Make a blind A/B choice from the saved files before seeing which condition produced them. Ties and “cannot judge” are valid outcomes.
First native TDD family: grader contract under review
Ten matched pairs completed with a pinned public TDD skill folder. Recorded task passes were 8 of 10 with it installed and 7 of 10 without. The skill body was seen in one treatment tool trace; nine reads were unobserved, and internal use is unknown. A frozen reference accepted ASCII DEL where the task said to reject it, so the first family is disputed. A new ten-case contract is sealed for review with no model runs. These counts are not a skill benefit claim. Blind human choices have not been collected.
Earlier Skill Issue benchmark
The August SkillsBench entry reported that the skill body opened on only 3 of 8 panel tasks. Its treated versus baseline solve rates showed no detectable lift, and the reported effect was within its A/A noise. Raw rollouts were later deleted, so those figures cannot be rechecked from the repository. The current SkillWire pilot is a separate native Codex task with an 8/8 tie. Do not combine their counts or infer a general result.
View your local comparison
Read in this browser only; no upload, publication or catalog score. Imported reports are not independently authenticated.
Method, corrections and original task evidence
SourcesWhere the skills and results come from
More agent add-onsOfficial repositories and documentation