* Cache scan verdicts and drop policy-failing entries from bump PRs
Three changes that together let the nightly bump clear any backlog in a
single run without blocking on a single bad upstream or re-burning Claude
time on already-scanned SHAs:
- bump-plugin-shas.yml: raise max-bumps default 20 -> 130 (above the
external entry count, so a single run can clear a full backlog) and add
an explicit 60-min job timeout. The cap was the only thing bounding the
blast radius of a single policy failure; the changes below take over
that role so the cap can be lifted.
- scan-plugins.yml: add a verdict cache keyed on (plugin, sha, policy
hash). The bump action force-resets bump/plugin-shas every night, which
makes the same SHAs reappear in the diff on consecutive nights — without
the cache the scan would re-burn ~90s of Claude time per entry per
night. Cached verdicts (pass and fail) are served from disk; only
uncached SHAs are scanned. The job still fails on cached failures so
the required check stays honest.
- revert-failed-bumps.yml (new): after a Scan Plugins workflow_run on
bump/plugin-shas concludes with a failure, drop just the failing
entries' source.sha back to main's pin via a follow-up signed commit
and re-dispatch the scan. The re-dispatch finds only cached-pass
entries and goes green in seconds. Bounded at 3 passes/night, restricted
to SHA-only diffs, and aborts if the bump branch was tampered with.
* Harden bump cache and revert workflows after review
- revert-failed-bumps: replace the time-based revert budget (anchored on
the PR head, which a revert commit immediately replaces — never
accumulating past 1) with a commit count: every nightly bump force-
resets to one commit and every revert pass adds exactly one, so
commits > MAX+1 is the budget without date math, pagination, or
exposure to comment spoofing.
- revert-failed-bumps: filter the bump PR by head owner so a fork PR
with a branch named bump/plugin-shas can't be selected.
- revert-failed-bumps: continue-on-error on the artifact download so a
scan that died before uploading (infra error) doesn't fail the revert
job — the missing-file guard downstream handles it.
- scan-plugins: add a per-ref concurrency group so concurrent scans
don't lose one another's cache writes; key the cache on run_attempt
so a re-run can save its own verdicts.
- scan-plugins: store the full source object in the cache and require
source equality on lookup, so a repo/path change at the same SHA
misses the cache instead of getting a stale verdict.
- scan-plugins / revert-failed-bumps: strip markdown control chars,
wrap model-generated text in code spans (neutralizes auto-linked
URLs), and redact key-shaped tokens before they reach the step
summary, artifact, cache, or PR comment.
Scan Plugins is meant to gate every change to marketplace.json, but two
gaps made that unenforceable:
1. The bump workflow opens PRs with GITHUB_TOKEN, which GitHub exempts
from on:pull_request triggers. Weekly bump PRs (e.g. #1809) get no
scan check at all.
2. The workflow had a paths filter, so a required-check ruleset for
`scan` would block every PR that doesn't touch marketplace.json
(no check run = pending forever).
Fixes:
scan-plugins.yml
- Drop the paths filter; replace with a step-level `git diff --quiet`
early-exit on the same paths. The check now reports on every PR,
which makes it safe to require.
- Fail closed when ANTHROPIC_API_KEY is unset and a scan is needed.
The shared action no-ops gracefully in that case (right default for
community repos), but a required check that silently does nothing is
a rubber stamp.
bump-plugin-shas.yml
- After the action opens the bump PR, `gh workflow run scan-plugins.yml
--ref bump/plugin-shas`. workflow_dispatch is exempt from the
GITHUB_TOKEN recursion guard, and the resulting check run lands on
the branch HEAD (= PR head), so it satisfies the required check.
- Add `actions: write` so the dispatch is allowed.
Follow-up: add a repo ruleset on main requiring the `scan` check
(integration: github-actions) once this merges.
* Tighten policy scan: hook scope, telemetry, disclosure; make blocking
policy/prompt.md — adds Part 2 (hook scope and disclosure):
- Enumerate every registered hook and read its source.
- Flag has_broad_scope_hooks when UserPromptSubmit/PreToolUse/
PostToolUse runs without a project-relevance gate, or any hook
reads user data beyond the plugin's stated scope — regardless of
whether it makes network calls.
- Flag has_undisclosed_telemetry when any hook or shipped code calls
a non-MCP host without explicit disclosure + opt-out.
- Flag description_matches_behavior=false when the install
description would not lead a reasonable user to expect the
hooks/telemetry/data-access found.
- passes=false when any of the above trip. Violations must cite the
specific hook/file and what the user wasn't told.
The bar is now "handles user data responsibly," not merely "isn't
malicious." A non-malicious plugin that observes more than its stated
purpose justifies will fail.
policy/schema.json — adds required hooks[], has_broad_scope_hooks,
has_undisclosed_telemetry, description_matches_behavior.
scan-plugins.yml:
- fail-on-findings: true (blocking — loosen later if FP rate too high)
- workflow_dispatch with scan_all input for full re-review of all
external entries
- timeout-minutes: 360 (full scan of 117 entries at ~96s each ≈ 3h)
- trigger on .github/policy/** so prompt edits get scanned
* Bump vercel SHA to test the tightened scan against it
* Adopt validate-plugins action suite; pin all external SHAs
Replaces the hand-rolled marketplace validator and bot-based bump
workflow with the shared composite actions (pinned at f846a0b).
marketplace.json:
- 62 external entries that were missing a `sha` are now pinned to
their current upstream HEAD (resolved via git ls-remote).
Workflows:
- validate-plugins.yml: invariants I1-I11 + claude plugin validate +
diff-gated clone-at-SHA validation of changed external entries.
SHA-pin (I5) is a hard error. I8/I11 stay warnings until the 15
known data issues (vendored dirs without manifests; one dotted
name) are cleaned up.
- bump-plugin-shas.yml: bot-free weekly refresh. Validates each new
SHA with claude plugin validate before opening one PR; works with
the default GITHUB_TOKEN (contents:write + pull-requests:write).
- scan-plugins.yml: Claude policy scan of changed external entries.
Non-blocking; graceful no-op if ANTHROPIC_API_KEY isn't set.
Removed:
- validate-marketplace.yml + the two TS helper scripts (superseded
by step 11/20 of validate-plugins).
validate-frontmatter.yml is kept — it's complementary (targeted
checks on agent/skill/command files for in-repo plugins).
* Remove 5 external entries that fail validation at HEAD
Step 30 (clone at pinned SHA + claude plugin validate) fails for
these at their current HEAD:
aiven Unrecognized key "logo" in plugin.json
atlassian-forge-skills skill YAML frontmatter parse error
sagemaker-ai skill YAML frontmatter parse error
speakai no plugin manifest at repo root
stagehand no plugin manifest at repo root
These can be re-added once the upstream repos are fixed.
* Wire scan-plugins to the detailed policy prompt
Adds .github/policy/prompt.md and schema.json (the full security
review rubric — malicious code, privacy, deception, safety
circumvention, exfiltration; plus network-call and software-install
flags) and points scan-plugins at it via the policy-prompt input.
With ANTHROPIC_API_KEY now configured on the repo, scan-plugins runs
the actual policy review on changed external entries instead of
no-op'ing.
* Bump scan-plugins action pin to include L11/L12 fixes
