The npm Worm Targets Your Test Pipeline: Hardening Playwright CI After Shai-Hulud

Introduction

Test pipelines are an easy target. They install hundreds of packages on every pull request, often with fresh resolution. They hold tokens that can publish packages, deploy, or post to pull requests. And nobody reviews what npm ci executes, because it's "just the install step."

In 2026, the Shai-Hulud family of npm worms went after exactly that:

April 22, 2026: a malicious @bitwarden/[email protected] was on npm for about 1.5 hours. According to Endor Labs, its preinstall script launched a payload that checked for more than 30 CI environments and only ran in CI. It harvested GitHub Actions secrets and cloud credentials, and republished other packages with stolen npm tokens.
August 4, 2026: keyv and more than 400 related packages were compromised. Cycode reports that versions such as [email protected], [email protected], and [email protected] carried a preinstall hook. The same campaign also committed .claude/settings.json and .vscode/tasks.json files that run a script when a developer opens the project in Claude Code or VS Code, without any install at all.

This article isn't a news recap. We tested the defenses a QA or platform engineer can add to a Playwright pipeline this week, and one of them turned out to do nothing on the npm version most CI runners use.

How a dependency reaches your CI runner

Two paths matter for test pipelines.

Patch releases inside your ranges. Cycode notes that the poisoned versions were patch bumps that matched ranges already in package.json, and that they had valid, GitHub-signed provenance. A caret range like ^13.0.18 accepts 13.0.20 the next time someone runs npm install or a bot updates the lockfile. Provenance tells you where a package was built. It doesn't tell you the code is safe.

The npm registry still shows when [email protected] was published, which matches the timeline in the reports:

npm view keyv time --json

"6.0.0-rc.1": "2026-08-03T19:21:08.158Z",
"6.0.0": "2026-08-04T09:35:00.763Z"

Lifecycle scripts during install. preinstall, install, and postinstall scripts run with the permissions of the job, including every environment variable the step can see.

Test 1: what an install script can read

We built a tiny package whose preinstall script writes a file containing the NPM_TOKEN environment variable, packed it, and installed it into an app with a lockfile:

{
  "name": "evil-cache",
  "version": "1.0.0",
  "scripts": {
    "preinstall": "node -e \"require('fs').writeFileSync(require('path').join(process.env.INIT_CWD, 'PREINSTALL_RAN.txt'), 'token=' + (process.env.NPM_TOKEN || 'none'))\""
  }
}

With a fake token in the environment:

Command	Script ran?	File contents
`npm ci`	Yes	`token=npm_FAKE_TOKEN_FOR_DEMO`
`npm ci --ignore-scripts`	No	no file

That's the whole attack surface in two lines. A real payload would send the token somewhere instead of writing it to disk.

Can your project live without install scripts?

Often, yes. Search your lockfile for packages that declare one:

grep -B8 '"hasInstallScript": true' package-lock.json | grep '"node_modules/'

On this site's own lockfile, which includes Next.js, Playwright, ESLint, and Tailwind CSS, exactly one package declares an install script: unrs-resolver, a development dependency of the ESLint setup. Playwright doesn't need one. Browsers are installed by the separate, explicit npx playwright install step.

Keep in mind what --ignore-scripts doesn't cover. The npm documentation says scripts you run on purpose, such as npm test or npm run build, still run. So does any malicious code your tests import. The flag removes the one step where code runs before anything you wrote.

Test 2: npm's release cooldown, and the version that ignores it

Most malicious versions are found and removed within hours. A cooldown refuses versions younger than a set number of days. npm added min-release-age (in days) in npm 11.10.0.

We asked two npm versions to install [email protected], which was published on August 3, 2026, with a 45-day cooldown on September 14:

npm version	Command	Result
11.19.1	`npm install [email protected] --min-release-age=45`	Refused: `ETARGET No matching version found for [email protected] with a date before 7/31/2026`
11.19.1	same, `--min-release-age=7`	Installed
10.8.2	`npm install [email protected] --min-release-age=45`	Installed, no warning

npm 10 doesn't know the option and ignores it silently. That matters because the npm bundled with Node.js 20 and 22 is npm 10, which is what actions/setup-node gives you by default. A .npmrc with min-release-age=7 looks like protection and does nothing unless npm 11.10 or newer runs the install.

Also notice where a cooldown applies: when versions are resolved. npm ci installs exactly what the lockfile says. The places that need a cooldown are the ones that change the lockfile: developer machines and dependency bots.

For developers, commit this to .npmrc and require npm 11.10 or newer:

min-release-age=7

For Dependabot, the equivalent is the cooldown option:

version: 2
updates:
  - package-ecosystem: npm
    directory: /
    schedule:
      interval: weekly
    cooldown:
      default-days: 7

Test 3: a pull request guard for files that run code

The keyv campaign's IDE and agent files are the part most teams have no check for. A .vscode/tasks.json with "runOn": "folderOpen" runs when the folder opens in VS Code. A Claude Code hook in .claude/settings.json runs when an agent session starts. Neither needs npm install, and both arrived in commits with titles like "chore: update config".

Many of these files are legitimate: shared editor settings, agent permissions, CI workflows. The goal is to make a change to them a deliberate, reviewed event, not to ban them. This script fails a pull request that adds or changes a file that can run code automatically, or adds hidden characters to an agent instruction file:

// agent-config-guard.mjs: fails a pull request that adds or changes files that make
// editors, AI coding agents, or CI run code, unless a reviewer has approved that on purpose.
// Usage: node agent-config-guard.mjs <base-ref>   (for example origin/main)
import { execFileSync } from "node:child_process";
import fs from "node:fs";
 
const base = process.argv[2] ?? "origin/main";
const allowed = process.env.ALLOW_AGENT_CONFIG_CHANGES === "true";
 
// Files that execute something when a folder is opened, an agent session starts, or CI runs.
const autoRun = [
  /^\.claude\/settings(\.local)?\.json$/,
  /^\.claude\/.*\.(m?js|cjs|sh|ps1)$/,
  /^\.vscode\/tasks\.json$/,
  /^\.vscode\/settings\.json$/,
  /^\.cursor\//,
  /^\.github\/workflows\//,
  /^\.npmrc$/,
];
// Files an agent reads as instructions. Changes are fine; hidden characters are not.
const instructions = [/(^|\/)CLAUDE\.md$/, /(^|\/)AGENTS\.md$/, /(^|\/)\.cursorrules$/, /(^|\/)GEMINI\.md$/];
const hidden = /[\u200B-\u200F\u202A-\u202E\u2060-\u2064\uFEFF]/;
 
const changed = execFileSync("git", ["diff", "--name-only", "--diff-filter=ACMR", `${base}...HEAD`], { encoding: "utf8" })
  .split("\n")
  .filter(Boolean);
 
const problems = [];
for (const file of changed) {
  if (autoRun.some((re) => re.test(file)) && !allowed) {
    problems.push(`${file}: can run code automatically; needs explicit reviewer approval`);
  }
  if (instructions.some((re) => re.test(file)) && hidden.test(fs.readFileSync(file, "utf8"))) {
    problems.push(`${file}: contains zero-width or bidirectional control characters`);
  }
  if (file.endsWith("package.json")) {
    const scripts = JSON.parse(fs.readFileSync(file, "utf8")).scripts ?? {};
    for (const hook of ["preinstall", "install", "postinstall", "prepare"]) {
      if (scripts[hook]) problems.push(`${file}: defines a "${hook}" script; confirm it is intended`);
    }
  }
}
 
for (const problem of problems) console.log(`::error title=Agent config guard::${problem}`);
console.log(`${changed.length} changed file(s), ${problems.length} problem(s)`);
process.exit(problems.length > 0 ? 1 : 0);

We ran it in a scratch repository against one branch per scenario:

docs-only change                 1 changed file(s), 0 problem(s)                               exit 0
agent hook added                 ::error title=Agent config guard::.claude/settings.json: can run code automatically; needs explicit reviewer approval
                                 ::error title=Agent config guard::.claude/setup.mjs: can run code automatically; needs explicit reviewer approval
                                 2 changed file(s), 2 problem(s)                               exit 1
agent hook added, approved       2 changed file(s), 0 problem(s)                               exit 0
zero-width character in CLAUDE.md ::error title=Agent config guard::CLAUDE.md: contains zero-width or bidirectional control characters
                                 1 changed file(s), 1 problem(s)                               exit 1
postinstall added                ::error title=Agent config guard::package.json: defines a "postinstall" script; confirm it is intended
                                 1 changed file(s), 1 problem(s)                               exit 1

Two limits to be honest about. The guard checks the pull request, so it can't protect a developer who already opened a compromised clone. And the postinstall check flags any lifecycle script in a changed package.json, not only new ones, so expect the occasional false alarm on projects that use prepare.

A mistake we made while writing it: our first version of the hidden-character pattern contained the invisible characters themselves instead of \u escapes. It worked, but it was unreviewable, and the guard would have flagged its own source if it had lived in an instruction file. Write such patterns with escapes.

A hardened Playwright workflow

Putting the tested pieces together:

name: e2e
 
on: pull_request
 
permissions:
  contents: read
 
jobs:
  guard:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
          persist-credentials: false
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      # Set this repository variable to "true" only for a reviewed change to agent or IDE config.
      - run: node scripts/agent-config-guard.mjs origin/${{ github.base_ref }}
        env:
          ALLOW_AGENT_CONFIG_CHANGES: ${{ vars.ALLOW_AGENT_CONFIG_CHANGES }}
 
  tests:
    needs: guard
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          persist-credentials: false
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      # No lifecycle scripts, and no secrets in this step's environment.
      - run: npm ci --ignore-scripts
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test

What each line is for:

permissions: contents: read limits what the workflow's GITHUB_TOKEN can do if something does run.
persist-credentials: false stops actions/checkout from leaving that token in .git/config, where any script in the job could read it.
npm ci --ignore-scripts removes install-time execution. If a dependency genuinely needs its script, run it explicitly with npm rebuild <package> after reviewing it.
No secrets.* at job level. Pass a secret only to the single step that needs it, never to the install step.
The guard runs first, so a pull request that adds a folder-open task never reaches a job with secrets.

A repository variable is a blunt approval switch because it applies to every pull request while it's set. For a team, a required review from code owners on these paths is the better long-term control. A CODEOWNERS entry for .claude/, .vscode/, and .github/workflows/ does that without any script.

If you installed an affected version

The reports agree on the order, and it's worth repeating because it differs from a normal incident:

Remove both persistence files (.claude/settings.json and .vscode/tasks.json) before anything else. Cycode found they point at each other's payloads, so deleting one leaves the other active.
Look for the token watcher before rotating tokens. Cycode reports a script at ~/.local/bin/gh-token-monitor.sh that triggers the payload when a token is revoked.
Rotate every credential the machine or runner could read: npm and GitHub tokens, cloud keys, SSH keys.
Treat CI runners that installed the version as compromised. GitHub-hosted runners are discarded after each job, but any secret the job could read is not.

Checklist

Search your lockfile for "hasInstallScript": true and run npm ci --ignore-scripts if you can.
Check npm --version wherever min-release-age is supposed to apply. On npm 10 it's ignored without a warning.
Add cooldown to Dependabot, and min-release-age to .npmrc for developers on npm 11.10 or newer.
Set permissions, persist-credentials: false, and step-level secrets in every test workflow.
Require review for .claude/, .vscode/, .cursor/, .github/workflows/, and .npmrc, with CODEOWNERS or the guard script.
Scan agent instruction files for zero-width characters.

GitHub Actions	CI/CD	Free plan
Playwright	UI Automation	Open source