Mainfast Rollcall

• never-ran, renamed and stopped-reporting: a required name that no PR reports, or that a similar non-required job replaced.
• skipped-as-success and neutral-as-success: GitHub counts skipped and neutral as passing, so the PR goes through.
• matrix-name-mismatch: the requirement is test, but the job reports test (20).
• path-filtered, branch-filtered and not-on-pull-request: the producing workflow does not run for the PR.
• merged-without, merged-with-failure and merged-while-pending: PRs that merged over a bypass or an unfinished check.
• protection-drift and ruleset-not-enforced: classic protection and rulesets disagree, or a ruleset is in evaluate or disabled mode.
• conditional-job, skips-when-needs-fail, ambiguous-name, app-mismatch and more. Each names the PRs that show it and carries its fix.

name: Mainfast Rollcall
on:
  schedule:
    - cron: "17 6 * * 1"   # weekly
  workflow_dispatch:
  branch_protection_rule:
  push:
    branches: [main]
    paths: [".github/workflows/**", ".mainfast.yml"]
permissions:
  contents: read
  pull-requests: read
  statuses: read
  checks: write
  issues: write
jobs:
  rollcall:
    runs-on: ubuntu-latest
    steps:
      - uses: mainfast/rollcall@v0

GITHUB_TOKEN=... npx mainfast-rollcall audit --github-repo owner/name --pulls 30 --open-pulls 10 --format md
npx mainfast-rollcall audit --snapshot snap.json --format json        # offline

Step by step, with what to check after each part: installation manual. Everything else: the README.

• contents: read: config, workflow files and classic required checks. Without it the run stops on the config read.
• pull-requests: read: merged and open PRs to sample. Without it the run stops.
• statuses: read: commit statuses on each sampled head. Without it statuses are treated as empty and noted.
• checks: write: read check runs and post the result. issues: write: the tracking issue. Without them a warning annotation is shown.
• No administration permission is needed. Rulesets come from the metadata-read endpoints; the admin endpoint is only a fallback, and a 403 becomes a note.
• A ruleset's bypass_actors are only returned with write access to the ruleset, so Rollcall does not read them.

• It makes read calls to the GitHub API with your GITHUB_TOKEN and writes a check run, a job summary and one issue. Nothing goes to a third party; with a licence key, only the key and the organisation id go to Polar to check it.
• The config is read through the API from the default branch and workflows from the audited branch, never from a checkout.
• Only an issue opened by the token's own identity is ever edited or closed; a marker pasted into someone else's issue is ignored.
• Check names, workflow paths, job names, if: expressions and ruleset names are escaped: no HTML, no links, no @ mentions.

• Ruleset changes trigger no workflow. After editing a ruleset, run the workflow by hand or wait for the weekly run.
• Check runs from the PR's test merge commit are not read; only the head SHA.
• Matrix names are matched by pattern, and expressions in a job name become wildcards.
• Reusable-workflow producers are matched as "caller / ..."; the called workflow is not opened.
• An open PR whose head has reported nothing yet is left out and mentioned in the notes.
• Up to 1000 open issues are scanned for the tracking issue.