Skip to main content

annotation-gate.yml

Fail a run that carries an annotation nobody allowed.

A check that emits a ::warning:: or ::notice:: on a green job tells the reader something is wrong without stopping the merge, and agents and authors skip it. This gate reads the annotations of every other job in the run through the check-runs API and fails when any notice, warning or failure annotation does not match an allow pattern.

Use

Add one job that lists every other job in needs and runs if: always(). See examples/annotation-gate.yml. Grant the job actions: read, checks: read and contents: read.

Inputs

  • levels: levels to enforce, default notice,warning,failure.
  • allow-patterns: newline-separated case-insensitive regexes for tracked exceptions.
  • job-name: optional extra name prefix of a job to skip. The gate’s own job is always skipped, whatever the caller named it.

The composite action Morrison-Lab/gha/annotation-gate@v3 also takes default-allow, which replaces the allowed-for-everyone list below, and token. The reusable workflow keeps the defaults. It takes three more inputs for the workflow_run mode described at the end: run-id (the run to gate instead of the current one), run-attempt, and head-sha (post the result as a check run on this commit).

Allowed for everyone

  • GitHub’s runner-image migration notice (“The ubuntu-latest label will migrate to …”).
  • A job’s own “Process completed with exit code N” failure annotation. A failed job already fails the run, and the gate runs under if: always(), so this one would only repeat it.
  • The forced-Node deprecation warning, until the action pins are bumped. It reads “Node.js N is deprecated” and then “The following actions target Node”.
  • The two annotations the review workflow emits when a newer push supersedes a review: the concurrency group’s “Canceling since a higher priority waiting request for …” and the stale-review skip “PR head moved …; not posting a stale review”. Both describe a run that was meant to stop, and the newer push gets its own review.
  • The failure annotation “The run was canceled by @user.”, which GitHub puts on a run someone cancelled by hand, from the UI or with gh run cancel (gha#1144). Like the concurrency cancellation, it describes a run that was meant to stop.
  • The notice the review workflow emits on a PR that edits workflow files: “PR edits workflow files; restored .github/workflows/ from the default branch …”. It reports the safe behavior of gha#598, so it is not a defect in the PR.
  • The failure annotation require-clean-verdict emits when a review finishes with findings: “Claude review verdict is not clean (verdict: …)”. require-clean-verdict already fails its own job, and in workflow_run mode each gated run posts a distinct annotation-gate check run that a later re-review on the same commit cannot clear.
  • The preview’s “Skipping element-level diff highlighting” notice. It says a page has more elements than the highlighter compares (500), so the preview shows that page without change highlights. A pull request that touches a large chapter always emits it, and no edit to the pull request removes it.

Scope

Annotations from the gate’s own job, and from a job that has not finished, are not seen. A reusable-workflow job is covered, because the jobs endpoint lists each by its full name.

One gate for a repository with many workflow files

The job above sees only the jobs of its own run. A repository whose checks live in separate workflow files calls annotation-gate-run.yml from one workflow_run workflow instead. It gates each run that finishes and posts the result as a check run named annotation-gate on that run’s commit, so it shows on the pull request.

name: Annotation gate
on: # zizmor: ignore[dangerous-triggers] -- the gate checks out nothing and runs no code from the triggering run
  workflow_run:
    workflows: ["*"]
    types: [completed]
jobs:
  gate:
    # The gate's own run must not trigger another gate.
    if: ${{ github.event.workflow_run.name != 'Annotation gate' }}
    permissions:
      actions: read
      checks: write
      contents: read
    uses: Morrison-Lab/gha/.github/workflows/annotation-gate-run.yml@v3
    with:
      run-id: ${{ github.event.workflow_run.id }}
      run-attempt: ${{ github.event.workflow_run.run_attempt }}
      head-sha: ${{ github.event.workflow_run.head_sha }}
      workflow-id: ${{ github.event.workflow_run.workflow_id }}-${{ github.event.workflow_run.event }}

It takes the same levels and allow-patterns inputs. It also takes a runs-on input (default: the caller’s CI_RUNNER variable, else ubuntu-latest), for a private repository that runs it on a self-hosted runner. Grant checks: write; the job posts one check run per gated run.

workflow-id is optional. It is any string that names the workflow and the event that triggered it, so a push run and a pull_request run of one workflow on the same commit keep separate gates; the example joins workflow_id and event. When set, a gate still queued or running for an earlier run of the same workflow and event on the same commit (a re-run or a later attempt) is cancelled when a newer one starts. A job-level concurrency group does this, named annotation-gate-run-<head-sha>-<workflow-id>, so a burst of finishing runs cannot flood a small self-hosted pool. The gate posts its check run in one call at the end, so a cancelled gate leaves none behind. When empty, the group falls back to the run id, which is unique, and each run keeps its own gate. Do not name a caller-level concurrency group with the prefix annotation-gate-run-: a job that shares its caller’s group is cancelled as a deadlock.

all-runs: true replaces the per-run gate with one gate per commit. The job lists the commit’s workflow runs, takes the newest completed run of each workflow and event, leaves out the caller’s own workflow, gates all of them and posts a single annotation-gate check run. A run still in progress is left out; its completion starts the next gate, which re-checks the whole commit. Pair it with a workflow-level concurrency group per commit (annotation-gate-<head-sha>, cancel-in-progress: false), so a burst of finishing runs queues one gate that does the work, not one per workflow. Leave cancel-in-progress false: a newer completion then replaces only the gate waiting behind the running one, while true cancels the running gate too, and completions arriving every few seconds on a busy commit never let one finish (gha#1649). With workflows: ["*"], a gate run’s own completion starts one more run of the caller, whose job the if: skips. Give that run a group of its own, keyed on its run id, as the example stub does: in the commit’s group it would replace the real gate waiting behind the running one, and that gate would never run (gha#1403). One commit-wide check-run listing gives each job’s annotation count, so a job without annotations costs no request. The commit then carries one annotation-gate check, not one per workflow.

defer-while-running: true (with all-runs, default false) makes a gate exit without gating or posting a check run while another workflow’s run on the commit is still queued or in progress. That run’s completion starts its own gate, so the last to finish gates the commit and earlier gates do no API work. Turn it on only when the caller triggers on workflow_run for every workflow, as the example stub does with workflows: ["*"]: with a narrower workflows: list, a deferring gate can wait on a run that never starts a gate, and the commit goes ungated. A quick failure is reported only when the slowest workflow on the commit finishes.

backfill: true repairs a commit that never got its check. A deferring gate posts nothing, and the last finishing run’s workflow_run event is the only thing that starts the gate that posts. When that event is lost, the commit carries no annotation-gate check run, the required check is missing, and GitHub blocks the pull request with nothing to re-run (gha#1754). Or it keeps an earlier failure that a re-run of the failing jobs has since fixed (gha#1752). With backfill, the job gates each commit that a pull request run touched in the last 24 hours and that has no annotation-gate check run, or whose newest one finished before a run on the commit did, and leaves a commit with a run still queued or in progress to that run’s own gate. Call the workflow from a schedule trigger with backfill: true, beside the workflow_run caller. It reads runs and check runs only, so it needs no permission beyond the ones the workflow already holds. head-sha is required by the workflow but unused in this mode, so pass any value.

repair-workflows (with all-runs; the workflow defaults it to Claude Code Review, the action to empty) repairs a commit whose gate predates a re-run that never started a gate. A finished re-run of the Claude review after its 429 sent no workflow_run event, so the commit kept the failure its first attempt posted until the next scheduled backfill, hours later (gha#2077). After a gate that did not defer posts its own result, it lists the recent runs of the named workflows and gates again each commit where a re-run of one (run_attempt above 1) finished in the last repair-hours (default 3) and the newest annotation-gate check is older. A first attempt is skipped, since its own completion starts a gate; checking every recent commit used up a busy repository’s API quota. A first attempt whose event is lost, or whose gate fails, is left to the scheduled backfill. It never fails the gate that called it.

To gate a commit again without a new commit and without re-running any other workflow, add a workflow_dispatch trigger with a head-sha input to the caller, as the example stub does, and pass github.event.workflow_run.head_sha || inputs.head-sha as head-sha. Dispatch it with the commit to re-check (gh workflow run annotation-gate.yml -f head-sha=<sha>). This dispatch works for a pull request that touches only files no path-filtered workflow watches, whose commit has no run to re-run. run-id may then be empty, because all-runs does not use it.

Without all-runs, each gated run posts its own check run named annotation-gate. When several workflows finish on one commit, the commit carries several checks of that name, and a failing one blocks a required check. annotation-gate-run.yml holds checks: write, so it is not in the read-only list on the permissions page. It calls the action at @v3, so it gates the right run only once v3 has slid past the commit that added these inputs; before that the unknown inputs are ignored and the gate checks its own run. See examples/annotation-gate-run.yml for the stub to copy.