IMPORTANT: Developer documentation for the current development branch. This content is unreleased, may change without notice, and must not be treated as Buildish release documentation.

Test Suite Layout

Maintainer guide to core, adapter, command, workflow, and harness verification.

Test Source Tree

The suite combines deterministic unit tests, local Git/GPG/SVN integration tests, command tests, schema and documentation checks, and GitHub workflow simulation.

Release tests

  • tests/release/core/ covers provider-neutral config dependencies, naming, versioning, state, manifests, and vote-profile rules.
  • tests/release/platforms/github/ covers GitHub checks, refs, Releases, direct publication, and candidate publication.
  • tests/release/foundations/asf/ covers ASF dist behavior and URL enforcement.
  • tests/release/publications/pypi/ covers PyPI JSON API polling, exact distribution matching, pooled-session reuse, PEP 740 evidence composition, and final-manifest revalidation.
  • tests/release/signing/ covers isolated OpenPGP import and signing.
  • tests/release/commands/ covers bounded CLI operations, lifecycle resolution, exact candidate promotion, publication, artifact registration, voting materials, and verification.
  • tests/release/workflows/ checks lifecycle workflow policy and handoff structure.

The remaining modules directly under tests/release/ cover shared adapters and the deeper RC, reproducibility, report, bundle, schema, and generated-documentation contracts.

Harness tests

tests/harness/ covers scenario loading, job selection, runtime behavior, command tracing, and act-based workflow execution. Checked-in scenarios under buildish-release-tooling/harness/ exercise direct release, candidate publication, exact promotion, and read-only candidate verification.

make release-integration-check executes the production candidate and promotion workflows in sequence through act. It builds and signs the real wheel and sdist, retains candidate assets, promotes their exact bytes, uses a workspace-local package-index fixture, and binds deterministic local provenance evidence into the final manifest. This service-isolated evidence tests workflow composition; the live TestPyPI run remains the Trusted Publisher and PEP 740 service integration gate. The runner may still need network access for actions, tools, and container images.

Harness tests replace external writes with controlled local state. They do not verify repository settings, GitHub Environment reviewers, service availability, or real credentials.

Tests directly under tests/ cover release legal reports, legal-file generation and distribution, archive safety, downloads, parsing, I/O, and wheel contents.

Running checks

Use the narrowest relevant test while iterating, then run the repository gate:

1make check

Changes to published site content or routes also require make site-check-local from the Buildish site repository. If a check needs unavailable external tooling, report that limitation rather than treating an unexecuted check as passing.