Very-large sites

This page is for publication systems that span many repositories, multiple product families, and strong permalink, trust-boundary, and localization needs.

Who this is for

  • many repositories or doc sources
  • multiple product families under one staged contract
  • localization, mounted content, and generated refs may all be in play

Start from a working large-site model

A very-large site is not a different schema. It is a large-site catalog with more independent sources, product groups, locales, mounts, and deployment targets operating through the same staged contract. Start from the Large sites packet, then add one dimension at a time.

A representative source layout might include:

 1site/
 2  catalog.yaml
 3  provider-snapshot.json
 4components/
 5  spark/
 6    docs/
 7      en/
 8      de/
 9  spark-operator/
10    docs/
11      en/
12      de/
13generated/
14  api-reference/
15vendor/
16  shared-brand/

Localization remains authored policy. For example, shared defaults can require every localized route to carry its locale prefix:

1defaults:
2  localization:
3    supportedLocales: [en, de]
4    defaultLocale: en
5    fallbackLocale: en
6    routeMode: prefixAll

Translated pages use the same translationKey while keeping their own authored content:

1docs/en/guide.md  -> translationKey: guide
2docs/de/guide.md  -> translationKey: guide

The pipeline validates that the locale, route shape, and translation linkage agree. It emits the cross-page relationship in data/translations.json; the renderer decides how to present a language switcher.

Use a gated operating flow

Run planning and validation before publishing a new stage:

1site-pipeline plan --for build --report-format json --report-schema-version 1
2site-pipeline check --report-format json --report-schema-version 1
3site-pipeline build

Treat each step as a separate decision:

  1. Planning inventories the selected local inputs and reports whether they are present, missing, stale, or unresolved.
  2. Consumer-owned SCM or cache tooling materializes missing inputs.
  3. check validates paths, links, provider records, translations, references, and route ownership without publishing a stage.
  4. build publishes one completed staged contract for renderers and deployment adapters.

Inspect the cross-site outputs

At this scale, sample representative pages and inspect the aggregate inventory:

 1site/.stage/
 2  manifest.json
 3  content/
 4  data/
 5    components.json
 6    compatibility.json
 7    content-index.json
 8    diagnostics.json
 9    redirects.json
10    routes.json
11    translations.json
12  static/

Use manifest.json to discover which aggregate files are present. Do not make renderers or deployment adapters infer routes, versions, translations, or compatibility from checkout layout.

Failure and recovery expectations

  • Missing or stale inputs stay visible in the planning report; acquisition remains outside Site Pipeline.
  • Unknown compatibility targets, duplicate translation locales, inconsistent translation routes, and route collisions block staging.
  • A failed build must not replace the last completed stage.
  • Deployment adapters should reject route or redirect entries belonging to an origin they do not own.
  • Provider input can enrich lifecycle state but cannot redefine component identity or public routes.

Read these first

  1. Integrate provider, compatibility, and translation data
  2. Scale Site Pipeline operations
  3. Create HTTP server configuration

What you should treat as first-class concerns

  • route ownership and long-lived permalink stability
  • trust boundaries between authored inputs, provider data, and deployment config
  • translation linkage and mounted/generated content boundaries
  • operational controls for planning, caches, and large inventories

Growth transition

There is no next size band. Growth means making ownership and operating limits more explicit: split planning by deployment unit where appropriate, monitor inventory sizes and build time, keep source acquisition replaceable, and test each renderer and deployment adapter against the staged contract.

Deeper unreleased development reference

These contracts describe unreleased development behavior and have not yet been published as release documentation.