IMPORTANT: Developer documentation for the current development branch. This content is unreleased, may change without notice, and must not be treated as Buildish release documentation.

Incremental bookkeeping types

Pipeline-internal emitted bookkeeping contracts stored under data/_pipeline/.

Back to the reference overview.

Type index

AggregateDependencyEntryV1 — One coordinator-owned aggregate and the units that may change its payload.
AggregateDependencyMapV1 — Shared-output dependency map for the current first-wave coordinator outputs.
OutputOwnershipClaimV1 — One exact published file or directory root together with its logical owner.
OutputOwnershipMapV1 — Published ownership inventory used to prune retained stages safely.
PersistedUnitContributionsV1 — Stable per-unit page contribution manifests retained in the visible stage.
StagedPageContributionWire — Metadata emitted by one page-staging worker for later aggregation.
UnitContributionManifestWire — Worker-emitted contribution fragment consumed by the coordinator.

AggregateDependencyEntryV1

One coordinator-owned aggregate and the units that may change its payload.

category: emitted
ownership: pipeline-derived
file contract: (inner type)

Field	Type	Required	Description
`stageRelativePath`	StageRelativePath	yes	Coordinator-owned aggregate file inside the visible stage.
`dependentUnitIds`	tuple[str, …]	no	Unit identifiers whose output can invalidate or change this aggregate file.

Selected field examples

stageRelativePath: Example: "data/components.json"

AggregateDependencyMapV1

Shared-output dependency map for the current first-wave coordinator outputs.

category: emitted
ownership: pipeline-derived
file contract: data/_pipeline/aggregate-dependencies.json

Field	Type	Required	Description
`schemaVersion`	Literal[1]	no	Schema version for the aggregate-dependency map.
`entries`	tuple[AggregateDependencyEntryV1, …]	no	Coordinator-owned aggregate files together with the units that may change them.

Example: Aggregate dependency map that marks which units can invalidate one shared file.

 1{
 2  "schemaVersion": 1,
 3  "entries": [
 4    {
 5      "stageRelativePath": "data/components.json",
 6      "dependentUnitIds": [
 7        "component:spark:runtime",
 8        "component:spark:site"
 9      ]
10    }
11  ]
12}

OutputOwnershipClaimV1

One exact published file or directory root together with its logical owner.

category: emitted
ownership: pipeline-derived
file contract: (inner type)

Field	Type	Required	Description
`ownerId`	str	yes	Logical owner identifier for the output, such as a unit owner id or the coordinator.
`unitId`	str	no	Unit identifier when the owning output belongs to one concrete staging unit.
`pathKind`	Literal[‘directory’, ‘file’]	yes	Whether the owned stage-relative path refers to one exact file or one directory root.
`stageRelativePath`	StageRelativePath	yes	Owned path inside the visible stage.

Selected field examples

ownerId: Example: "coordinator"
unitId: Example: "component:spark:runtime"
stageRelativePath: Example: "content/spark"

OutputOwnershipMapV1

Published ownership inventory used to prune retained stages safely.

category: emitted
ownership: pipeline-derived
file contract: data/_pipeline/output-ownership.json

Field	Type	Required	Description
`schemaVersion`	Literal[1]	no	Schema version for the output-ownership map.
`claims`	tuple[OutputOwnershipClaimV1, …]	no	Ownership claims for every retained stage file or directory root.

Example: Ownership claims for one unit directory and one coordinator aggregate file.

 1{
 2  "schemaVersion": 1,
 3  "claims": [
 4    {
 5      "ownerId": "artifact:spark/runtime",
 6      "unitId": "component:spark:runtime",
 7      "pathKind": "directory",
 8      "stageRelativePath": "content/spark/4.0.0"
 9    },
10    {
11      "ownerId": "coordinator",
12      "pathKind": "file",
13      "stageRelativePath": "data/components.json"
14    }
15  ]
16}

PersistedUnitContributionsV1

Stable per-unit page contribution manifests retained in the visible stage.

category: emitted
ownership: pipeline-derived
file contract: data/_pipeline/unit-contributions.json

Field	Type	Required	Description
`schemaVersion`	Literal[1]	no	Schema version for the persisted unit-contribution manifest file.
`units`	tuple[UnitContributionManifestWire, …]	no	Worker contribution manifests retained for every staged unit that currently owns visible output.

Example: Retained per-unit contribution map with one staged page.

 1{
 2  "schemaVersion": 1,
 3  "units": [
 4    {
 5      "unitId": "component:spark:runtime",
 6      "pages": [
 7        {
 8          "stageRelativePath": "content/spark/4.0.0/docs/getting-started/index.md",
 9          "componentSlug": "spark",
10          "artifactKey": "runtime",
11          "section": "documentation",
12          "pageKind": "docsPage",
13          "publicPath": "/spark/4.0.0/docs/getting-started/",
14          "componentPath": "/spark/",
15          "originKey": "archive",
16          "versionKind": "released",
17          "version": "4.0.0",
18          "defaultLocale": false,
19          "title": "Getting Started",
20          "sourcePath": "components/runtime/docs/getting-started.md",
21          "source": {
22            "key": "runtime",
23            "path": "docs/getting-started.md",
24            "repository": "https://github.com/apache/spark",
25            "viewRef": "main",
26            "editRef": "main"
27          }
28        }
29      ]
30    }
31  ]
32}

StagedPageContributionWire

Metadata emitted by one page-staging worker for later aggregation.

Field	Type	Required	Description
`stageRelativePath`	str	yes	Stage-relative file path of the staged page contribution.
`componentSlug`	str	yes	Owning component slug for the staged page.
`artifactKey`	str	no	Owning artifact key when the page belongs to a specific artifact.
`section`	str	yes	Higher-level section label for the staged page.
`pageKind`	str	yes	Short page-kind label for the staged page.
`publicPath`	str	yes	Published public path for the staged page.
`publicUrl`	str	no	Canonical absolute URL for the staged page, if known.
`componentPath`	str	yes	Public root path for the owning component.
`componentUrl`	str	no	Absolute URL root for the owning component, if known.
`originKey`	str	yes	Origin key selected for the staged page.
`versionContext`	dict[str, object]	no	Serialized version-context payload associated with the staged page.
`versionKind`	str	no	Version-record kind associated with the staged page, if present.
`versionRef`	str	no	Source-control ref associated with the staged page, if present.
`version`	str	no	Exact version string associated with the staged page, if present.
`locale`	str	no	Locale key for the staged page when localization is enabled.
`defaultLocale`	bool	no	Whether the staged page represents the default locale within its translation group.
`translationKey`	str	no	Shared key that ties translated sibling pages together.
`title`	str	no	Primary page title extracted during staging.
`linkTitle`	str	no	Shorter link title extracted during staging, if present.
`description`	str	no	Primary page description extracted or derived during staging, if present.
`derivedTitle`	str	no	Body-derived page title inferred from authored content during staging, if present.
`derivedDescription`	str	no	Body-derived page description inferred from authored content during staging, if present.
`sourcePath`	str	no	Source file path that produced the staged page. Private worker fragments may use an absolute path; the coordinator rewrites workspace sources to workspace-relative form and omits other sources before retaining this internal metadata in the stage.
`source`	PageSourceProvenance	no	Coordinator-derived repository provenance for the authored source file. Worker-supplied and retained values are overwritten during normalization.
`canonicalUrl`	str	no	Explicit canonical URL for the page when it should differ from `publicUrl`.

Selected field examples

stageRelativePath: Example: "content/spark/4.0.0/docs/getting-started/index.md"
componentSlug: Example: "spark"
artifactKey: Example: "runtime"
section: Example: "documentation"
pageKind: Example: "docsPage"
publicPath: Example: "/spark/4.0.0/docs/getting-started/"
componentPath: Example: "/spark/"
originKey: Example: "archive"
versionKind: Example: "release"
versionRef: Example: "refs/tags/v4.0.0"
version: Example: "4.0.0"
locale: Example: "en"
translationKey: Example: "spark-overview"
title: Example: "Getting Started"
linkTitle: Example: "Start"
derivedTitle: Example: "Getting Started"
sourcePath: Example: "components/runtime/docs/getting-started.md"

UnitContributionManifestWire

Worker-emitted contribution fragment consumed by the coordinator.

Field	Type	Required	Description
`unitId`	str	yes	Worker unit identifier that owns this contribution manifest.
`pages`	tuple[StagedPageContributionWire, …]	no	Staged page contributions emitted by the worker for later aggregation.

Selected field examples

unitId: Example: "component:spark:runtime"