Matia Docusaurus — Architecture

Recent Changes

• DEV-49004 / sources-list meta refactor: ETL docs/connectors-meta/etl/* is generated from sources-list.json only (slug = toKebabCase(sourceName)). Docs-tree scan remains for overview.ts generation only.
• Explorer catalog overview loader: loadConnectorPageOverviewFromDir reads page overview.json and merges ETL meta from disk (no overview.ts import in Node). Throws when a docs leaf matches sources-list.json but the expected connectors-meta/etl/{metaKebabName}/overview.json is missing.
• ETL overview.ts scaffold-once: Phase 2 writes overview.ts only when the file is missing; existing modules are left untouched (static boilerplate for MDX).
• sources-list meta refactor (task 03): writeEtlConnectorPageOverviewArtifacts accepts metaKebabName (decoupled from docs folder slug); runner Phase 2 scans ETL docs leaves and writes only overview.ts with meta import path from metaKebabName.
• DEV-49004: Added per-stream supportsReimport to ConnectorStream, passthrough to generated schema-streams.json, and UI indicators in Available Streams (parenthetical) and Schema Table (badge).

Shared Layer (src/shared/, src/entities/)

Entities Layer (src/entities/connector/model/)

• ConnectorStream — optional supportsReimport?: boolean indicates the stream supports re-import in Matia product data.

export interface ConnectorStream {
  streamName: string;
  supportedSyncModes: string[];
  supportsReimport?: boolean;
  // ...
}

Features Layer (src/static-pages/)

Connectors — Available Streams (src/static-pages/connectors/available-streams/)

• AvailableStreamItem — appends (Supports Re-import) after sync-mode parenthetical when supportsReimport === true.

Connectors — Schema Table (src/static-pages/connectors/schema-table/)

• StreamCardHeader — renders Supports Re-import badge (same styling as sync-mode badges) for parent streams with supportsReimport === true.
• schema-streams-utils.ts — exports STREAM_REIMPORT_LABEL and formatAvailableStreamReimportSuffix.

Generate Scripts (src/app/generate-connectors-script/)

ETL connector meta generation (pnpm gen:connectors-meta)

• Phase 1 (sources-driven): For each entry in src/app/connectors-source/sources-list.json, write overview.json, features.json, and schema-streams.json (when streams non-empty) to docs/connectors-meta/etl/{toKebabCase(sourceName)}/.
• Phase 2 (docs-driven): Scan docs/connectors/ for ETL connector leaves; scaffold overview.ts only when missing (static merge module for MDX — not overwritten on later runs).
• No docs validation: Orphan meta dirs and missing docs pages are acceptable; build fails on bad MDX import paths.
• Helpers: getEtlMetaEntriesFromSourcesList, generateEtlConnectorsMeta in src/app/generate-connectors-script/runner/.

Explorer catalog overview lookup

• loadConnectorPageOverviewFromDir(connectorDir, leafName, etlMetaEntries) — merges page overview.json with ETL meta read from docs/connectors-meta/etl/{metaKebabName}/overview.json.
• Doc-only connectors (no sources-list entry): returns page overview.json only.
• Missing meta: throws with path hint and pnpm gen:connectors-meta remediation (no silent fallback).

Schema streams generation

• Input: src/app/connectors-source/sources-list.json stream objects (includes supportsReimport).
• Output: docs/connectors-meta/etl/{connector}/schema-streams.json via pnpm gen:connectors-meta.
• Passthrough: generateConnectorsMeta writes source.streams without stripping optional fields.