Pipeline stages

Overview

DISCOVER -> PARSE -> EXPAND -> INTROSPECT -> PLAN -> EXECUTE

1. Discover

Glob for YAML files in conventional subdirectories (tables/, enums/, functions/, views/, roles/, mixins/) and SQL files in pre/, post/.

Each file gets a SHA-256 hash for change detection.

2. Parse

Read each YAML file into typed TypeScript objects. Validate required fields and apply defaults.

The parser auto-detects file type by content:

• table: field -> TableSchema
• name: + values: -> EnumSchema
• name: + body: -> FunctionSchema
• name: + query: -> ViewSchema or MaterializedViewSchema
• role: -> RoleSchema
• extensions: -> ExtensionsSchema
• mixin: -> MixinSchema

3. Expand

Load mixin definitions and merge their contributions (columns, indexes, triggers, policies, grants, checks) into consuming table schemas. Substitute {table} placeholders with table names.

4. Introspect

Query pg_catalog and information_schema filtered to the target pgSchema to read current database state:

• Tables and columns (types, defaults, nullability, generated expressions)
• Indexes (columns, uniqueness, method, WHERE, INCLUDE, opclass)
• Constraints (checks, foreign keys, unique)
• Enums (values and order)
• Functions (body, args, return type, attributes)
• Views and materialized views (query)
• Roles (attributes, memberships)
• Grants, triggers, policies, comments

5. Plan

Before diffing, declared SQL expressions are normalized to the form Postgres stores so they don’t read as spurious changes: policy USING/WITH CHECK and table CHECK expressions, partial-index WHERE clauses, column defaults, and view bodies are round-tripped through the database, and function return/argument types are resolved to their canonical names (timestamptz → timestamp with time zone).

Diff desired (YAML) state vs actual (DB) state. Produce an ordered list of typed Operation objects, each with:

• type: the operation type (e.g., create_table, add_column)
• objectName: what it operates on
• sql: the SQL to execute
• phase: execution phase (determines order)
• concurrent: whether to run outside transaction

Destructive operations are separated into a blocked list unless allowDestructive is true.

6. Execute

Run operations in phased order, committing one transaction per table so a migration never holds every table’s lock at once — each group is guarded by lock_timeout and retried on contention. See execution phases.

Pre-scripts can mutate the database in ways the plan can’t express (e.g. a column rename), so after they run the plan is recomputed against the new state before the main apply. After an apply that performed a wide CASCADE drop, the plan is recomputed again to recreate declared objects the cascade removed and to report anything still pending — so a single run converges. See post-apply convergence.