Sync Modes
This page is the authoritative reference for ETL sync modes. Sync mode is configured per table in the integration Schema tab. Available modes depend on the source connector.
| Mode | What is synced | Cursor / key | Typical use case |
|---|---|---|---|
| Full refresh | All records every run | No | Initial load, full replace |
| Incremental | New/updated records | Yes (cursor field) | Ongoing change sync |
| Append only | New records only | Depends on connector | Events, logs |
Full Refresh
- Behavior: Every sync reads all records from the source table and writes them to the destination. Existing destination data for that table is replaced or overwritten according to connector behavior.
- When to use: When incremental or append-only is not supported, or when you need a complete replace (e.g. after fixing data or schema).
- Cursor: Not used.
Incremental
- Behavior: After an initial full load, each sync reads only records that are new or updated since the last successful sync. A cursor field (e.g.
updated_at,id) identifies the cutoff. Matia stores the last cursor value and queries the source for records with a cursor value greater than that. - When to use: For large tables that change over time; reduces volume and runtime.
- Cursor: Depending on the connector, the cursor field is either defined automatically for the source, or can be manually set by a user in the Schema tab for each table. Common choices: timestamp column for last update, or incrementing id. The source must support filtering on this column.
Note
If the source does not expose a usable cursor, Matia uses full refresh for that table. Connector documentation specifies which tables support incremental and which columns can be used as the cursor.
Append Only
- Behavior: Each sync reads only new records from the source (e.g. records appended since last run). Existing records in the destination are not updated or deleted. No cursor may be required if the connector uses another mechanism (e.g. log position).
- When to use: For event streams, logs, or other append-only data where updates and deletes are not expected.
Additional Information
For run-level details (status, volume, emitted/committed), see Sync logs and run history.
For configuration steps, see Configure incremental sync.