Mongodb Setup Guide

Prerequisites

• A MongoDB replica set

• (Optional) A dedicated read-only MongoDB user. Creating a dedicated read-only user is recommended for better permission control and auditing. If you choose to skip this step, you can use an existing user in your database.

• We automatically select a suitable incremental sync mode based on your MongoDB version. Please note the following requirements:

  Incremental Mode	Requirements
  Change Stream	MongoDB version 5.1+, Permissions (optional*): read@local
  Standard (Oplog)	Permissions: read@local

* Read access to your deployment's oplog will enable us to perform additional analyses for your integration, such as verifying that the size of your oplog meets the recommended minimum. Highly recommended.

Setup Guide

1. Enter the Username and Password for your MongoDB user.

2. Enter the Hostname for your MongoDB. Find your host identifiers using either MongoDB Atlas or the MongoDB shell.

3. (Optional) Enter the Port for your MongoDB. If left empty, Matia will use the SRV connection format to connect to your database.

4. Enter the Database for which you need to replicate data.

5. If SSL/TLS is enabled on your database, set the SSL/TLS toggle to ON.

6. Select your preferred Schema Mode.

   There are two options for how your data will be structured in your destination. Consider the following MongoDB document:

   {
       "_id": 1,
       "foo": "2",
       "nested": {
         "bar": 3
       }
   }

   In Dynamic (Unpacked) mode, the top-level properties of the document are converted into separate columns:

   _ID	FOO	NESTED
   1	"2"	{"bar":3}

   Static (Packed) mode retains the entire document in a single _data column:

   _ID	_DATA
   1	{"_id":1, "foo":"2", nested":{"bar":3}}

7. Select your preferred conversion type for BINARY fields. Please note that invalid UUIDs will be converted to Base64.

8. Choose how many cursors we are allowed to open at the same time against a single collection. More cursors will decrease Full Refresh sync times but increase the load on your database.

9. Enter a Name for the connector.

10. (Optional) Enter a Description for the connector.

11. Select the Owner of the connector.

12. (Optional) Verify that your MongoDB is successfully connected by clicking on Test Connection.

13. Click Connect.

Notes

• On Full Refresh, we automatically save a suitable Change Stream cursor.
• Oplog and Change Stream cursors may expire. This can occur due to the connector's sync frequency or the size limit of the oplog. When a cursor expires, we automatically perform a full re-sync of the affected stream.
• Your collections' full schemas are discovered during the replication process. Schema updates are contingent upon all known properties being set to sync: ON.
• By default, we unpack the topmost layer of document properties.
• MongoDB's built-in _id field serves as the primary key for all replicated tables.
• We do not sync the following tables:
  • Views, including timeseries collections
  • System collections (database.system.\*)

Schema

• As we process your data, we transform MongoDB's BSON data types into formats supported by Matia. The following table outlines how we perform this conversion:

MongoDB Type	Matia Type	Snowflake Type
ARRAY	JSON	VARIANT
BINARY	STRING	STRING
BOOLEAN	BOOLEAN	BOOLEAN
DATE	INSTANT	TIMESTAMP_TZ
DECIMAL128	BIGDECIMAL	FLOAT
DOUBLE	DOUBLE	FLOAT
INT32	INT	NUMBER
INT64	LONG	NUMBER
JAVASCRIPT_CODE	STRING	STRING
JAVASCRIPT_CODE_WITH_SCOPE	JSON	VARIANT
MAX_KEY	JSON	VARIANT
MIN_KEY	JSON	VARIANT
OBJECT	JSON	VARIANT
OBJECT_ID	STRING	STRING
REGEX	STRING	STRING
STRING	STRING	STRING
SYMBOL	STRING	STRING
TIMESTAMP	INSTANT	TIMESTAMP_TZ

Supported Streams

This connector outputs a dynamic list of streams, derived from the collections in your database.

Supported Sync Modes

• Full Refresh
• Incremental (Change Stream)
• Append-Only Incremental (Change Stream)