我的源数据在MongoDB中被 struct 化为事件,我们正在Snowflake中构建一个数据仓库.每种类型的事件都与DW中的操作有关,例如行插入或更新等.
如何将MongoDB中的数据连续加载到Snowflake中?
我的源数据在MongoDB中被 struct 化为事件,我们正在Snowflake中构建一个数据仓库.每种类型的事件都与DW中的操作有关,例如行插入或更新等.
如何将MongoDB中的数据连续加载到Snowflake中?
你(至少)有两个 Select :
Use an integration tool such as dbt:
https://docs.snowflake.com/en/user-guide/ecosystem-etl.html
Create your own bespoke pipleline with SnowPipe via a storage integration. This will involve creating a process that exports the events from MongoDB to cloud storage. Obviously, this process needs to keep track of which events have already been processed.
https://docs.snowflake.com/en/user-guide/data-load-snowpipe-intro.html
https://docs.snowflake.com/en/sql-reference/sql/create-storage-integration.html