This is an automated email from the ASF dual-hosted git repository. smiletan pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push: new b79ce6e3097 [dbt](docs)dbt docs modify for catalog and new version adapter (#2742) b79ce6e3097 is described below commit b79ce6e309722ecf785c46db4d975687b418e5ac Author: catpineapple <catpineapple1...@gmail.com> AuthorDate: Fri Aug 15 10:22:15 2025 +0800 [dbt](docs)dbt docs modify for catalog and new version adapter (#2742) ## Versions - [x] dev - [x] 3.0 - [x] 2.1 - [ ] 2.0 ## Languages - [x] Chinese - [x] English ## Docs Checklist - [ ] Checked by AI - [ ] Test Cases Built --- docs/ecosystem/dbt-doris-adapter.md | 47 ++++++++++++++++++++-- .../current/ecosystem/dbt-doris-adapter.md | 47 ++++++++++++++++++++-- .../version-2.1/ecosystem/dbt-doris-adapter.md | 47 ++++++++++++++++++++-- .../version-3.0/ecosystem/dbt-doris-adapter.md | 47 ++++++++++++++++++++-- .../version-2.1/ecosystem/dbt-doris-adapter.md | 47 ++++++++++++++++++++-- .../version-3.0/ecosystem/dbt-doris-adapter.md | 47 ++++++++++++++++++++-- 6 files changed, 258 insertions(+), 24 deletions(-) diff --git a/docs/ecosystem/dbt-doris-adapter.md b/docs/ecosystem/dbt-doris-adapter.md index af1688e39db..ff1712e30d1 100644 --- a/docs/ecosystem/dbt-doris-adapter.md +++ b/docs/ecosystem/dbt-doris-adapter.md @@ -8,15 +8,16 @@ # DBT Doris Adapter [DBT(Data Build Tool)](https://docs.getdbt.com/docs/introduction) is a component that focuses on doing T (Transform) in ELT (extraction, loading, transformation) - the "transformation data" link -The `dbt-doris` adapter is developed based on `dbt-core` 1.5.0 and relies on the `mysql-connector-python` driver to convert data to doris. +The `dbt-doris` adapter is developed based on `dbt-core` and relies on the `mysql-connector-python` driver to convert data to doris. git: https://github.com/apache/doris/tree/master/extension/dbt-doris ## version -| doris | python | dbt-core | -|---------|--------------|----------| -| >=1.2.5 | >=3.8,<=3.10 | >=1.5.0 | +| doris | python | dbt-core | dbt-doris | +|---------|-------------|----------|----------| +| >=1.2.5 | >=3.8,<=3.10| >=1.5.0 | <=0.3 | +| >=1.2.5 | >=3.9 | >=1.8.0 | >=0.4 | ## dbt-doris adapter Instructions @@ -463,3 +464,41 @@ select {% endif %} ``` + +### Customizing table data column types and precision sample reference + +The `schema.yaml` file configures `data_type` for `columns` in `models` as follows: + +```yaml +models: + - name: sell_user + description: "A dbt model named sell_user" + columns: + - name: user_id + data_type: BIGINT + - name: account_id + data_type: VARCHAR(12) + - name: status + - name: cost_sum + data_type: DECIMAL(38,9) + - name: update_time + data_type: DATETIME + - name: create_time + data_type: DATETIME +``` + +### Access catalog sample reference + +The [Data Catalog](../lakehouse/catalog-overview.md) is a reference to different data sources within the Doris data lake functionality, layered above the Database. +It is recommended to access it through the dbt-doris built-in Macros: `catalog_source` + +```sql +{{ config(materialized='table', replication_num=1) }} + +select * +-- use macros 'catalog_source' not macros 'source' +-- catalog name is 'mysql_catalog' +-- database name is 'dbt_source' +-- table name is 'sell_user' +from {{ catalog_source('mysql_catalog', 'dbt_source', 'sell_user') }} +``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/dbt-doris-adapter.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/dbt-doris-adapter.md index dfe42b2e7a3..45acbdc986f 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/dbt-doris-adapter.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/dbt-doris-adapter.md @@ -6,15 +6,16 @@ --- [DBT(Data Build Tool)](https://docs.getdbt.com/docs/introduction) 是专注于做 ELT(提取、加载、转换)中的 T(Transform)—— “转换数据”环节的组件 -`dbt-doris` adapter 是基于`dbt-core` 1.5.0 开发,依赖于`mysql-connector-python`驱动对 doris 进行数据转换。 +`dbt-doris` adapter 是基于`dbt-core` 开发,依赖于`mysql-connector-python`驱动对 doris 进行数据转换。 代码仓库:https://github.com/apache/doris/tree/master/extension/dbt-doris ## 版本支持 -| doris | python | dbt-core | -|---------|--------------|----------| -| >=1.2.5 | >=3.8,<=3.10 | >=1.5.0 | +| doris | python | dbt-core | dbt-doris | +|---------|-------------|----------|----------| +| >=1.2.5 | >=3.8,<=3.10| >=1.5.0 | <=0.3 | +| >=1.2.5 | >=3.9 | >=1.8.0 | >=0.4 | ## dbt-doris adapter 使用 @@ -479,3 +480,41 @@ select {% endif %} ``` + +### 自定义表数据列类型及精度样例参考 + +`schema.yaml` 文件对 `models` 中 `columns` 的 `data_type` 配置如下: + +```yaml +models: + - name: sell_user + description: "A dbt model named sell_user" + columns: + - name: user_id + data_type: BIGINT + - name: account_id + data_type: VARCHAR(12) + - name: status + - name: cost_sum + data_type: DECIMAL(38,9) + - name: update_time + data_type: DATETIME + - name: create_time + data_type: DATETIME +``` + +### 访问 catalog 样例参考 + +[Data Catalog](../lakehouse/catalog-overview.md) 是 Doris 数据湖功能中指向不同的数据源,其层级在 Database 之上。 +对其访问 推荐通过 dbt-doris 内置 Macros : `catalog_source` + +```sql +{{ config(materialized='table', replication_num=1) }} + +select * +-- use macros 'catalog_source' not macros 'source' +-- catalog name is 'mysql_catalog' +-- database name is 'dbt_source' +-- table name is 'sell_user' +from {{ catalog_source('mysql_catalog', 'dbt_source', 'sell_user') }} +``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/ecosystem/dbt-doris-adapter.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/ecosystem/dbt-doris-adapter.md index dfe42b2e7a3..45acbdc986f 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/ecosystem/dbt-doris-adapter.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/ecosystem/dbt-doris-adapter.md @@ -6,15 +6,16 @@ --- [DBT(Data Build Tool)](https://docs.getdbt.com/docs/introduction) 是专注于做 ELT(提取、加载、转换)中的 T(Transform)—— “转换数据”环节的组件 -`dbt-doris` adapter 是基于`dbt-core` 1.5.0 开发,依赖于`mysql-connector-python`驱动对 doris 进行数据转换。 +`dbt-doris` adapter 是基于`dbt-core` 开发,依赖于`mysql-connector-python`驱动对 doris 进行数据转换。 代码仓库:https://github.com/apache/doris/tree/master/extension/dbt-doris ## 版本支持 -| doris | python | dbt-core | -|---------|--------------|----------| -| >=1.2.5 | >=3.8,<=3.10 | >=1.5.0 | +| doris | python | dbt-core | dbt-doris | +|---------|-------------|----------|----------| +| >=1.2.5 | >=3.8,<=3.10| >=1.5.0 | <=0.3 | +| >=1.2.5 | >=3.9 | >=1.8.0 | >=0.4 | ## dbt-doris adapter 使用 @@ -479,3 +480,41 @@ select {% endif %} ``` + +### 自定义表数据列类型及精度样例参考 + +`schema.yaml` 文件对 `models` 中 `columns` 的 `data_type` 配置如下: + +```yaml +models: + - name: sell_user + description: "A dbt model named sell_user" + columns: + - name: user_id + data_type: BIGINT + - name: account_id + data_type: VARCHAR(12) + - name: status + - name: cost_sum + data_type: DECIMAL(38,9) + - name: update_time + data_type: DATETIME + - name: create_time + data_type: DATETIME +``` + +### 访问 catalog 样例参考 + +[Data Catalog](../lakehouse/catalog-overview.md) 是 Doris 数据湖功能中指向不同的数据源,其层级在 Database 之上。 +对其访问 推荐通过 dbt-doris 内置 Macros : `catalog_source` + +```sql +{{ config(materialized='table', replication_num=1) }} + +select * +-- use macros 'catalog_source' not macros 'source' +-- catalog name is 'mysql_catalog' +-- database name is 'dbt_source' +-- table name is 'sell_user' +from {{ catalog_source('mysql_catalog', 'dbt_source', 'sell_user') }} +``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/ecosystem/dbt-doris-adapter.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/ecosystem/dbt-doris-adapter.md index dfe42b2e7a3..45acbdc986f 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/ecosystem/dbt-doris-adapter.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.0/ecosystem/dbt-doris-adapter.md @@ -6,15 +6,16 @@ --- [DBT(Data Build Tool)](https://docs.getdbt.com/docs/introduction) 是专注于做 ELT(提取、加载、转换)中的 T(Transform)—— “转换数据”环节的组件 -`dbt-doris` adapter 是基于`dbt-core` 1.5.0 开发,依赖于`mysql-connector-python`驱动对 doris 进行数据转换。 +`dbt-doris` adapter 是基于`dbt-core` 开发,依赖于`mysql-connector-python`驱动对 doris 进行数据转换。 代码仓库:https://github.com/apache/doris/tree/master/extension/dbt-doris ## 版本支持 -| doris | python | dbt-core | -|---------|--------------|----------| -| >=1.2.5 | >=3.8,<=3.10 | >=1.5.0 | +| doris | python | dbt-core | dbt-doris | +|---------|-------------|----------|----------| +| >=1.2.5 | >=3.8,<=3.10| >=1.5.0 | <=0.3 | +| >=1.2.5 | >=3.9 | >=1.8.0 | >=0.4 | ## dbt-doris adapter 使用 @@ -479,3 +480,41 @@ select {% endif %} ``` + +### 自定义表数据列类型及精度样例参考 + +`schema.yaml` 文件对 `models` 中 `columns` 的 `data_type` 配置如下: + +```yaml +models: + - name: sell_user + description: "A dbt model named sell_user" + columns: + - name: user_id + data_type: BIGINT + - name: account_id + data_type: VARCHAR(12) + - name: status + - name: cost_sum + data_type: DECIMAL(38,9) + - name: update_time + data_type: DATETIME + - name: create_time + data_type: DATETIME +``` + +### 访问 catalog 样例参考 + +[Data Catalog](../lakehouse/catalog-overview.md) 是 Doris 数据湖功能中指向不同的数据源,其层级在 Database 之上。 +对其访问 推荐通过 dbt-doris 内置 Macros : `catalog_source` + +```sql +{{ config(materialized='table', replication_num=1) }} + +select * +-- use macros 'catalog_source' not macros 'source' +-- catalog name is 'mysql_catalog' +-- database name is 'dbt_source' +-- table name is 'sell_user' +from {{ catalog_source('mysql_catalog', 'dbt_source', 'sell_user') }} +``` diff --git a/versioned_docs/version-2.1/ecosystem/dbt-doris-adapter.md b/versioned_docs/version-2.1/ecosystem/dbt-doris-adapter.md index aa88f54af4c..c8ca5da0786 100644 --- a/versioned_docs/version-2.1/ecosystem/dbt-doris-adapter.md +++ b/versioned_docs/version-2.1/ecosystem/dbt-doris-adapter.md @@ -8,15 +8,16 @@ # DBT Doris Adapter [DBT(Data Build Tool)](https://docs.getdbt.com/docs/introduction) is a component that focuses on doing T (Transform) in ELT (extraction, loading, transformation) - the "transformation data" link -The `dbt-doris` adapter is developed based on `dbt-core` 1.5.0 and relies on the `mysql-connector-python` driver to convert data to doris. +The `dbt-doris` adapter is developed based on `dbt-core` and relies on the `mysql-connector-python` driver to convert data to doris. git: https://github.com/apache/doris/tree/master/extension/dbt-doris ## version -| doris | python | dbt-core | -|---------|--------------|----------| -| >=1.2.5 | >=3.8,<=3.10 | >=1.5.0 | +| doris | python | dbt-core | dbt-doris | +|---------|-------------|----------|----------| +| >=1.2.5 | >=3.8,<=3.10| >=1.5.0 | <=0.3 | +| >=1.2.5 | >=3.9 | >=1.8.0 | >=0.4 | ## dbt-doris adapter Instructions @@ -463,3 +464,41 @@ select {% endif %} ``` + +### Customizing table data column types and precision sample reference + +The `schema.yaml` file configures `data_type` for `columns` in `models` as follows: + +```yaml +models: + - name: sell_user + description: "A dbt model named sell_user" + columns: + - name: user_id + data_type: BIGINT + - name: account_id + data_type: VARCHAR(12) + - name: status + - name: cost_sum + data_type: DECIMAL(38,9) + - name: update_time + data_type: DATETIME + - name: create_time + data_type: DATETIME +``` + +### Access catalog sample reference + +The [Data Catalog](../lakehouse/catalog-overview.md) is a reference to different data sources within the Doris data lake functionality, layered above the Database. +It is recommended to access it through the dbt-doris built-in Macros: `catalog_source` + +```sql +{{ config(materialized='table', replication_num=1) }} + +select * +-- use macros 'catalog_source' not macros 'source' +-- catalog name is 'mysql_catalog' +-- database name is 'dbt_source' +-- table name is 'sell_user' +from {{ catalog_source('mysql_catalog', 'dbt_source', 'sell_user') }} +``` diff --git a/versioned_docs/version-3.0/ecosystem/dbt-doris-adapter.md b/versioned_docs/version-3.0/ecosystem/dbt-doris-adapter.md index aa88f54af4c..c8ca5da0786 100644 --- a/versioned_docs/version-3.0/ecosystem/dbt-doris-adapter.md +++ b/versioned_docs/version-3.0/ecosystem/dbt-doris-adapter.md @@ -8,15 +8,16 @@ # DBT Doris Adapter [DBT(Data Build Tool)](https://docs.getdbt.com/docs/introduction) is a component that focuses on doing T (Transform) in ELT (extraction, loading, transformation) - the "transformation data" link -The `dbt-doris` adapter is developed based on `dbt-core` 1.5.0 and relies on the `mysql-connector-python` driver to convert data to doris. +The `dbt-doris` adapter is developed based on `dbt-core` and relies on the `mysql-connector-python` driver to convert data to doris. git: https://github.com/apache/doris/tree/master/extension/dbt-doris ## version -| doris | python | dbt-core | -|---------|--------------|----------| -| >=1.2.5 | >=3.8,<=3.10 | >=1.5.0 | +| doris | python | dbt-core | dbt-doris | +|---------|-------------|----------|----------| +| >=1.2.5 | >=3.8,<=3.10| >=1.5.0 | <=0.3 | +| >=1.2.5 | >=3.9 | >=1.8.0 | >=0.4 | ## dbt-doris adapter Instructions @@ -463,3 +464,41 @@ select {% endif %} ``` + +### Customizing table data column types and precision sample reference + +The `schema.yaml` file configures `data_type` for `columns` in `models` as follows: + +```yaml +models: + - name: sell_user + description: "A dbt model named sell_user" + columns: + - name: user_id + data_type: BIGINT + - name: account_id + data_type: VARCHAR(12) + - name: status + - name: cost_sum + data_type: DECIMAL(38,9) + - name: update_time + data_type: DATETIME + - name: create_time + data_type: DATETIME +``` + +### Access catalog sample reference + +The [Data Catalog](../lakehouse/catalog-overview.md) is a reference to different data sources within the Doris data lake functionality, layered above the Database. +It is recommended to access it through the dbt-doris built-in Macros: `catalog_source` + +```sql +{{ config(materialized='table', replication_num=1) }} + +select * +-- use macros 'catalog_source' not macros 'source' +-- catalog name is 'mysql_catalog' +-- database name is 'dbt_source' +-- table name is 'sell_user' +from {{ catalog_source('mysql_catalog', 'dbt_source', 'sell_user') }} +``` --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org