[
https://issues.apache.org/jira/browse/FLINK-41028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hongshun Wang updated FLINK-41028:
----------------------------------
Description:
The Fluss pipeline sink currently accepts `properties.table.*` and
`properties.client.*`, but rejects lake-specific options such as
`properties.paimon.{*}` and `properties.iceberg.{*}`.
This prevents YAML pipelines from configuring the underlying lake tables
when automatically creating Fluss tables.
{{For example, the following sink configuration should be supported:}}
{code:java}
sink:
type: fluss
bootstrap.servers: localhost:9123
properties.table.datalake.enabled: 'true'
properties.table.datalake.freshness: 30s
properties.paimon.file.format: orc
properties.paimon.deletion-vectors.enabled: 'true' {code}
was:
{{The Fluss pipeline sink currently accepts `properties.table.*` and
`properties.client.*`, but rejects lake-specific options such as
`properties.paimon.*` and `properties.iceberg.*`.}}
{{}}
{{This prevents YAML pipelines from configuring the underlying lake tables
when automatically creating Fluss tables.}}
{{}}
{{For example, the following sink configuration should be supported:}}
{{}}
{code:java}
sink:
type: fluss
bootstrap.servers: localhost:9123
properties.table.datalake.enabled: 'true'
properties.table.datalake.freshness: 30s
properties.paimon.file.format: orc
properties.paimon.deletion-vectors.enabled: 'true' {code}
{{{}{}}}{{{}{}}}
{{}}
{{}}
> Support data lake table properties in the Fluss pipeline sink
> -------------------------------------------------------------
>
> Key: FLINK-41028
> URL: https://issues.apache.org/jira/browse/FLINK-41028
> Project: Flink
> Issue Type: New Feature
> Components: Flink CDC
> Affects Versions: cdc-3.7.0
> Reporter: Hongshun Wang
> Priority: Major
> Fix For: cdc-3.8.0
>
>
> The Fluss pipeline sink currently accepts `properties.table.*` and
> `properties.client.*`, but rejects lake-specific options such as
> `properties.paimon.{*}` and `properties.iceberg.{*}`.
>
> This prevents YAML pipelines from configuring the underlying lake tables
> when automatically creating Fluss tables.
> {{For example, the following sink configuration should be supported:}}
> {code:java}
> sink:
> type: fluss
> bootstrap.servers: localhost:9123
> properties.table.datalake.enabled: 'true'
> properties.table.datalake.freshness: 30s
> properties.paimon.file.format: orc
> properties.paimon.deletion-vectors.enabled: 'true' {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
