[jira] [Commented] (FLINK-34935) TIMESTAMP_LTZ type Unsupported when using JdbcCatalog to read from Postgres
[ https://issues.apache.org/jira/browse/FLINK-34935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17848664#comment-17848664 ] Pietro commented on FLINK-34935: Related issue: FLINK-35053 > TIMESTAMP_LTZ type Unsupported when using JdbcCatalog to read from Postgres > --- > > Key: FLINK-34935 > URL: https://issues.apache.org/jira/browse/FLINK-34935 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.18.1 > Environment: flink: 1.17.0 > flink-connector-jdbc: 3.1.0-1.17 > postgres: 14.5 > java: 11 >Reporter: Du Yuzhou >Priority: Major > Labels: flink, jdbc_connector, postgres > Original Estimate: 72h > Remaining Estimate: 72h > > When I use JdbcCatalog to select from postgres table, it throw Exception: > {code:java} > Exception in thread "main" java.lang.UnsupportedOperationException: > Unsupported type:TIMESTAMP_LTZ(6) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) > at > org.apache.flink.connector.jdbc.internal.converter.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) > at > org.apache.flink.connector.jdbc.internal.converter.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) > at > org.apache.flink.connector.jdbc.internal.converter.PostgresRowConverter.(PostgresRowConverter.java:47) > at > org.apache.flink.connector.jdbc.dialect.psql.PostgresDialect.getRowConverter(PostgresDialect.java:50) > at > org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:182) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:466) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) > at > org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) > at > org.apache.calcite.rel.core.RelFactories$TableScanFactoryImpl.createScan(RelFactories.java:481) > at org.apache.calcite.tools.RelBuilder.scan(RelBuilder.java:1757) > at > org.apache.flink.table.planner.plan.QueryOperationConverter$SingleRelVisitor.visit(QueryOperationConverter.java:366) > at > org.apache.flink.table.planner.plan.QueryOperationConverter$SingleRelVisitor.visit(QueryOperationConverter.java:158) > at > org.apache.flink.table.operations.SourceQueryOperation.accept(SourceQueryOperation.java:86) > at > org.apache.flink.table.planner.plan.QueryOperationConverter.defaultMethod(QueryOperationConverter.java:155) > at > org.apache.flink.table.planner.plan.QueryOperationConverter.defaultMethod(QueryOperationConverter.java:135) > at > org.apache.flink.table.operations.utils.QueryOperationDefaultVisitor.visit(QueryOperationDefaultVisitor.java:92) > at > org.apache.flink.table.operations.SourceQueryOperation.accept(SourceQueryOperation.java:86) > at > org.apache.flink.table.planner.calcite.FlinkRelBuilder.queryOperation(FlinkRelBuilder.java:261) > at > org.apache.flink.table.planner.catalog.QueryOperationCatalogViewTable.convertToRel(QueryOperationCatalogViewTable.java:80) > at > org.apache.flink.table.planner.plan.schema.ExpandingPreparingTable.expand(ExpandingPreparingTable.java:70) > at > org.apache.flink.table.planner.plan.schema.ExpandingPreparingTable.toRel(ExpandingPreparingTable.java:57) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:3743) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2666) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2233) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2147) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2092) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:700) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:686) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3589) > at >
[jira] [Updated] (FLINK-35053) TIMESTAMP with LOCAL TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP_LTZ{}}}, nor {{TIMESTAMP WITH TIME ZONE}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.externalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L246], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with LOCAL TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP_LTZ{}}}, nor {{TIMESTAMP WITH TIME ZONE}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.externalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L246], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with LOCAL TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Summary: TIMESTAMP with LOCAL TIME ZONE not supported by JDBC connector for Postgres (was: TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres) > TIMESTAMP with LOCAL TIME ZONE not supported by JDBC connector for Postgres > --- > > Key: FLINK-35053 > URL: https://issues.apache.org/jira/browse/FLINK-35053 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.19.0, 1.18.1, jdbc-3.1.2 >Reporter: Pietro >Priority: Major > Attachments: Timestamp.png, TimestampData.png, > createExternalConverter.png > > > The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, > nor {{TIMESTAMP_LTZ}} types. > Related issues: FLINK-22199, FLINK-20869 > h2. Problem Explanation > A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . > {code:sql} > -- Postgres DDL > CREATE TABLE target_table ( > tm_tz TIMESTAMP WITH TIME ZONE > ) > {code} > In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and > our goal is to sink it to {{{}target_table{}}}. > {code:sql} > -- Flink DDL > CREATE TABLE sink ( > tm_tz TIMESTAMP_LTZ(6) > ) WITH ( > 'connector' = 'jdbc', > 'table-name' = 'target_table' > ... > ) > {code} > According to > [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while > {{TIMESTAMP_WITH_TIME_ZONE}} is not. > However, when the converter is created via > [AbstractJdbcRowConverter.externalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L246], > it throws an {{UnsupportedOperationException}} since > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while > [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] > is. > {code:java} > Exception in thread "main" java.lang.UnsupportedOperationException: > Unsupported type:TIMESTAMP_LTZ(6) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) > at > org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) > at > org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) > at >
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.externalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L246], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.externalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L246], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.externalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L246], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.externalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L246], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.externalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L246], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.externalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L246], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: !createExternalConverter.png! The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Attachment: createExternalConverter.png > TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres > - > > Key: FLINK-35053 > URL: https://issues.apache.org/jira/browse/FLINK-35053 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.19.0, 1.18.1, jdbc-3.1.2 >Reporter: Pietro >Priority: Major > Attachments: Timestamp.png, TimestampData.png, > createExternalConverter.png > > > The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, > nor {{TIMESTAMP_LTZ}} types. > Related issues: FLINK-22199, FLINK-20869 > h2. Problem Explanation > A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . > {code:sql} > -- Postgres DDL > CREATE TABLE target_table ( > tm_tz TIMESTAMP WITH TIME ZONE > ) > {code} > In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and > our goal is to sink it to {{{}target_table{}}}. > {code:sql} > -- Flink DDL > CREATE TABLE sink ( > tm_tz TIMESTAMP_LTZ(6) > ) WITH ( > 'connector' = 'jdbc', > 'table-name' = 'target_table' > ... > ) > {code} > According to > [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while > {{TIMESTAMP_WITH_TIME_ZONE}} is not. > However, when the converter is created via > [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], > it throws an {{UnsupportedOperationException}} since > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while > [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] > is. > {code:java} > Exception in thread "main" java.lang.UnsupportedOperationException: > Unsupported type:TIMESTAMP_LTZ(6) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) > at > org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) > at > org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) > at >
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Attachment: Timestamp.png TimestampData.png > TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres > - > > Key: FLINK-35053 > URL: https://issues.apache.org/jira/browse/FLINK-35053 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.19.0, 1.18.1, jdbc-3.1.2 >Reporter: Pietro >Priority: Major > Attachments: Timestamp.png, TimestampData.png > > > The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, > nor {{TIMESTAMP_LTZ}} types. > Related issues: FLINK-22199, FLINK-20869 > h2. Problem Explanation > A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . > {code:sql} > -- Postgres DDL > CREATE TABLE target_table ( > tm_tz TIMESTAMP WITH TIME ZONE > ) > {code} > In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and > our goal is to sink it to {{{}target_table{}}}. > {code:sql} > -- Flink DDL > CREATE TABLE sink ( > tm_tz TIMESTAMP_LTZ(6) > ) WITH ( > 'connector' = 'jdbc', > 'table-name' = 'target_table' > ... > ) > {code} > According to > [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while > {{TIMESTAMP_WITH_TIME_ZONE}} is not. > However, when the converter is created via > [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], > it throws an {{UnsupportedOperationException}} since > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while > [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] > is. > {code:java} > Exception in thread "main" java.lang.UnsupportedOperationException: > Unsupported type:TIMESTAMP_LTZ(6) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) > at > org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) > at > org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) > at >
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Description: The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at org.apache.flink.table.planner.operations.SqlNodeConvertContext.toRelRoot(SqlNodeConvertContext.java:69) at org.apache.flink.table.planner.operations.converters.SqlQueryConverter.convertSqlNode(SqlQueryConverter.java:48) at
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Attachment: (was: image-2024-04-08-18-05-00-820.png) > TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres > - > > Key: FLINK-35053 > URL: https://issues.apache.org/jira/browse/FLINK-35053 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.19.0, 1.18.1, jdbc-3.1.2 >Reporter: Pietro >Priority: Major > > The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, > nor {{TIMESTAMP_LTZ}} types. > Related issues: FLINK-22199, FLINK-20869 > h2. Problem Explanation > A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . > {code:sql} > -- Postgres DDL > CREATE TABLE target_table ( > tm_tz TIMESTAMP WITH TIME ZONE > ) > {code} > In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and > our goal is to sink it to {{{}target_table{}}}. > {code:sql} > -- Flink DDL > CREATE TABLE sink ( > tm_tz TIMESTAMP_LTZ(6) > ) WITH ( > 'connector' = 'jdbc', > 'table-name' = 'target_table' > ... > ) > {code} > According to > [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while > {{TIMESTAMP_WITH_TIME_ZONE}} is not. > However, when the converter is created via > [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], > it throws an {{UnsupportedOperationException}} since > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while > [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] > is. > {code:java} > Exception in thread "main" java.lang.UnsupportedOperationException: > Unsupported type:TIMESTAMP_LTZ(6) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) > at > org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) > at > org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) > at >
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Attachment: (was: image-2024-04-08-18-05-09-213.png) > TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres > - > > Key: FLINK-35053 > URL: https://issues.apache.org/jira/browse/FLINK-35053 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.19.0, 1.18.1, jdbc-3.1.2 >Reporter: Pietro >Priority: Major > > The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, > nor {{TIMESTAMP_LTZ}} types. > Related issues: FLINK-22199, FLINK-20869 > h2. Problem Explanation > A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . > {code:sql} > -- Postgres DDL > CREATE TABLE target_table ( > tm_tz TIMESTAMP WITH TIME ZONE > ) > {code} > In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and > our goal is to sink it to {{{}target_table{}}}. > {code:sql} > -- Flink DDL > CREATE TABLE sink ( > tm_tz TIMESTAMP_LTZ(6) > ) WITH ( > 'connector' = 'jdbc', > 'table-name' = 'target_table' > ... > ) > {code} > According to > [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while > {{TIMESTAMP_WITH_TIME_ZONE}} is not. > However, when the converter is created via > [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], > it throws an {{UnsupportedOperationException}} since > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while > [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] > is. > {code:java} > Exception in thread "main" java.lang.UnsupportedOperationException: > Unsupported type:TIMESTAMP_LTZ(6) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) > at > org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) > at > org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) > at >
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Attachment: image-2024-04-08-18-05-09-213.png > TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres > - > > Key: FLINK-35053 > URL: https://issues.apache.org/jira/browse/FLINK-35053 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.19.0, 1.18.1, jdbc-3.1.2 >Reporter: Pietro >Priority: Major > Attachments: image-2024-04-08-18-05-00-820.png, > image-2024-04-08-18-05-09-213.png > > > The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, > nor {{TIMESTAMP_LTZ}} types. > Related issues: FLINK-22199, FLINK-20869 > h2. Problem Explanation > A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . > {code:sql} > -- Postgres DDL > CREATE TABLE target_table ( > tm_tz TIMESTAMP WITH TIME ZONE > ) > {code} > In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and > our goal is to sink it to {{{}target_table{}}}. > {code:sql} > -- Flink DDL > CREATE TABLE sink ( > tm_tz TIMESTAMP_LTZ(6) > ) WITH ( > 'connector' = 'jdbc', > 'table-name' = 'target_table' > ... > ) > {code} > According to > [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while > {{TIMESTAMP_WITH_TIME_ZONE}} is not. > However, when the converter is created via > [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], > it throws an {{UnsupportedOperationException}} since > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while > [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] > is. > {code:java} > Exception in thread "main" java.lang.UnsupportedOperationException: > Unsupported type:TIMESTAMP_LTZ(6) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) > at > org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) > at > org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) > at >
[jira] [Updated] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
[ https://issues.apache.org/jira/browse/FLINK-35053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pietro updated FLINK-35053: --- Attachment: image-2024-04-08-18-05-00-820.png > TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres > - > > Key: FLINK-35053 > URL: https://issues.apache.org/jira/browse/FLINK-35053 > Project: Flink > Issue Type: Bug > Components: Connectors / JDBC >Affects Versions: 1.19.0, 1.18.1, jdbc-3.1.2 >Reporter: Pietro >Priority: Major > Attachments: image-2024-04-08-18-05-00-820.png, > image-2024-04-08-18-05-09-213.png > > > The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, > nor {{TIMESTAMP_LTZ}} types. > Related issues: FLINK-22199, FLINK-20869 > h2. Problem Explanation > A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . > {code:sql} > -- Postgres DDL > CREATE TABLE target_table ( > tm_tz TIMESTAMP WITH TIME ZONE > ) > {code} > In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and > our goal is to sink it to {{{}target_table{}}}. > {code:sql} > -- Flink DDL > CREATE TABLE sink ( > tm_tz TIMESTAMP_LTZ(6) > ) WITH ( > 'connector' = 'jdbc', > 'table-name' = 'target_table' > ... > ) > {code} > According to > [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while > {{TIMESTAMP_WITH_TIME_ZONE}} is not. > However, when the converter is created via > [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], > it throws an {{UnsupportedOperationException}} since > {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while > [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] > is. > {code:java} > Exception in thread "main" java.lang.UnsupportedOperationException: > Unsupported type:TIMESTAMP_LTZ(6) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) > at > org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) > at > org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) > at > org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) > at > org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) > at > org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) > at > org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) > at > org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) > at >
[jira] [Created] (FLINK-35053) TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres
Pietro created FLINK-35053: -- Summary: TIMESTAMP with TIME ZONE not supported by JDBC connector for Postgres Key: FLINK-35053 URL: https://issues.apache.org/jira/browse/FLINK-35053 Project: Flink Issue Type: Bug Components: Connectors / JDBC Affects Versions: jdbc-3.1.2, 1.18.1, 1.19.0 Reporter: Pietro The JDBC sink for Postgres does not support {{{}TIMESTAMP WITH TIME ZONE{}}}, nor {{TIMESTAMP_LTZ}} types. Related issues: FLINK-22199, FLINK-20869 h2. Problem Explanation A Postgres {{target_table}} has a field {{tm_tz}} of type {{timestamptz}} . {code:sql} -- Postgres DDL CREATE TABLE target_table ( tm_tz TIMESTAMP WITH TIME ZONE ) {code} In Flink we have a table with a column of type {{{}TIMESTAMP_LTZ(6){}}}, and our goal is to sink it to {{{}target_table{}}}. {code:sql} -- Flink DDL CREATE TABLE sink ( tm_tz TIMESTAMP_LTZ(6) ) WITH ( 'connector' = 'jdbc', 'table-name' = 'target_table' ... ) {code} According to [AbstractPostgresCompatibleDialect.supportedTypes()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L109], {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is supported, while {{TIMESTAMP_WITH_TIME_ZONE}} is not. However, when the converter is created via [AbstractJdbcRowConverter.createInternalConverter()|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L132], it throws an {{UnsupportedOperationException}} since {{TIMESTAMP_WITH_LOCAL_TIME_ZONE}} is *not* among the available types, while [{{TIMESTAMP_WITH_TIME_ZONE}}|https://github.com/apache/flink-connector-jdbc/blob/7025642d88ff661e486745b23569595e1813a1d0/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/converter/AbstractJdbcRowConverter.java#L168] is. {code:java} Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported type:TIMESTAMP_LTZ(6) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createInternalConverter(AbstractJdbcRowConverter.java:186) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createPrimitiveConverter(PostgresRowConverter.java:99) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.createInternalConverter(PostgresRowConverter.java:58) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.createNullableInternalConverter(AbstractJdbcRowConverter.java:118) at org.apache.flink.connector.jdbc.converter.AbstractJdbcRowConverter.(AbstractJdbcRowConverter.java:68) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresRowConverter.(PostgresRowConverter.java:47) at org.apache.flink.connector.jdbc.databases.postgres.dialect.PostgresDialect.getRowConverter(PostgresDialect.java:51) at org.apache.flink.connector.jdbc.table.JdbcDynamicTableSource.getScanRuntimeProvider(JdbcDynamicTableSource.java:184) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.validateScanSource(DynamicSourceUtils.java:478) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.prepareDynamicSource(DynamicSourceUtils.java:161) at org.apache.flink.table.planner.connectors.DynamicSourceUtils.convertSourceToRel(DynamicSourceUtils.java:125) at org.apache.flink.table.planner.plan.schema.CatalogSourceTable.toRel(CatalogSourceTable.java:118) at org.apache.calcite.sql2rel.SqlToRelConverter.toRel(SqlToRelConverter.java:4002) at org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:2872) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2432) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2346) at org.apache.calcite.sql2rel.SqlToRelConverter.convertFrom(SqlToRelConverter.java:2291) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:728) at org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:714) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:3848) at org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:618) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.org$apache$flink$table$planner$calcite$FlinkPlannerImpl$$rel(FlinkPlannerImpl.scala:229) at org.apache.flink.table.planner.calcite.FlinkPlannerImpl.rel(FlinkPlannerImpl.scala:205) at
[jira] [Created] (FLINK-34722) Support conditional upserts with Postgres JDBC sink
Pietro created FLINK-34722: -- Summary: Support conditional upserts with Postgres JDBC sink Key: FLINK-34722 URL: https://issues.apache.org/jira/browse/FLINK-34722 Project: Flink Issue Type: Improvement Components: Connectors / JDBC Affects Versions: jdbc-3.1.2 Reporter: Pietro The default Postgres dialect used by the JDBC sink for PostgreSQL DBs does not support custom _WHERE_ conditions inside upsert statements at the moment. Indeed, upsert statements returned by the {{[getUpsertStatement()|https://github.com/apache/flink-connector-jdbc/blob/95294ffbc57c93c2af34cda94c27fc5781e06177/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java#L61]}} method are limited to: {code:sql} ON CONFLICT (col1, ..., colN) DO UPDATE SET (col1=EXCLUDED.col1, ..., colN=EXCLUDED.colN) {code} PostgreSQL allows a finer-grained control of upsert statements by specifying a _WHERE_ statement (see [ON CONFLICT Clause|https://www.postgresql.org/docs/current/sql-insert.html#SQL-ON-CONFLICT]), for instance: {code:sql} ON CONFLICT (col1, ..., colN) DO UPDATE SET (col1=EXCLUDED.col1, ..., colN=EXCLUDED.colN) WHERE colN < EXCLUDED.colN {code} This could be useful in many use cases, for instance, in a CDC scenario where a batch reconciliation process has written records in the destination, which now face the risk of being overwritten by late arriving, stale, records in the streaming pipeline (adding a condition on the operation timestamp could protect from these events). My proposal is to extend the {{[AbstractPostgresCompatibleDialect|https://github.com/apache/flink-connector-jdbc/blob/main/flink-connector-jdbc/src/main/java/org/apache/flink/connector/jdbc/dialect/AbstractPostgresCompatibleDialect.java]}} functionalities by making the upsert query support _WHERE_ statements provided by users. I'm thinking of two possible approaches, but I'd love to hear your opinion on this: # provide the statement through options of the JDBC sink connector. # allow users to plug custom dialects without them having to rewrite the whole JDBC sink (about this I'll open a separate issue soon) Thanks for your consideration -- This message was sent by Atlassian Jira (v8.20.10#820010)