[jira] [Commented] (FLINK-14987) JDBCTableSource can't support DataTypes.DECIMAL
[
https://issues.apache.org/jira/browse/FLINK-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987454#comment-16987454
]
Jark Wu commented on FLINK-14987:
-
This issue has been fixed by FLINK-14645, and I also added a test case to cover
this.
> JDBCTableSource can't support DataTypes.DECIMAL
> ---
>
> Key: FLINK-14987
> URL: https://issues.apache.org/jira/browse/FLINK-14987
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Dezhi Cai
>Assignee: Zhenghua Gao
>Priority: Blocker
> Fix For: 1.10.0
>
>
>
> sample code 1 fail with ValidationException. After investigation, i find that
> the root cause may be related to the conversion between DecimalType and
> TypeInformation, please see sample code 2.
>
> sampe code 1:
> {code:java}
> public static void main(String[] args) {
> JDBCOptions options = JDBCOptions.builder()
> .setDBUrl("jdbc:mysql://127.0.0.1/test")
> .setTableName("table1")
> .setDriverName("com.mysql.jdbc.Driver")
> .setUsername("root")
> .setPassword("password")
> .build();
> TableSchema schema = TableSchema.builder()
> .field("a", DataTypes.INT())
> .field("b", DataTypes.BIGINT())
> .field("c", DataTypes.FLOAT())
> .field("d", DataTypes.DOUBLE())
> .field("e", DataTypes.DECIMAL(24,3))
> .field("f", DataTypes.TIMESTAMP(3))
> .build();
> JDBCTableSource source = JDBCTableSource.builder()
> .setOptions(options)
> .setSchema(schema)
> .build();
> TableSourceValidation.validateTableSource(source);
> }
> {code}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Type DECIMAL(24, 3) of table field 'LEGACY(BigDecimal)' does not match with
> type 'e; of the field 'LEGACY(BigDecimal)' of the TableSource return
> type.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Type DECIMAL(24, 3) of table
> field 'LEGACY(BigDecimal)' does not match with type 'e; of the field
> 'LEGACY(BigDecimal)' of the TableSource return type. at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalTypeEqualsPhysical(TableSourceValidation.java:184)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalToPhysicalMapping(TableSourceValidation.java:156)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateTableSource(TableSourceValidation.java:69)
> at com.moodys.demo.Demo.main(Demo.java:43)
>
> sample code 2 :
> {code:java}
> public static void main(String[] args) {
> DataType originalDataType = DataTypes.DECIMAL(24,3);
> TypeInformation legacyType =
> LegacyTypeInfoDataTypeConverter.toLegacyTypeInfo(originalDataType);
> DataType dataType =
> LegacyTypeInfoDataTypeConverter.toDataType(legacyType);
> System.out.println(originalDataType.equals(dataType));
> }
> // output: false{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14987) JDBCTableSource can't support DataTypes.DECIMAL
[
https://issues.apache.org/jira/browse/FLINK-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987437#comment-16987437
]
Kurt Young commented on FLINK-14987:
Sorry, looks like this issue is already covered by FLINK-14645
> JDBCTableSource can't support DataTypes.DECIMAL
> ---
>
> Key: FLINK-14987
> URL: https://issues.apache.org/jira/browse/FLINK-14987
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Dezhi Cai
>Assignee: Zhenghua Gao
>Priority: Blocker
> Fix For: 1.10.0
>
>
>
> sample code 1 fail with ValidationException. After investigation, i find that
> the root cause may be related to the conversion between DecimalType and
> TypeInformation, please see sample code 2.
>
> sampe code 1:
> {code:java}
> public static void main(String[] args) {
> JDBCOptions options = JDBCOptions.builder()
> .setDBUrl("jdbc:mysql://127.0.0.1/test")
> .setTableName("table1")
> .setDriverName("com.mysql.jdbc.Driver")
> .setUsername("root")
> .setPassword("password")
> .build();
> TableSchema schema = TableSchema.builder()
> .field("a", DataTypes.INT())
> .field("b", DataTypes.BIGINT())
> .field("c", DataTypes.FLOAT())
> .field("d", DataTypes.DOUBLE())
> .field("e", DataTypes.DECIMAL(24,3))
> .field("f", DataTypes.TIMESTAMP(3))
> .build();
> JDBCTableSource source = JDBCTableSource.builder()
> .setOptions(options)
> .setSchema(schema)
> .build();
> TableSourceValidation.validateTableSource(source);
> }
> {code}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Type DECIMAL(24, 3) of table field 'LEGACY(BigDecimal)' does not match with
> type 'e; of the field 'LEGACY(BigDecimal)' of the TableSource return
> type.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Type DECIMAL(24, 3) of table
> field 'LEGACY(BigDecimal)' does not match with type 'e; of the field
> 'LEGACY(BigDecimal)' of the TableSource return type. at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalTypeEqualsPhysical(TableSourceValidation.java:184)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalToPhysicalMapping(TableSourceValidation.java:156)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateTableSource(TableSourceValidation.java:69)
> at com.moodys.demo.Demo.main(Demo.java:43)
>
> sample code 2 :
> {code:java}
> public static void main(String[] args) {
> DataType originalDataType = DataTypes.DECIMAL(24,3);
> TypeInformation legacyType =
> LegacyTypeInfoDataTypeConverter.toLegacyTypeInfo(originalDataType);
> DataType dataType =
> LegacyTypeInfoDataTypeConverter.toDataType(legacyType);
> System.out.println(originalDataType.equals(dataType));
> }
> // output: false{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14987) JDBCTableSource can't support DataTypes.DECIMAL
[
https://issues.apache.org/jira/browse/FLINK-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986624#comment-16986624
]
Kurt Young commented on FLINK-14987:
Thanks [~docete], assigned to you.
> JDBCTableSource can't support DataTypes.DECIMAL
> ---
>
> Key: FLINK-14987
> URL: https://issues.apache.org/jira/browse/FLINK-14987
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Dezhi Cai
>Priority: Blocker
> Fix For: 1.10.0
>
>
>
> sample code 1 fail with ValidationException. After investigation, i find that
> the root cause may be related to the conversion between DecimalType and
> TypeInformation, please see sample code 2.
>
> sampe code 1:
> {code:java}
> public static void main(String[] args) {
> JDBCOptions options = JDBCOptions.builder()
> .setDBUrl("jdbc:mysql://127.0.0.1/test")
> .setTableName("table1")
> .setDriverName("com.mysql.jdbc.Driver")
> .setUsername("root")
> .setPassword("password")
> .build();
> TableSchema schema = TableSchema.builder()
> .field("a", DataTypes.INT())
> .field("b", DataTypes.BIGINT())
> .field("c", DataTypes.FLOAT())
> .field("d", DataTypes.DOUBLE())
> .field("e", DataTypes.DECIMAL(24,3))
> .field("f", DataTypes.TIMESTAMP(3))
> .build();
> JDBCTableSource source = JDBCTableSource.builder()
> .setOptions(options)
> .setSchema(schema)
> .build();
> TableSourceValidation.validateTableSource(source);
> }
> {code}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Type DECIMAL(24, 3) of table field 'LEGACY(BigDecimal)' does not match with
> type 'e; of the field 'LEGACY(BigDecimal)' of the TableSource return
> type.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Type DECIMAL(24, 3) of table
> field 'LEGACY(BigDecimal)' does not match with type 'e; of the field
> 'LEGACY(BigDecimal)' of the TableSource return type. at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalTypeEqualsPhysical(TableSourceValidation.java:184)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalToPhysicalMapping(TableSourceValidation.java:156)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateTableSource(TableSourceValidation.java:69)
> at com.moodys.demo.Demo.main(Demo.java:43)
>
> sample code 2 :
> {code:java}
> public static void main(String[] args) {
> DataType originalDataType = DataTypes.DECIMAL(24,3);
> TypeInformation legacyType =
> LegacyTypeInfoDataTypeConverter.toLegacyTypeInfo(originalDataType);
> DataType dataType =
> LegacyTypeInfoDataTypeConverter.toDataType(legacyType);
> System.out.println(originalDataType.equals(dataType));
> }
> // output: false{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14987) JDBCTableSource can't support DataTypes.DECIMAL
[
https://issues.apache.org/jira/browse/FLINK-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986621#comment-16986621
]
Zhenghua Gao commented on FLINK-14987:
--
[~ykt836] OK, i will take it and prepare a PR soon
> JDBCTableSource can't support DataTypes.DECIMAL
> ---
>
> Key: FLINK-14987
> URL: https://issues.apache.org/jira/browse/FLINK-14987
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Dezhi Cai
>Priority: Blocker
> Fix For: 1.10.0
>
>
>
> sample code 1 fail with ValidationException. After investigation, i find that
> the root cause may be related to the conversion between DecimalType and
> TypeInformation, please see sample code 2.
>
> sampe code 1:
> {code:java}
> public static void main(String[] args) {
> JDBCOptions options = JDBCOptions.builder()
> .setDBUrl("jdbc:mysql://127.0.0.1/test")
> .setTableName("table1")
> .setDriverName("com.mysql.jdbc.Driver")
> .setUsername("root")
> .setPassword("password")
> .build();
> TableSchema schema = TableSchema.builder()
> .field("a", DataTypes.INT())
> .field("b", DataTypes.BIGINT())
> .field("c", DataTypes.FLOAT())
> .field("d", DataTypes.DOUBLE())
> .field("e", DataTypes.DECIMAL(24,3))
> .field("f", DataTypes.TIMESTAMP(3))
> .build();
> JDBCTableSource source = JDBCTableSource.builder()
> .setOptions(options)
> .setSchema(schema)
> .build();
> TableSourceValidation.validateTableSource(source);
> }
> {code}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Type DECIMAL(24, 3) of table field 'LEGACY(BigDecimal)' does not match with
> type 'e; of the field 'LEGACY(BigDecimal)' of the TableSource return
> type.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Type DECIMAL(24, 3) of table
> field 'LEGACY(BigDecimal)' does not match with type 'e; of the field
> 'LEGACY(BigDecimal)' of the TableSource return type. at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalTypeEqualsPhysical(TableSourceValidation.java:184)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalToPhysicalMapping(TableSourceValidation.java:156)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateTableSource(TableSourceValidation.java:69)
> at com.moodys.demo.Demo.main(Demo.java:43)
>
> sample code 2 :
> {code:java}
> public static void main(String[] args) {
> DataType originalDataType = DataTypes.DECIMAL(24,3);
> TypeInformation legacyType =
> LegacyTypeInfoDataTypeConverter.toLegacyTypeInfo(originalDataType);
> DataType dataType =
> LegacyTypeInfoDataTypeConverter.toDataType(legacyType);
> System.out.println(originalDataType.equals(dataType));
> }
> // output: false{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14987) JDBCTableSource can't support DataTypes.DECIMAL
[
https://issues.apache.org/jira/browse/FLINK-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986540#comment-16986540
]
Kurt Young commented on FLINK-14987:
[~caidezhi655] [~docete] Would some of you like to take this?
> JDBCTableSource can't support DataTypes.DECIMAL
> ---
>
> Key: FLINK-14987
> URL: https://issues.apache.org/jira/browse/FLINK-14987
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Dezhi Cai
>Priority: Blocker
> Fix For: 1.10.0
>
>
>
> sample code 1 fail with ValidationException. After investigation, i find that
> the root cause may be related to the conversion between DecimalType and
> TypeInformation, please see sample code 2.
>
> sampe code 1:
> {code:java}
> public static void main(String[] args) {
> JDBCOptions options = JDBCOptions.builder()
> .setDBUrl("jdbc:mysql://127.0.0.1/test")
> .setTableName("table1")
> .setDriverName("com.mysql.jdbc.Driver")
> .setUsername("root")
> .setPassword("password")
> .build();
> TableSchema schema = TableSchema.builder()
> .field("a", DataTypes.INT())
> .field("b", DataTypes.BIGINT())
> .field("c", DataTypes.FLOAT())
> .field("d", DataTypes.DOUBLE())
> .field("e", DataTypes.DECIMAL(24,3))
> .field("f", DataTypes.TIMESTAMP(3))
> .build();
> JDBCTableSource source = JDBCTableSource.builder()
> .setOptions(options)
> .setSchema(schema)
> .build();
> TableSourceValidation.validateTableSource(source);
> }
> {code}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Type DECIMAL(24, 3) of table field 'LEGACY(BigDecimal)' does not match with
> type 'e; of the field 'LEGACY(BigDecimal)' of the TableSource return
> type.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Type DECIMAL(24, 3) of table
> field 'LEGACY(BigDecimal)' does not match with type 'e; of the field
> 'LEGACY(BigDecimal)' of the TableSource return type. at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalTypeEqualsPhysical(TableSourceValidation.java:184)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalToPhysicalMapping(TableSourceValidation.java:156)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateTableSource(TableSourceValidation.java:69)
> at com.moodys.demo.Demo.main(Demo.java:43)
>
> sample code 2 :
> {code:java}
> public static void main(String[] args) {
> DataType originalDataType = DataTypes.DECIMAL(24,3);
> TypeInformation legacyType =
> LegacyTypeInfoDataTypeConverter.toLegacyTypeInfo(originalDataType);
> DataType dataType =
> LegacyTypeInfoDataTypeConverter.toDataType(legacyType);
> System.out.println(originalDataType.equals(dataType));
> }
> // output: false{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14987) JDBCTableSource can't support DataTypes.DECIMAL
[
https://issues.apache.org/jira/browse/FLINK-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986534#comment-16986534
]
Kurt Young commented on FLINK-14987:
Yes, I think it's better to fix it in 1.10
> JDBCTableSource can't support DataTypes.DECIMAL
> ---
>
> Key: FLINK-14987
> URL: https://issues.apache.org/jira/browse/FLINK-14987
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Dezhi Cai
>Priority: Blocker
>
>
> sample code 1 fail with ValidationException. After investigation, i find that
> the root cause may be related to the conversion between DecimalType and
> TypeInformation, please see sample code 2.
>
> sampe code 1:
> {code:java}
> public static void main(String[] args) {
> JDBCOptions options = JDBCOptions.builder()
> .setDBUrl("jdbc:mysql://127.0.0.1/test")
> .setTableName("table1")
> .setDriverName("com.mysql.jdbc.Driver")
> .setUsername("root")
> .setPassword("password")
> .build();
> TableSchema schema = TableSchema.builder()
> .field("a", DataTypes.INT())
> .field("b", DataTypes.BIGINT())
> .field("c", DataTypes.FLOAT())
> .field("d", DataTypes.DOUBLE())
> .field("e", DataTypes.DECIMAL(24,3))
> .field("f", DataTypes.TIMESTAMP(3))
> .build();
> JDBCTableSource source = JDBCTableSource.builder()
> .setOptions(options)
> .setSchema(schema)
> .build();
> TableSourceValidation.validateTableSource(source);
> }
> {code}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Type DECIMAL(24, 3) of table field 'LEGACY(BigDecimal)' does not match with
> type 'e; of the field 'LEGACY(BigDecimal)' of the TableSource return
> type.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Type DECIMAL(24, 3) of table
> field 'LEGACY(BigDecimal)' does not match with type 'e; of the field
> 'LEGACY(BigDecimal)' of the TableSource return type. at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalTypeEqualsPhysical(TableSourceValidation.java:184)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalToPhysicalMapping(TableSourceValidation.java:156)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateTableSource(TableSourceValidation.java:69)
> at com.moodys.demo.Demo.main(Demo.java:43)
>
> sample code 2 :
> {code:java}
> public static void main(String[] args) {
> DataType originalDataType = DataTypes.DECIMAL(24,3);
> TypeInformation legacyType =
> LegacyTypeInfoDataTypeConverter.toLegacyTypeInfo(originalDataType);
> DataType dataType =
> LegacyTypeInfoDataTypeConverter.toDataType(legacyType);
> System.out.println(originalDataType.equals(dataType));
> }
> // output: false{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-14987) JDBCTableSource can't support DataTypes.DECIMAL
[
https://issues.apache.org/jira/browse/FLINK-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984714#comment-16984714
]
Zhenghua Gao commented on FLINK-14987:
--
Yes. The JDBCTableSource can't support DataTypes.DECIMAL (which support
precision and scale) now. The reason is the JDBCTableSource use a deprecated
interface *getReturnType* for the result type, which is the old type system
style. The recommended way is to use *getProducedDataType* instead which use
the new type system on DataTypes.
[~jark] [~ykt836] Should we repair all connectors to support new type system in
1.10 ?
> JDBCTableSource can't support DataTypes.DECIMAL
> ---
>
> Key: FLINK-14987
> URL: https://issues.apache.org/jira/browse/FLINK-14987
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Table SQL / API
>Affects Versions: 1.9.0, 1.9.1
>Reporter: Dezhi Cai
>Priority: Blocker
>
>
> sample code 1 fail with ValidationException. After investigation, i find that
> the root cause may be related to the conversion between DecimalType and
> TypeInformation, please see sample code 2.
>
> sampe code 1:
> {code:java}
> public static void main(String[] args) {
> JDBCOptions options = JDBCOptions.builder()
> .setDBUrl("jdbc:mysql://127.0.0.1/test")
> .setTableName("table1")
> .setDriverName("com.mysql.jdbc.Driver")
> .setUsername("root")
> .setPassword("password")
> .build();
> TableSchema schema = TableSchema.builder()
> .field("a", DataTypes.INT())
> .field("b", DataTypes.BIGINT())
> .field("c", DataTypes.FLOAT())
> .field("d", DataTypes.DOUBLE())
> .field("e", DataTypes.DECIMAL(24,3))
> .field("f", DataTypes.TIMESTAMP(3))
> .build();
> JDBCTableSource source = JDBCTableSource.builder()
> .setOptions(options)
> .setSchema(schema)
> .build();
> TableSourceValidation.validateTableSource(source);
> }
> {code}
> Exception in thread "main" org.apache.flink.table.api.ValidationException:
> Type DECIMAL(24, 3) of table field 'LEGACY(BigDecimal)' does not match with
> type 'e; of the field 'LEGACY(BigDecimal)' of the TableSource return
> type.Exception in thread "main"
> org.apache.flink.table.api.ValidationException: Type DECIMAL(24, 3) of table
> field 'LEGACY(BigDecimal)' does not match with type 'e; of the field
> 'LEGACY(BigDecimal)' of the TableSource return type. at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalTypeEqualsPhysical(TableSourceValidation.java:184)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateLogicalToPhysicalMapping(TableSourceValidation.java:156)
> at
> org.apache.flink.table.sources.TableSourceValidation.validateTableSource(TableSourceValidation.java:69)
> at com.moodys.demo.Demo.main(Demo.java:43)
>
> sample code 2 :
> {code:java}
> public static void main(String[] args) {
> DataType originalDataType = DataTypes.DECIMAL(24,3);
> TypeInformation legacyType =
> LegacyTypeInfoDataTypeConverter.toLegacyTypeInfo(originalDataType);
> DataType dataType =
> LegacyTypeInfoDataTypeConverter.toDataType(legacyType);
> System.out.println(originalDataType.equals(dataType));
> }
> // output: false{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
