[
https://issues.apache.org/jira/browse/FLINK-23269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375299#comment-17375299
]
silence commented on FLINK-23269:
---------------------------------
[~jark]I think this is a problem of implicit conversion of JSON type. Just as
JSON array and JSON object can be defined as varchar, JSON number should also
be OK. Now it works normally without defining decimal
> json format decode number to vatchar error when define Decimal type in ddl
> --------------------------------------------------------------------------
>
> Key: FLINK-23269
> URL: https://issues.apache.org/jira/browse/FLINK-23269
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
> Reporter: silence
> Priority: Minor
>
> when use json format and define decimal
> json:
> {"c1":50.0,"c2":50.0}
> ddl:
> create table(
> c1 varchar,
> c2 decimal
> )with(
> 'format'='json'
> )
> output:
> {"c1":"5E+1","c2":50.0}
> And the following unit tests will produce the following results
> {"double1":50.0,"double2":50.0,"double3":"50.0","float1":20.0,"float2":20.0,"float3":"20.0"}
> java.lang.AssertionError:
> Expected :+I[50.0, 50.0, 50.0, 20.0, 20.0, 20.0]
> Actual :+I[5E+1, 50.0, 50.0, 2E+1, 20.0, 20.0]
> {code:java}
> @Test
> public void testDeserialization() throws Exception {
> double doubleValue = 50.0;
> float floatValue = 20.0f;
> ObjectMapper objectMapper = new ObjectMapper();
> ObjectNode root = objectMapper.createObjectNode();
> root.put("double1", doubleValue);
> root.put("double2", doubleValue);
> root.put("double3", String.valueOf(doubleValue));
> root.put("float1", floatValue);
> root.put("float2", floatValue);
> root.put("float3", String.valueOf(floatValue));
> byte[] serializedJson = objectMapper.writeValueAsBytes(root);
> System.out.println(new String(serializedJson));
> DataType dataType =
> ROW(
> FIELD("double1", STRING()),
> FIELD("double2", DECIMAL(10,1)),
> FIELD("double3", DOUBLE()),
> FIELD("float1", STRING()),
> FIELD("float2", DECIMAL(10,1)),
> FIELD("float3", FLOAT()));
> RowType rowType = (RowType) dataType.getLogicalType();
> JsonRowDataDeserializationSchema deserializationSchema =
> new JsonRowDataDeserializationSchema(
> rowType,
> InternalTypeInfo.of(rowType),
> false,
> false,
> TimestampFormat.ISO_8601);
> Row expected = new Row(6);
> expected.setField(0, String.valueOf(doubleValue));
> expected.setField(1, String.valueOf(doubleValue));
> expected.setField(2, doubleValue);
> expected.setField(3, String.valueOf(floatValue));
> expected.setField(4, String.valueOf(floatValue));
> expected.setField(5, floatValue);
> RowData rowData = deserializationSchema.deserialize(serializedJson);
> Row actual = convertToExternal(rowData, dataType);
> assertEquals(expected, actual);
> }
> {code}
> when define the DecimalType
> ObjectMapper will enable USE_BIG_DECIMAL_FOR_FLOATS
> and jsonNode.asText() will call BigDecimal toString method
> {code:java}
> boolean hasDecimalType =
> LogicalTypeChecks.hasNested(rowType, t -> t instanceof
> DecimalType);
> if (hasDecimalType) {
>
> objectMapper.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
