[
https://issues.apache.org/jira/browse/FLINK-23269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu updated FLINK-23269:
----------------------------
Component/s: Table SQL / Ecosystem
> json format decode number to vatchar error when define Decimal type in ddl
> --------------------------------------------------------------------------
>
> Key: FLINK-23269
> URL: https://issues.apache.org/jira/browse/FLINK-23269
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
> Reporter: silence
> Priority: Minor
>
> when use json format and define decimal
> json:
> {"c1":50.0,"c2":50.0}
> ddl:
> create table(
> c1 varchar,
> c2 decimal
> )with(
> 'format'='json'
> )
> output:
> {"c1":"5E+1","c2":50.0}
> And the following unit tests will produce the following results
> {"double1":50.0,"double2":50.0,"double3":"50.0","float1":20.0,"float2":20.0,"float3":"20.0"}
> java.lang.AssertionError:
> Expected :+I[50.0, 50.0, 50.0, 20.0, 20.0, 20.0]
> Actual :+I[5E+1, 50.0, 50.0, 2E+1, 20.0, 20.0]
> {code:java}
> @Test
> public void testDeserialization() throws Exception {
> double doubleValue = 50.0;
> float floatValue = 20.0f;
> ObjectMapper objectMapper = new ObjectMapper();
> ObjectNode root = objectMapper.createObjectNode();
> root.put("double1", doubleValue);
> root.put("double2", doubleValue);
> root.put("double3", String.valueOf(doubleValue));
> root.put("float1", floatValue);
> root.put("float2", floatValue);
> root.put("float3", String.valueOf(floatValue));
> byte[] serializedJson = objectMapper.writeValueAsBytes(root);
> System.out.println(new String(serializedJson));
> DataType dataType =
> ROW(
> FIELD("double1", STRING()),
> FIELD("double2", DECIMAL(10,1)),
> FIELD("double3", DOUBLE()),
> FIELD("float1", STRING()),
> FIELD("float2", DECIMAL(10,1)),
> FIELD("float3", FLOAT()));
> RowType rowType = (RowType) dataType.getLogicalType();
> JsonRowDataDeserializationSchema deserializationSchema =
> new JsonRowDataDeserializationSchema(
> rowType,
> InternalTypeInfo.of(rowType),
> false,
> false,
> TimestampFormat.ISO_8601);
> Row expected = new Row(6);
> expected.setField(0, String.valueOf(doubleValue));
> expected.setField(1, String.valueOf(doubleValue));
> expected.setField(2, doubleValue);
> expected.setField(3, String.valueOf(floatValue));
> expected.setField(4, String.valueOf(floatValue));
> expected.setField(5, floatValue);
> RowData rowData = deserializationSchema.deserialize(serializedJson);
> Row actual = convertToExternal(rowData, dataType);
> assertEquals(expected, actual);
> }
> {code}
> when define the DecimalType
> ObjectMapper will enable USE_BIG_DECIMAL_FOR_FLOATS
> and jsonNode.asText() will call BigDecimal toString method
> {code:java}
> boolean hasDecimalType =
> LogicalTypeChecks.hasNested(rowType, t -> t instanceof
> DecimalType);
> if (hasDecimalType) {
>
> objectMapper.enable(DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
