KnightChess commented on code in PR #11637:
URL: https://github.com/apache/hudi/pull/11637#discussion_r1680945257
##########
hudi-common/src/main/java/org/apache/hudi/avro/MercifulJsonConverter.java:
##########
@@ -301,6 +306,14 @@ private static JsonToAvroFieldProcessor
generateFixedTypeHandler() {
return new JsonToAvroFieldProcessor() {
@Override
public Pair<Boolean, Object> convert(Object value, String name, Schema
schema, boolean shouldSanitize, String invalidCharMask) {
+ // the value can be Number
+ if (value instanceof Number) {
+ LogicalTypes.Decimal decimalType = (LogicalTypes.Decimal)
schema.getLogicalType();
+ BigDecimal bigDecimal = new
java.math.BigDecimal(value.toString()).setScale(decimalType.getScale(),
RoundingMode.HALF_UP);
+ GenericFixed genericFixed = DECIMAL_CONVERSION.toFixed(bigDecimal,
schema, schema.getLogicalType());
+ return Pair.of(true, new GenericData.Fixed(schema,
genericFixed.bytes()));
+ }
+
// The ObjectMapper use List to represent FixedType
// eg: "decimal_val": [0, 0, 14, -63, -52] will convert to
ArrayList<Integer>
List<Integer> converval = (List<Integer>) value;
Review Comment:
hi, do you know why this value is a list?
##########
hudi-common/src/test/java/org/apache/hudi/avro/TestMercifulJsonConverter.java:
##########
@@ -99,4 +106,44 @@ public void conversionWithFieldNameAliases() throws
IOException {
Assertions.assertEquals(rec, CONVERTER.convert(json, sanitizedSchema));
}
+
+ @Test
+ public void testConvertNumberToFixed() throws IOException {
+ String testSchemaStr = "{\"type\": \"record\",\"name\":
\"test_record\",\"namespace\": \"test_namespace\",\"fields\": "
+ + "[{\"name\": \"decimal_field\",\"type\": [\"null\",{\"type\":
\"fixed\",\"name\": \"fixed\",\"namespace\":
\"test_namespace.decimal_field\",\"size\": 9,"
+ + " \"logicalType\": \"decimal\",\"precision\": 20,\"scale\":
0}],\"default\": null},"
+ + "{\"name\": \"decimal2_field\",\"type\": [\"null\",{\"type\":
\"fixed\",\"name\": \"fixed\",\"namespace\":
\"test_namespace.decimal2_field\",\"size\": 9,"
+ + " \"logicalType\": \"decimal\",\"precision\": 20,\"scale\":
2}],\"default\": null},"
+ + "{\"name\": \"decimal3_field\",\"type\": [\"null\",{\"type\":
\"fixed\",\"name\": \"fixed\",\"namespace\":
\"test_namespace.decimal3_field\",\"size\": 9,"
+ + " \"logicalType\": \"decimal\",\"precision\": 20,\"scale\":
2}],\"default\": null},"
+ + "{\"name\": \"int_field\",\"type\": [\"null\",\"int\"],\"default\":
null},"
+ + "{\"name\": \"long_field\",\"type\":
[\"null\",\"long\"],\"default\": null},"
+ + "{\"name\": \"string_field\",\"type\":
[\"null\",\"string\"],\"default\": null}]}";
+ Schema schema = Schema.parse(testSchemaStr);
+
+ String testValueStr = "{\n"
+ + " \"decimal_field\": 1720623716,\n"
Review Comment:
the schema type is `Fixed`, why use `Number` value for test?
##########
hudi-common/src/main/java/org/apache/hudi/avro/MercifulJsonConverter.java:
##########
@@ -301,6 +306,14 @@ private static JsonToAvroFieldProcessor
generateFixedTypeHandler() {
return new JsonToAvroFieldProcessor() {
@Override
public Pair<Boolean, Object> convert(Object value, String name, Schema
schema, boolean shouldSanitize, String invalidCharMask) {
+ // the value can be Number
+ if (value instanceof Number) {
Review Comment:
Why number type will use Fixed type schema?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
