pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy
default values of fields if not present when rewriting incoming record with new
schema
URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r396992378
##########
File path:
hudi-common/src/test/java/org/apache/hudi/common/util/TestHoodieAvroUtils.java
##########
@@ -57,4 +60,16 @@ public void testPropsPresent() {
}
Assert.assertTrue("column pii_col doesn't show up", piiPresent);
}
+
+ @Test
+ public void testDefaultValue() {
+ GenericRecord rec = new GenericData.Record(new
Schema.Parser().parse(EXAMPLE_SCHEMA));
+ rec.put("_row_key", "key1");
+ rec.put("non_pii_col", "val1");
+ rec.put("pii_col", "val2");
+ rec.put("timestamp", 3.5);
Review comment:
No its not that the original record has default values as null. Its just
while getting the values from the record, default values are not considered.
Please have a look at this function from avro-1.8.2 library -
@Override public Object get(String key) {
Field field = schema.getField(key);
if (field == null) return null;
return values[field.pos()];
}
Ideally the above function should return field.defaultVal() in case
values[field.pos()] is null, but that is not the case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
