vaibhav-sinha commented on a change in pull request #2923:
URL: https://github.com/apache/hudi/pull/2923#discussion_r646715922
##########
File path:
hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java
##########
@@ -380,12 +380,12 @@ public void testUpsertWithDelete() throws Exception {
@Test
public void testInsertWithMiniBatches() throws Exception {
// reset the config option
- conf.setDouble(FlinkOptions.WRITE_BATCH_SIZE, 0.0006); // 630 bytes batch
size
+ conf.setDouble(FlinkOptions.WRITE_BATCH_SIZE, 0.00075); // 786 bytes batch
size
Review comment:
I think the reason for this is as follows:
For timestamp field, HoodieAvroUtils now returns an instance of
LocalDateTime instead of Long. When Row data is being converted to
HoodieRecordPayload, then `orderingVal` is being fetched using
`HoodieAvroUtils.getNestedFieldVal`. Hence the structure of the payload has
changed and is of larger size than before.
```
public HoodieRecordPayload<?> createPayload(GenericRecord record, boolean
isDelete) throws Exception {
if (shouldCombine) {
ValidationUtils.checkState(preCombineField != null);
Comparable<?> orderingVal = (Comparable<?>)
HoodieAvroUtils.getNestedFieldVal(record,
preCombineField, false);
return (HoodieRecordPayload<?>) constructor.newInstance(
isDelete ? null : record, orderingVal);
} else {
return (HoodieRecordPayload<?>)
this.constructor.newInstance(Option.of(record));
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
