codope commented on code in PR #6657:
URL: https://github.com/apache/hudi/pull/6657#discussion_r981255335
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java:
##########
@@ -188,7 +188,7 @@ public void testUpsertWithDelete() throws Exception {
@Test
public void testInsertWithMiniBatches() throws Exception {
// reset the config option
- conf.setDouble(FlinkOptions.WRITE_BATCH_SIZE, 0.0008); // 839 bytes batch
size
+ conf.setDouble(FlinkOptions.WRITE_BATCH_SIZE, 0.00008); // 839 bytes batch
size
Review Comment:
Hey, this is a good point! Previously, I thought the object sizes were being
overestimated. But, then I actually looked into the records and I realized this
JOL CLassLayout is under-estimating. It's giving a shallow size. So, we have
got to use GraphLayout to get the deep size (including the size of objects that
are being referenced from the current object). I'll update the patch soon after
doing some more tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
