alexeykudinkin commented on code in PR #6657:
URL: https://github.com/apache/hudi/pull/6657#discussion_r980500200
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/TestWriteCopyOnWrite.java:
##########
@@ -188,7 +188,7 @@ public void testUpsertWithDelete() throws Exception {
@Test
public void testInsertWithMiniBatches() throws Exception {
// reset the config option
- conf.setDouble(FlinkOptions.WRITE_BATCH_SIZE, 0.0008); // 839 bytes batch
size
+ conf.setDouble(FlinkOptions.WRITE_BATCH_SIZE, 0.00008); // 839 bytes batch
size
Review Comment:
Just for my own education: why did we need to change this?
##########
hudi-common/src/main/java/org/apache/hudi/common/util/ObjectSizeCalculator.java:
##########
@@ -127,6 +129,16 @@ public synchronized long calculateObjectSize(Object obj) {
}
obj = pending.removeFirst();
}
+ } catch (IllegalAccessException | SecurityException e) {
+ // JDK versions 16 or later enforce strong encapsulation and do now
allow to invoke `setAccessible` on a field,
+ // especially when the `isAccessible` is false. More details in JEP 403.
While integrating Hudi with other
+ // software packages that compile against JDK 16 or later (e.g. Trino),
the IllegalAccessException will be thrown.
+ // In that case, we use Java Object Layout (JOL) to estimate the object
size.
+ //
+ // NOTE: We cannot get the object size base on the amount of byte
serialized because there is no guarantee
+ // that the incoming object is serializable. We could have used
Java's Instrumentation API but it
+ // needs an instrumentation agent that can be hooked to the JVM.
In lieu of that, we are using JOL.
+ return ClassLayout.parseClass(obj.getClass()).instanceSize();
Review Comment:
@codope is there a reason we wouldn't want to use JOL to replace this impl?
##########
hudi-common/src/main/java/org/apache/hudi/common/util/ObjectSizeCalculator.java:
##########
@@ -127,6 +129,16 @@ public synchronized long calculateObjectSize(Object obj) {
}
obj = pending.removeFirst();
}
+ } catch (IllegalAccessException | SecurityException e) {
+ // JDK versions 16 or later enforce strong encapsulation and do now
allow to invoke `setAccessible` on a field,
Review Comment:
Typo: "not"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
