codope opened a new pull request, #6657: URL: https://github.com/apache/hudi/pull/6657
JDK versions 16 or later enforce strong encapsulation and do now allow to invoke `setAccessible` on a field, especially when the `isAccessible` is false. More details in [JEP 403](https://openjdk.org/jeps/403). This PR attempts to address that for `ObjectSizeCalculator`. An immediate use case where this would be beneficial is integrating Hudi with other libraries, e.g. Trino, that are using higher versions of JDK. We evaluated two other approaches: 1. Get the object size base on the amount of byte serialized. However, this runs into error if the incoming object does not implement `Serializable`. 2. Use Java's [Instrumentation](https://docs.oracle.com/en/java/javase/11/docs/api/java.instrument/java/lang/instrument/Instrumentation.html) API. For this, we need to create an instrumentation agent that can be hooked to the JVM. If Hudi was a standalone project, then we could have taken but since Hudi is integrated into other projects as well, we need to invest some time to figure out how to hook our instrumentation agent into those running JVMs. ### Change Logs * If fields are accessible, run the usual code. * Else, use Java Object Layout ([JOL](https://github.com/openjdk/jol)) to get object size. * Add test with different types of objects. ### Impact High. This PR makes changes in `ObjectSizeEstimator`, which is on the hot path. **Risk level: none | low | medium | high** High. Tests have been added to cover various types of objects, both serializable and no serializable. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
