[
https://issues.apache.org/jira/browse/HUDI-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921431#comment-16921431
]
Davis Eric Broda edited comment on HUDI-234 at 9/3/19 1:39 PM:
---------------------------------------------------------------
Instrumentation is a shallow check, so it's actually noticing the string as an
object, and giving the basic size it gives any object, with maybe some shallow
consideration of subfields (it sees a char array, and returns the minimum size
an array can have, not considering contained elements). Using instrumentation
to get a more accurate estimate is more complicated than just calling
getObjectSize, requiring the duplication of a lot of the current
ObjectSizeEstimator's logic around recursivly going through an object's
subfields, array elements, etc. Not an elegant solution at all, but I don't
know of any other API that will work on arbitrary JVMs.
I wonder how different various JVMs are in memory handling. If they aren't that
different, then maybe we could just take the value a given object would have in
HotSpot JVM, and multiply by some safety factor (1.5? or 2?) to get something a
bit larger.
was (Author: davisbroda):
Instrumentation is a shallow check, so it's actually noticing the string as an
object, and giving the basic size it gives any object. Using instrumentation to
get a more accurate estimate is more complicated than just calling
getObjectSize, requiring the duplication of a lot of the current
ObjectSizeEstimator's logic around recursivly going through an object's
subfields, array elements, etc. Not an elegant solution at all, but I don't
know of any other API that will work on arbitrary JVMs.
I wonder how different various JVMs are in memory handling. If they aren't that
different, then maybe we could just take the value a given object would have in
HotSpot JVM, and multiply by some safety factor (1.5? or 2?) to get something a
bit larger.
> Graceful degradation of ObjectSizeCalculator for non hotspot jvms
> -----------------------------------------------------------------
>
> Key: HUDI-234
> URL: https://issues.apache.org/jira/browse/HUDI-234
> Project: Apache Hudi (incubating)
> Issue Type: Bug
> Components: Write Client
> Affects Versions: 0.5.0
> Reporter: Vinoth Chandar
> Priority: Major
>
> https://github.com/apache/incubator-hudi/issues/860 bug report
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
