[
https://issues.apache.org/jira/browse/SPARK-6030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-6030.
------------------------------
Resolution: Fixed
Fix Version/s: 1.4.0
Issue resolved by pull request 4783
[https://github.com/apache/spark/pull/4783]
> SizeEstimator gives wrong result for Integer object on 64bit JVM with
> UseCompressedOops on
> ------------------------------------------------------------------------------------------
>
> Key: SPARK-6030
> URL: https://issues.apache.org/jira/browse/SPARK-6030
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.0.2, 1.1.1, 1.2.1
> Reporter: Xianjin YE
> Priority: Minor
> Fix For: 1.4.0
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> Integer on 64bit JVM with UseCompressedOops on is 16bytes (verified by a
> related article
> http://www.javaworld.com/article/2077496/testing-debugging/java-tip-130--do-you-know-your-data-size-.html,
> I created a gist for that code
> https://gist.github.com/advancedxy/2ae7c9cc7629f3aeb679), however
> SizeEstimator give 24bytes for Integer.
> SizeEstimator gives the wrong answer because it alignSize on internal
> shellSize. For Integer, there is a parent class called Number, which has zero
> fields. Thus the shellSize for Number is 12bytes but was aligned to 16bytes,
> which resulted the Integer's shellSize to be 20bytes, aligned to 24bytes.
> The right path should be
> 1. Object-> shellSize: 12bytes, realSize: 16bytes
> 2. Number -> shellSize: 12bytes + 0, realSize: 16bytes
> 3. Integer -> shellSize: 12bytes + 4bytes(the int value), realSize: 16bytes
> The fix is rather simple, I will submit a pr later.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
