Xianjin YE created SPARK-6030:
---------------------------------
Summary: SizeEstimator gives wrong result for Integer object on
64bit JVM with UseCompressedOops on
Key: SPARK-6030
URL: https://issues.apache.org/jira/browse/SPARK-6030
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.2.1, 1.1.1, 1.0.2
Reporter: Xianjin YE
Priority: Minor
Fix For: 1.3.1
Integer on 64bit JVM with UseCompressedOops on is 16bytes (verified by a
related article
http://www.javaworld.com/article/2077496/testing-debugging/java-tip-130--do-you-know-your-data-size-.html,
I created a gist for that code
https://gist.github.com/advancedxy/2ae7c9cc7629f3aeb679), however SizeEstimator
give 24bytes for Integer.
SizeEstimator gives the wrong answer because it alignSize on internal
shellSize. For Integer, there is a parent class called Number, which has zero
fields. Thus the shellSize for Number is 12bytes but was aligned to 16bytes,
which resulted the Integer's shellSize to be 20bytes, aligned to 24bytes.
The right path should be
1. Object-> shellSize: 12bytes, realSize: 16bytes
2. Number -> shellSize: 12bytes + 0, realSize: 16bytes
3. Integer -> shellSize: 12bytes + 4bytes(the int value), realSize: 16bytes
The fix is rather simple, I will submit a pr later.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
