[
https://issues.apache.org/jira/browse/SPARK-17262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748660#comment-15748660
]
George Shuklin commented on SPARK-17262:
----------------------------------------
My original point was slightly different: let suppose a transient field class
is not in spark classpath. In that case there is no point to throw exception
during size estimation as actual value from the driver/client app will be not
serialized and so it will null in executor after deserialization. Later this
value in any case can not be created and assigned to the field in executor as
corresponding class does not exist in classpath. So size is always 0. This was
a real scenario that we faced during development.
> Spark SizeEstimator does not ignore transient fields in java classes when
> calculates class size
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-17262
> URL: https://issues.apache.org/jira/browse/SPARK-17262
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.6.2, 2.0.0
> Reporter: George Shuklin
> Priority: Minor
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
