GitHub user SemMulder opened a pull request:
https://github.com/apache/spark/pull/9244
[SPARK-11276][CORE] SizeEstimator prevents class unloading
The SizeEstimator keeps a cache of ClassInfos but this cache uses Class
objects as keys.
Which results in strong references to the Class objects. If these classes
are dynamically created
this prevents the corresponding ClassLoader from being GCed. Leading to
PermGen exhaustion.
We use a Map with WeakKeys to prevent this issue.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Site2Mobile/spark
fix-sizeestimator-classunloading
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9244.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9244
----
commit 22a02a22401128cb249417ff38ef8f9d24face32
Author: Sem Mulder <[email protected]>
Date: 2015-10-22T16:26:16Z
[SPARK-11276][CORE] SizeEstimator prevents class unloading
The SizeEstimator keeps a cache of ClassInfos but this cache uses Class
objects as keys.
Which results in strong references to the Class objects. If these classes
are dynamically created
this prevents the corresponding ClassLoader from being GCed. Leading to
PermGen exhaustion.
We use a Map with WeakKeys to prevent this issue.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
