[
https://issues.apache.org/jira/browse/SYSTEMML-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Boehm closed SYSTEMML-1430.
------------------------------------
> Robust memory handling broadcasts
> ---------------------------------
>
> Key: SYSTEMML-1430
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1430
> Project: SystemML
> Issue Type: Sub-task
> Components: APIs, Runtime
> Reporter: Matthias Boehm
> Assignee: Matthias Boehm
> Fix For: SystemML 1.0
>
>
> This task aims to make the memory handling of re-used and temporary broadcast
> variables more robust in order to avoid unnecessary OOMs.
> 1) Explicitly destroy temporary broadcast variables because we would
> otherwise never clean them up and Spark's ContextCleaner seems to be a best
> effort daemon that triggers every 30min or on garbage collection (which might
> be too late if a large object is allocated)
> 2) Keep track of currently softly reachable (re-used) broadcasts in order to
> take their size into account when deciding on guarded collect.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
