[
https://issues.apache.org/jira/browse/SYSTEMML-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Boehm closed SYSTEMML-1922.
------------------------------------
Resolution: Fixed
Assignee: Matthias Boehm
Fix Version/s: SystemML 1.0
> Memory management spark broadcasts
> ----------------------------------
>
> Key: SYSTEMML-1922
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1922
> Project: SystemML
> Issue Type: Bug
> Reporter: Matthias Boehm
> Assignee: Matthias Boehm
> Fix For: SystemML 1.0
>
>
> So far, we assumed that spark's broadcast simply blockifies the given
> broadcast object (which creates a deep copy). However, additionally also
> holds a strong reference to the passed object which causes unnecessary memory
> overhead (and thus OOMs) when the passed object is later evicted.
> This task aims to remove this unnecessary memory overhead and fix the used
> memory estimates for broadcast creation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
