[
https://issues.apache.org/jira/browse/SYSTEMML-2163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Boehm closed SYSTEMML-2163.
------------------------------------
Resolution: Fixed
Assignee: Matthias Boehm
Fix Version/s: SystemML 1.1
> Performance large partitioned broadcasts
> ----------------------------------------
>
> Key: SYSTEMML-2163
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2163
> Project: SystemML
> Issue Type: Bug
> Reporter: Matthias Boehm
> Assignee: Matthias Boehm
> Priority: Major
> Fix For: SystemML 1.1
>
>
> Due to Spark's limitations with broadcasts larger than 2G, in SystemML we use
> partitioned broadcasts that split a large side input into potentially many
> broadcast variables. For historic reasons the meta data is still maintained
> in the individual partitioned blocks. However, for many operations this meta
> data is accessed on the first partitioned block which leads to potentially
> unnecessary broadcast fetches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
