Matthias Boehm created SYSTEMML-2163:
----------------------------------------
Summary: Performance large partitioned broadcasts
Key: SYSTEMML-2163
URL: https://issues.apache.org/jira/browse/SYSTEMML-2163
Project: SystemML
Issue Type: Bug
Reporter: Matthias Boehm
Due to Spark's limitations with broadcasts larger than 2G, in SystemML we use
partitioned broadcasts that split a large side input into potentially many
broadcast variables. For historic reasons the meta data is still maintained in
the individual partitioned blocks. However, for many operations this meta data
is accessed on the first partitioned block which leads to potentially
unnecessary broadcast fetches.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
