[ https://issues.apache.org/jira/browse/SYSTEMML-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Boehm updated SYSTEMML-2173: ------------------------------------- Issue Type: Sub-task (was: Bug) Parent: SYSTEMML-2061 > Upper bound nnz propagation over spark instructions > --------------------------------------------------- > > Key: SYSTEMML-2173 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2173 > Project: SystemML > Issue Type: Sub-task > Reporter: Matthias Boehm > Priority: Major > > Due to lazy evaluation, many spark instructions propagate nnz only if they > can be exactly inferred from the input meta data. Especially for ultra-sparse > matrices this poses a challenge because intermediates quickly reach a state > where they're assumed to be dense. However, many operations such as right > indexing allow to upper bound the nnz with the nnz in the input, which can be > propagated and exploited in subsequent instructions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)