Matthias Boehm created SYSTEMML-2172:
----------------------------------------

             Summary: Repartitioning before caching ulta-sparse matrices
                 Key: SYSTEMML-2172
                 URL: https://issues.apache.org/jira/browse/SYSTEMML-2172
             Project: SystemML
          Issue Type: Bug
            Reporter: Matthias Boehm


Ultra-sparse matrices have dedicated serialized block representation which 
means that their in-memory storage in CSR can be much larger than on disk which 
leads to a blow-up of 128MB partitions to >1GB partitions. Accordingly, we 
should repartition the data before the initial caching in order to remove 
memory pressure and exploit the full parallelism.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to