[ https://issues.apache.org/jira/browse/SYSTEMML-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Boehm deleted SYSTEMML-2172: ------------------------------------- > Repartitioning before caching ulta-sparse matrices > -------------------------------------------------- > > Key: SYSTEMML-2172 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2172 > Project: SystemML > Issue Type: Bug > Reporter: Matthias Boehm > Priority: Major > > Ultra-sparse matrices have dedicated serialized block representation which > means that their in-memory storage in CSR can be much larger than on disk > which leads to a blow-up of 128MB partitions to >1GB partitions. Accordingly, > we should repartition the data before the initial caching in order to remove > memory pressure and exploit the full parallelism. -- This message was sent by Atlassian JIRA (v7.6.3#76005)