[
https://issues.apache.org/jira/browse/SYSTEMML-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Boehm deleted SYSTEMML-2172:
-------------------------------------
> Repartitioning before caching ulta-sparse matrices
> --------------------------------------------------
>
> Key: SYSTEMML-2172
> URL: https://issues.apache.org/jira/browse/SYSTEMML-2172
> Project: SystemML
> Issue Type: Bug
> Reporter: Matthias Boehm
> Priority: Major
>
> Ultra-sparse matrices have dedicated serialized block representation which
> means that their in-memory storage in CSR can be much larger than on disk
> which leads to a blow-up of 128MB partitions to >1GB partitions. Accordingly,
> we should repartition the data before the initial caching in order to remove
> memory pressure and exploit the full parallelism.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
