[
https://issues.apache.org/jira/browse/SYSTEMML-1029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Boehm updated SYSTEMML-1029:
-------------------------------------
Description:
The data generator for descriptive statistics shows severe performance issues,
due to update-in-place. In order to avoid buffer pool serialization and
evictions via shallow serialize, we represent sparse matrices which are subject
to update in-place in CSR. This works very well for row major updates but has
severe problems on column-wise updates (due to shifting of values).
There are three potential options:
* (1) Workaround: Change the data generator script from column-wise indexing to
row-wise indexing and transpose after the parfor loop.
* (2) Parfor rewrite: Disable update in-place, if sparse input, no parfor
update in-place, and column-wise access pattern.
* (3) Improved left indexing over CSR: Reallocate and shift the output once.
This is feasible as left indexing (specifically sparse-sparse and dense-sparse)
the only operations which update the CSR representation.
> Perftest: Data generator for descriptive statistics (slowdown due to update
> in-place)
> -------------------------------------------------------------------------------------
>
> Key: SYSTEMML-1029
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1029
> Project: SystemML
> Issue Type: Bug
> Reporter: Matthias Boehm
>
> The data generator for descriptive statistics shows severe performance
> issues, due to update-in-place. In order to avoid buffer pool serialization
> and evictions via shallow serialize, we represent sparse matrices which are
> subject to update in-place in CSR. This works very well for row major updates
> but has severe problems on column-wise updates (due to shifting of values).
> There are three potential options:
> * (1) Workaround: Change the data generator script from column-wise indexing
> to row-wise indexing and transpose after the parfor loop.
> * (2) Parfor rewrite: Disable update in-place, if sparse input, no parfor
> update in-place, and column-wise access pattern.
> * (3) Improved left indexing over CSR: Reallocate and shift the output once.
> This is feasible as left indexing (specifically sparse-sparse and
> dense-sparse) the only operations which update the CSR representation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
