Matthias Boehm resolved SYSTEMML-1029.
       Resolution: Fixed
         Assignee: Matthias Boehm
    Fix Version/s: SystemML 0.11

> Perftest: Data generator for descriptive statistics (slowdown due to update 
> in-place)
> -------------------------------------------------------------------------------------
>                 Key: SYSTEMML-1029
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1029
>             Project: SystemML
>          Issue Type: Bug
>            Reporter: Matthias Boehm
>            Assignee: Matthias Boehm
>            Priority: Blocker
>             Fix For: SystemML 0.11
> The data generator for descriptive statistics shows severe performance 
> issues, due to update-in-place. In order to avoid buffer pool serialization 
> and evictions via shallow serialize, we represent sparse matrices which are 
> subject to update in-place in CSR. This works very well for row major updates 
> but has severe problems on column-wise updates (due to shifting of values). 
> There are three potential options:
> * (1) Workaround: Change the data generator script from column-wise indexing 
> to row-wise indexing and transpose after the parfor loop.
> * (2) Parfor rewrite: Disable update in-place, if sparse input, no parfor 
> update in-place, and column-wise access pattern.
> * (3) Improved left indexing over CSR: Reallocate and shift the output once. 
> This is feasible as left indexing (specifically sparse-sparse and 
> dense-sparse) are the only operations which update the CSR representation.

This message was sent by Atlassian JIRA

Reply via email to