Fei Hu reassigned SYSTEMML-1831:

    Assignee: Fei Hu

> Improve the efficiency of matrix subsetting
> -------------------------------------------
>                 Key: SYSTEMML-1831
>                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1831
>             Project: SystemML
>          Issue Type: Improvement
>            Reporter: Fei Hu
>            Assignee: Fei Hu
> For the {{rangeReIndex}} operation, it needs to read the whole input matrix 
> into memory first and do the subsetting in the memory. It is not efficient 
> because it needs to read a lot of unnecessary data, and even out of memory if 
> the size of input matrix exceeds the available memory. 
> The plan here is to read the keys in the Hadoop sequence file, and identify 
> the keys overlapped with the input range index. Then the values specified by 
> the identified keys will be read.

This message was sent by Atlassian JIRA

Reply via email to