[
https://issues.apache.org/jira/browse/SYSTEMML-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Boehm updated SYSTEMML-996:
------------------------------------
Description:
The implementation of our spark frame right indexing instruction unnecessarily
tries to create aligned partial blocks, which causes unnecessary shuffle and
merge issues (on merging these partial blocks) as frame can have variable block
sizes. We should avoid merge/shuffle altogether and simply slice out the
relevant sub blocks.
cc [[email protected]] [~acs_s]
was:cc [[email protected]] [~acs_s]
> Unnecessary shuffle / block mismatch in frame right indexing
> ------------------------------------------------------------
>
> Key: SYSTEMML-996
> URL: https://issues.apache.org/jira/browse/SYSTEMML-996
> Project: SystemML
> Issue Type: Bug
> Affects Versions: SystemML 0.11
> Reporter: Matthias Boehm
> Assignee: Matthias Boehm
>
> The implementation of our spark frame right indexing instruction
> unnecessarily tries to create aligned partial blocks, which causes
> unnecessary shuffle and merge issues (on merging these partial blocks) as
> frame can have variable block sizes. We should avoid merge/shuffle altogether
> and simply slice out the relevant sub blocks.
> cc [[email protected]] [~acs_s]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
