[ https://issues.apache.org/jira/browse/SYSTEMML-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias Boehm closed SYSTEMML-1884. ------------------------------------ > Unnecessary spark/mr instructions on removeEmpty select > ------------------------------------------------------- > > Key: SYSTEMML-1884 > URL: https://issues.apache.org/jira/browse/SYSTEMML-1884 > Project: SystemML > Issue Type: Bug > Affects Versions: SystemML 0.14 > Reporter: Matthias Boehm > Assignee: Matthias Boehm > Fix For: SystemML 0.15 > > > The builtin function {{removeEmpty}} takes an optional select parameter. If > this parameter is provided, we directly use it instead of explicitly > computing it. However, the compilation of removeEmpty handled this > incorrectly, setting the forced execution type to the initially compiled > execution type of removeEmpty. This is problematic in scenarios with dynamic > recompilation. For example, consider the following scenario: > {code} > C = replace(target=A, pattern=2, replacement=0); > D = removeEmpty(B, margin="rows", select=C); > {code} > If the {{removeEmpty}} is initially compiled to SPARK, this also mistakenly > set the forced execution type of {{replace}} to SPARK. During recompilation, > the {{removeEmpty}} might be compiled to CP but because {{replace}} has a > forced execution type, it will remain in SPARK. > Workaround: The workaround for this issue is to put a {{while(FALSE){}}} just > before the {{removeEmpty}} call (more precisely, C in above example needs to > be computed in a separate HOP DAG). -- This message was sent by Atlassian JIRA (v6.4.14#64029)