[systemds] branch master updated: [MINOR] Fix RDD/broadcast dependency management frame-frame spark ops

[mboehm7]

This is an automated email from the ASF dual-hosted git repository.

mboehm7 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/systemds.git


The following commit(s) were added to refs/heads/master by this push:
     new a2ae7cc  [MINOR] Fix RDD/broadcast dependency management frame-frame 
spark ops
a2ae7cc is described below

commit a2ae7ccd716f844369d01f02c28de44ea45e74f5
Author: Matthias Boehm <mboe...@gmail.com>
AuthorDate: Mon Aug 10 22:55:29 2020 +0200

    [MINOR] Fix RDD/broadcast dependency management frame-frame spark ops
    
    This patch fixes a bad merge of #1009 where I corrupted the working
    dependency management for dropInvalidType. The issue was that we handled
    the broadcast like any SystemDS spark broadcast, but this operation
    bypasses this mechanisms and thus, dependency management and reuse is
    not in the hands of SystemDS.
---
 .../runtime/instructions/spark/BinaryFrameFrameSPInstruction.java     | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git 
a/src/main/java/org/apache/sysds/runtime/instructions/spark/BinaryFrameFrameSPInstruction.java
 
b/src/main/java/org/apache/sysds/runtime/instructions/spark/BinaryFrameFrameSPInstruction.java
index 021ac84..82ca398 100644
--- 
a/src/main/java/org/apache/sysds/runtime/instructions/spark/BinaryFrameFrameSPInstruction.java
+++ 
b/src/main/java/org/apache/sysds/runtime/instructions/spark/BinaryFrameFrameSPInstruction.java
@@ -80,9 +80,7 @@ public class BinaryFrameFrameSPInstruction extends 
BinarySPInstruction {
                //set output RDD and maintain dependencies
                sec.setRDDHandleForVariable(output.getName(), out);
                sec.addLineageRDD(output.getName(), input1.getName());
-               if( getOpcode().equals("dropInvalidType") )
-                       sec.addLineageBroadcast(output.getName(), 
input2.getName());
-               else
+               if( !getOpcode().equals("dropInvalidType") )
                        sec.addLineageRDD(output.getName(), input2.getName());
        }