Re: right outer joins on Datasets

2016-05-24 Thread Zhan
_INT => "-1" >> ... >> } >> >> >> >> -- >> View this message in context: >> http://apache-spark-developers-list.1001551.n3.nabble.com/right-outer-joins-on-Datasets-tp17542p17651.html >> Sent from the Apache Spark Developers L

Re: right outer joins on Datasets

2016-05-24 Thread Koert Kuipers
; def defaultValue(jt: String): String = jt match { > ... > case JAVA_INT => "-1" > ... > } > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/right-outer-joins-on-Datasets-tp17542p17

Re: right outer joins on Datasets

2016-05-24 Thread Zhan Zhang
The reason for "-1" is that the default value for Integer is -1 if the value is null def defaultValue(jt: String): String = jt match { ... case JAVA_INT => "-1" ... } -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabbl

Re: right outer joins on Datasets

2016-05-20 Thread Reynold Xin
I filed https://issues.apache.org/jira/browse/SPARK-15441 On Thu, May 19, 2016 at 8:48 AM, Andres Perez wrote: > Hi all, I'm getting some odd behavior when using the joinWith > functionality for Datasets. Here is a small test case: > > val left = List(("a", 1), ("a", 2),

right outer joins on Datasets

2016-05-19 Thread Andres Perez
Hi all, I'm getting some odd behavior when using the joinWith functionality for Datasets. Here is a small test case: val left = List(("a", 1), ("a", 2), ("b", 3), ("c", 4)).toDS() val right = List(("a", "x"), ("b", "y"), ("d", "z")).toDS() val joined = left.toDF("k", "v").as[(String,