[ https://issues.apache.org/jira/browse/SPARK-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng Lian updated SPARK-5509: ------------------------------ Summary: EqualTo operator doesn't handle binary type properly (was: SQL relational operators doesn't handle binary type properly) > EqualTo operator doesn't handle binary type properly > ---------------------------------------------------- > > Key: SPARK-5509 > URL: https://issues.apache.org/jira/browse/SPARK-5509 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.1.0, 1.1.1, 1.2.0, 1.3.0, 1.2.1 > Reporter: Cheng Lian > > Binary type is mapped to {{Array\[Byte\]}}, which can't be compared with > {{==}} directly. However, {{EqualTo.eval()}} uses plain {{==}} to compare > values. Run the following {{spark-shell}} snippet with Spark 1.2.0 to > reproduce this issue: > {code} > import org.apache.spark.sql.SQLContext > import sc._ > val sqlContext = new SQLContext(sc) > import sqlContext._ > case class KV(key: Int, value: Array[Byte]) > def toBinary(s: String): Array[Byte] = s.toString.getBytes("UTF-8") > registerFunction("toBinary", toBinary _) > parallelize(1 to 1024).map(i => KV(i, > toBinary(i.toString))).registerTempTable("bin") > // OK > sql("select * from bin where value < toBinary('100')").collect() > // Oops, returns nothing > sql("select * from bin where value = toBinary('100')").collect() > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org