[ https://issues.apache.org/jira/browse/SPARK-13886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198763#comment-15198763 ]
MahmoudHanafy edited comment on SPARK-13886 at 3/17/16 5:24 AM: ---------------------------------------------------------------- I think List extends Seq !! In this case, How can you differentiate between: 1- ArrayType(ByteType) => Seq[Byte] 2- BinaryType => as you said should be: List[Byte] was (Author: mahmoud.hanafy): I think List extends Seq !! In this case, How can you differentiate between: 1- ArrayType(ByteType) => Seq[Byte] 2- BinaryType => Array[Byte] > ArrayType of BinaryType not supported in Row.equals method > ----------------------------------------------------------- > > Key: SPARK-13886 > URL: https://issues.apache.org/jira/browse/SPARK-13886 > Project: Spark > Issue Type: Bug > Components: SQL > Reporter: MahmoudHanafy > Priority: Minor > > There are multiple types that are supoprted by Spark SQL, One of them is > ArrayType(Seq) which can be of any element type > So it can be BinaryType(Array\[Byte\]) > In equals method in Row class, there is no handling for ArrayType of > BinaryType. > So for example: > {code:xml} > val a = Row( Seq( Array(1.toByte) ) ) > val b = Row( Seq( Array(1.toByte) ) ) > a.equals(b) // this will return false > {code} > Also, this doesn't work for MapType of BinaryType. > {code:xml} > val a = Row( Map(1 -> Array(1.toByte) ) ) > val b = Row( Map(1 -> Array(1.toByte) ) ) > a.equals(b) // this will return false > {code} > Question1: Can the key in MapType be of BinaryType ? > Question2: Isn't there another way to handle BinaryType by using scala type > instead of Array ? > I want to contribute by fixing this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org