LuciferYang commented on pull request #29598: URL: https://github.com/apache/spark/pull/29598#issuecomment-688867694
@cloud-fan @Ngone51 I think the reason for the problem is https://github.com/apache/spark/blob/e7d9a245656655e7bb1df3e04df30eb3cc9e23ad/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/AttributeSet.scala#L106-L113 The `--` of `LinkedHashSet` method is different between Scala 2.12 and Scala 2.13. The impl of Scala 2.12(`SetLike.scala`) as follow: ``` override def --(xs: GenTraversableOnce[A]): This = clone() --= xs.seq ``` The impl of Scala 2.13(`Set.scala`) as follow: ``` def -- (that: IterableOnce[A]): C = fromSpecific(coll.toSet.removedAll(that)) ``` From the above code we can found that after `baseSet -- otherSet.baseSet`, the result order changed . Maybe we can use `baseSet.diff(otherSet.baseSet)` instead of `baseSet -- otherSet.baseSet`, but I need to confirm the feasibility. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
