LuciferYang edited a comment on pull request #29598: URL: https://github.com/apache/spark/pull/29598#issuecomment-688867694
@cloud-fan @Ngone51 I think the reason for this problem is https://github.com/apache/spark/blob/e7d9a245656655e7bb1df3e04df30eb3cc9e23ad/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/AttributeSet.scala#L106-L113 The `--` method of `LinkedHashSet` is different between Scala 2.12 and Scala 2.13. in Scala 2.12 it use implementation of `SetLike.scala` as follow: ``` override def --(xs: GenTraversableOnce[A]): This = clone() --= xs.seq ``` in Scala 2.13 it use implementation of `Set.scala` as follow: ``` def -- (that: IterableOnce[A]): C = fromSpecific(coll.toSet.removedAll(that)) ``` From the above code we can found that after `baseSet -- otherSet.baseSet`, the result order changed . Maybe we can use `baseSet.diff(otherSet.baseSet)` instead of `baseSet -- otherSet.baseSet`, but I need to confirm the feasibility. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
