Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21802#discussion_r204087168
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -2086,6 +2087,20 @@ class Analyzer(
}
}
+ /**
+ * Set the seed for random number generation in Shuffle expressions.
+ */
+ object ResolvedShuffleExpressions extends Rule[LogicalPlan] {
+ private lazy val random = new Random()
+
+ override def apply(plan: LogicalPlan): LogicalPlan = plan.transformUp {
+ case p if p.resolved => p
+ case p => p transformExpressionsUp {
+ case Shuffle(child, None) => Shuffle(child,
Some(random.nextLong()))
--- End diff --
Yeah, in `Uuid` we want to make sure the same query plan can return the
same result. It is more deterministic between retries.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
