ulysses-you commented on code in PR #59535:
URL: https://github.com/apache/spark/pull/59535#discussion_r4333207096
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala:
##########
@@ -1182,25 +1214,50 @@ object KeyedPartitioning {
KeyLayout(comparablePartitionKeys, factory.dataTypes, isGrouped,
isCollapsed = false))
}
+ /**
+ * The unknown-keys state a layout built from several members carries: the
marked members have to
+ * agree, since one layout cannot carry two descriptions of where an
undeclared row sits, and a
+ * disagreement leaves `ROUTING_LOST`. An unmarked member takes no part.
Asked of the members'
+ * states, so a new origin joins without this changing.
+ */
+ def combine(states: Seq[Option[UnknownKeysOrigin]]):
Option[UnknownKeysOrigin] =
+ states.flatten.distinct match {
+ case Seq() => None
+ case Seq(one) => Some(one)
+ case _ => Some(ROUTING_LOST)
+ }
+
/**
* Concatenates partitionings that agree on their expressions, which is what
a `UnionExec` does to
* its children's partitions. The result reports one key per output
partition, so its keys are the
* children's keys in child order.
*
* Keys repeating across children is not a collapse. Only a child's own
collapse carries over,
* since such a key still stands for several finer-grained ones in the
concatenation.
+ *
+ * One marked child marks the result, and the concatenation reports the
weakest routing of its
Review Comment:
addressed in
[1b4b96c](https://github.com/apache/spark/pull/59535/commits/1b4b96cb80fab6c387410a5ca2055a52da2cebb5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
