Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/10209#discussion_r47309024
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala
---
@@ -122,11 +122,22 @@ case class Except(left: LogicalPlan, right:
LogicalPlan) extends SetOperation(le
override def output: Seq[Attribute] = left.output
}
+object Join {
+ def apply(
+ left: LogicalPlan,
+ right: LogicalPlan,
+ joinType: JoinType,
+ condition: Option[Expression]): Join = {
+ Join(left, right, joinType, condition, None)
+ }
+}
+
case class Join(
left: LogicalPlan,
right: LogicalPlan,
joinType: JoinType,
- condition: Option[Expression]) extends BinaryNode {
+ condition: Option[Expression],
+ generatedExpressions: Option[EquivalentExpressions]) extends BinaryNode {
--- End diff --
Literal(5) would work for equivalence but we want to track more than
equality. If it was t1.key join t2.key where t1.key = t2.key and t1.key > 5,
we'd similarly want to add t2.key > 5.
Are you suggesting we don't change the operator and walk the tree bottom up
to collect these constraints? This seems extremely expensive to do.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
