Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/6751#discussion_r32382644
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala
---
@@ -130,6 +130,14 @@ case class Join(
}
}
+/**
+ * A hint for the optimizer that we should broadcast the `child` if used
in a join operator.
+ */
+case class BroadcastHint(child: LogicalPlan) extends UnaryNode {
--- End diff --
I'm thinking about this more and actually maybe what we want to do is have
this specific node, and a patter that recognizes `canBroadcast`. This pattern
can check either for a small enough size or this hint and we can use that
anywhere we are planning a broadcast operator. The reasoning is that messing
with statistics could have other consequences.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
