cloud-fan commented on a change in pull request #27935: [SPARK-25121][SQL]
Supports multi-part table names for broadcast hint resolution
URL: https://github.com/apache/spark/pull/27935#discussion_r394402509
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala
##########
@@ -64,31 +64,63 @@ object ResolveHints {
_.toUpperCase(Locale.ROOT)).contains(hintName.toUpperCase(Locale.ROOT))))
}
+ // This method checks if given multi-part identifiers are matched with
each other.
+ // The [[ResolveJoinStrategyHints]] rule is applied before the resolution
batch
+ // in the analyzer and we cannot semantically compare them at this stage.
+ // Therefore, we follow simple rules; the process is independent of a
session catalog
+ // (`currentDb` in [[SessionCatalog]]) and it just compares them literally.
+ //
+ // Specifically, we follow simple three rules below:
+ //
+ // 1. they match if an identifier in a hint only has one part and it is
the same with
+ // a relation name in a query. If a relation has a namespace
(`db1.t`), we just ignore it.
+ // For example, in a query `SELECT /*+ BROADCAST(t) */ * FROM db1.t
JOIN t`,
+ // the broadcast hint will match both tables, `db1.t` and `t`.
+ //
+ // 2. they match if an identifier in a hint has two parts and it is the
same with
Review comment:
What if the identifier has more than 2 parts like `cata.ns1.ns2.tbl` ? How
about we define a simple rule: `If identInHint is a tail of identInQuery`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
