cloud-fan commented on a change in pull request #35304:
URL: https://github.com/apache/spark/pull/35304#discussion_r791437289
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2006,21 +2006,33 @@ class Analyzer(override val catalogManager:
CatalogManager)
override def apply(plan: LogicalPlan): LogicalPlan = {
val externalFunctionNameSet = new mutable.HashSet[Seq[String]]()
-
plan.resolveExpressionsWithPruning(_.containsAnyPattern(UNRESOLVED_FUNCTION)) {
- case f @ UnresolvedFunction(nameParts, _, _, _, _) =>
- if
(ResolveFunctions.lookupBuiltinOrTempFunction(nameParts).isDefined) {
+ def lookupFunction[T <: TreeNode[_]](
+ f: T,
+ nameParts: Seq[String],
+ lookupBuiltinOrTempFunc: Seq[String] => Option[ExpressionInfo]): T =
{
+ if (lookupBuiltinOrTempFunc(nameParts).isDefined) {
+ f
+ } else {
+ val CatalogAndIdentifier(catalog, ident) =
expandIdentifier(nameParts)
+ val fullName = normalizeFuncName(catalog.name +: ident.namespace :+
ident.name)
+ if (externalFunctionNameSet.contains(fullName)) {
+ f
+ } else if (catalog.asFunctionCatalog.functionExists(ident)) {
Review comment:
this checks both scalar and table functions, shall we differentiate them?
One problem is: only the v1 catalog supports table function, so there is no
v2 API for it. The logic will be a bit complicatd
```
if (isSessionCatalog) {
sessionCatalog.tableFunctionExist(...)
} else {
false
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
