Github user kevinyu98 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20795#discussion_r176276263
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala ---
@@ -175,6 +175,8 @@ private[sql] class HiveSessionCatalog(
super.functionExists(name) || hiveFunctions.contains(name.funcName)
}
+ override def externalFunctionExists(name: FunctionIdentifier): Boolean =
functionExists(name)
--- End diff --
@WeichenXu123 thanks very much for reviewing. I am a little confused. So
HiveSessionCatalog's builtinFunctionExists is essentially same as its parent.
That is the reason i didn't override it in HiveSessionCatalog. However the
logic to lookup an external function is different in HiveSessionCatalog as we
also have to handle the special function "histogram_numeric". Thats why i
choose to override the externalFunctionExists. One clarification is that
builtinFunctionExists solely looks at FunctionRegistry to lookup a function.
1. builtInFunctionExists => Looks up a function in FunctionRegistry (same
for both SessionCatalog and HiveSessionCatalog
2. externalFunctionExists => Looks up an external catalog to find the
function. HiveSessionCatalog has one additional semantics to handle the special
function called histogram_numeric.
3. as you point out, I need to change the override externalFunctionExists
to this:
`override def externalFunctionExists(name: FunctionIdentifier): Boolean = {
super.externalFunctionExists(name) ||
hiveFunctions.contains(name.funcName)`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
