Ganesha S created SPARK-60805:
---------------------------------
Summary: CREATE FUNCTION rejects function bodies that call system-
or catalog-qualified functions
Key: SPARK-60805
URL: https://issues.apache.org/jira/browse/SPARK-60805
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.4.0
Reporter: Ganesha S
CREATE FUNCTION rejects a function body that calls a system- or
catalog-qualified function, although the same call works in a query. Before
analysing the body, the cyclic-reference check looks up every called name as a
persistent function: builtin. and session. are treated as schema names in
spark_catalog, and names with more than two parts are rejected. The
temporary-function scan has the same problem.
*Steps to reproduce (default settings; verified on master):*
{code:java}
-- The same calls work in a query:
SELECT builtin.abs(-1), system.builtin.abs(-1); -- 1, 1
CREATE FUNCTION g() RETURNS INT RETURN 5;
SELECT spark_catalog.default.g(); -- 5
-- ...but fail in a function body:
CREATE FUNCTION f1() RETURNS INT RETURN builtin.abs(-1); --
SCHEMA_NOT_FOUND
CREATE FUNCTION f2() RETURNS INT RETURN system.builtin.abs(-1); --
IDENTIFIER_TOO_MANY_NAME_PARTS
CREATE FUNCTION f5() RETURNS INT RETURN spark_catalog.default.g(); --
IDENTIFIER_TOO_MANY_NAME_PARTS
-- A session-qualified temporary function gets the wrong error:
CREATE TEMPORARY FUNCTION temp_fn() RETURNS INT RETURN 1;
CREATE FUNCTION f3() RETURNS INT RETURN session.temp_fn(); --
SCHEMA_NOT_FOUND
CREATE FUNCTION f4() RETURNS INT RETURN system.session.temp_fn(); --
IDENTIFIER_TOO_MANY_NAME_PARTS
-- Even a temporary function cannot call one this way:
CREATE TEMPORARY FUNCTION t2() RETURNS INT RETURN system.session.temp_fn(); --
IDENTIFIER_TOO_MANY_NAME_PARTS
Expected behavior:
- f1, f2 and f5 are created and return the same results as the query.
- f3 and f4 fail with INVALID_TEMP_OBJ_REFERENCE, because a permanent function
cannot
reference a temporary function.
- t2 is created.
- Cycles through qualified names are still detected (e.g. CREATE OR REPLACE
TEMPORARY FUNCTION
temp_fn() RETURNS INT RETURN session.temp_fn() fails with
CYCLIC_FUNCTION_REFERENCE).
{code}
*Proposed fix:* in CREATE FUNCTION's cyclic-reference check and
temporary-function scan, resolve builtin. and session. names (optionally
system.-prefixed) the way function resolution does, including the legacy
spark.sql.legacy.persistentCatalogFirst behavior. Look up three-part
spark_catalog names directly. Leave functions from other catalogs to analysis.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
