Ganesha S created SPARK-60806:
---------------------------------
Summary: CREATE FUNCTION should reject only the temporary
functions a permanent function's body actually calls
Key: SPARK-60806
URL: https://issues.apache.org/jira/browse/SPARK-60806
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.4.0
Reporter: Ganesha S
CREATE FUNCTION decides whether a permanent SQL function references a temporary
function by scanning the unresolved body for called names that match a
temporary function, not by checking which function each call actually resolves
to. So it rejects valid functions and accepts invalid ones:
- Over-rejection: a temporary function that only shares a name with a built-in
blocks a body
calling that built-in.
- Under-rejection: the scan only matches plain scalar calls, so a body that
calls a temporary
table function, or a temporary function named through IDENTIFIER(...), is
accepted. The
permanent function then only works in the session that created it.
*Steps to reproduce (default settings; verified on master):*
{code:java}
-- Over-rejection: the built-in owns abs, yet the body is rejected.
CREATE TEMPORARY FUNCTION abs(x INT) RETURNS INT RETURN 42;
SELECT abs(-1); -- 1
(the built-in)
CREATE FUNCTION p_abs() RETURNS INT RETURN abs(-1); --
INVALID_TEMP_OBJ_REFERENCE
DROP TEMPORARY FUNCTION abs;
-- Under-rejection: permanent functions that depend on temporary functions are
accepted.
CREATE TEMPORARY FUNCTION temp_tvf() RETURNS TABLE (a INT) RETURN SELECT 1;
CREATE FUNCTION p_tvf() RETURNS TABLE (a INT) RETURN SELECT * FROM temp_tvf();
-- succeeds
CREATE TEMPORARY FUNCTION temp_fn() RETURNS INT RETURN 1;
CREATE FUNCTION p_id() RETURNS INT RETURN IDENTIFIER('temp_fn')(); --
succeeds
SELECT * FROM p_tvf(); -- 1
(this session only)
SELECT p_id(); -- 1
(this session only)
-- In any other session (e.g. spark.newSession()):
SELECT * FROM p_tvf(); --
UNRESOLVABLE_TABLE_VALUED_FUNCTION
SELECT p_id(); --
UNRESOLVED_ROUTINE
Expected behavior:
- p_abs is created and returns 1, because the built-in owns the call.
- p_tvf and p_id fail with INVALID_TEMP_OBJ_REFERENCE.
{code}
{*}Proposed fix{*}: as CREATE VIEW already does, reject only the temporary
functions that analysis of the body actually binds
(AnalysisContext.referredTempFunctionNames), and report them in sorted order so
the error is deterministic.
*Compatibility:* rejecting p_tvf and p_id is a behavior change for CREATE
FUNCTION statements that succeed today. It needs a migration-guide entry, and
possibly a legacy config to restore the old behavior.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
