[
https://issues.apache.org/jira/browse/SPARK-59970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anshul Baliga updated SPARK-59970:
----------------------------------
Description:
A persistent SQL function whose body reads a temporary variable only inside an
IDENTIFIER clause is created successfully instead of being rejected with
INVALID_TEMP_OBJ_REFERENCE, and then fails when it is called from another
session (or after the variable is dropped), because the variable no longer
exists there.
{code:java}
DECLARE OR REPLACE VARIABLE col STRING DEFAULT 'a';{code}
{code:java}
CREATE FUNCTION f(x INT) RETURNS INT RETURN (SELECT IDENTIFIER(col) FROM VALUES
(1) AS T(a));
-- succeeds; in a new session:{code}
{code:java}
SELECT f(1);
-- [NOT_A_CONSTANT_STRING.NOT_CONSTANT] The expression col used for the routine
or clause
-- IDENTIFIER must be a constant STRING which is NOT NULL.{code}
A direct reference (RETURN x + col) is rejected as expected.
Cause: CreateSQLFunctionCommand analyzes the body with Analyzer.execute, which
opens a fresh AnalysisContext, so the names that ResolveIdentifierClause
records in AnalysisContext.referredTempVariableNamesUnderIdentifier (added by
SPARK-59323 for views) are discarded with that context.
verifyTemporaryObjectsNotExists then only runs
ViewHelper.collectTemporaryVariables on the analyzed plan, where the
VariableReference has already been replaced by the evaluated name. The same
applies to the referredTempVariableNames property written for a temporary
function.
SPARK-59323 fixed this for CREATE VIEW, ALTER VIEW, CACHE TABLE AS SELECT and
metric views; its review noted the SQL function path as pre-existing and to be
tracked separately.
Fix: analyze the body in a context the command owns, read the accumulator
before leaving it, and union those names into the temporary-variable check for
persistent functions and into the temporary-function properties.
was:
A persistent SQL function whose body reads a temporary variable only inside an
IDENTIFIER clause is created successfully instead of being rejected with
INVALID_TEMP_OBJ_REFERENCE, and then fails when it is called from another
session (or after the variable is dropped), because the variable no longer
exists there.
{code:java}
DECLARE OR REPLACE VARIABLE col STRING DEFAULT 'a'; CREATE FUNCTION f(x INT)
RETURNS INT RETURN (SELECT IDENTIFIER(col) FROM VALUES (1) AS T(a)); --
succeeds; in a new session: SELECT f(1); --
[NOT_A_CONSTANT_STRING.NOT_CONSTANT] The expression col used for the routine or
clause IDENTIFIER must be a constant STRING which is NOT NULL.{code}
A direct reference (RETURN x + col) is rejected as expected.
Cause: CreateSQLFunctionCommand analyzes the body with Analyzer.execute, which
opens a fresh AnalysisContext, so the names that ResolveIdentifierClause
records in AnalysisContext.referredTempVariableNamesUnderIdentifier (added by
SPARK-59323 for views) are discarded with that context.
verifyTemporaryObjectsNotExists then only runs
ViewHelper.collectTemporaryVariables on the analyzed plan, where the
VariableReference has already been replaced by the evaluated name. The same
applies to the referredTempVariableNames property written for a temporary
function.
SPARK-59323 fixed this for CREATE VIEW, ALTER VIEW, CACHE TABLE AS SELECT and
metric views; its review noted the SQL function path as pre-existing and to be
tracked separately.
Fix: analyze the body in a context the command owns, read the accumulator
before leaving it, and union those names into the temporary-variable check for
persistent functions and into the temporary-function properties.
> Persistent SQL function accepts a temporary variable read through an
> IDENTIFIER clause
> --------------------------------------------------------------------------------------
>
> Key: SPARK-59970
> URL: https://issues.apache.org/jira/browse/SPARK-59970
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.1.0, 4.2.0, 4.3.0
> Reporter: Anshul Baliga
> Priority: Major
>
> A persistent SQL function whose body reads a temporary variable only inside
> an IDENTIFIER clause is created successfully instead of being rejected with
> INVALID_TEMP_OBJ_REFERENCE, and then fails when it is called from another
> session (or after the variable is dropped), because the variable no longer
> exists there.
>
> {code:java}
> DECLARE OR REPLACE VARIABLE col STRING DEFAULT 'a';{code}
>
> {code:java}
> CREATE FUNCTION f(x INT) RETURNS INT RETURN (SELECT IDENTIFIER(col) FROM
> VALUES (1) AS T(a));
> -- succeeds; in a new session:{code}
>
> {code:java}
> SELECT f(1);
> -- [NOT_A_CONSTANT_STRING.NOT_CONSTANT] The expression col used for the
> routine or clause
> -- IDENTIFIER must be a constant STRING which is NOT NULL.{code}
>
>
> A direct reference (RETURN x + col) is rejected as expected.
>
> Cause: CreateSQLFunctionCommand analyzes the body with Analyzer.execute,
> which opens a fresh AnalysisContext, so the names that
> ResolveIdentifierClause records in
> AnalysisContext.referredTempVariableNamesUnderIdentifier (added by
> SPARK-59323 for views) are discarded with that context.
> verifyTemporaryObjectsNotExists then only runs
> ViewHelper.collectTemporaryVariables on the analyzed plan, where the
> VariableReference has already been replaced by the evaluated name. The same
> applies to the referredTempVariableNames property written for a temporary
> function.
>
> SPARK-59323 fixed this for CREATE VIEW, ALTER VIEW, CACHE TABLE AS SELECT and
> metric views; its review noted the SQL function path as pre-existing and to
> be tracked separately.
>
> Fix: analyze the body in a context the command owns, read the accumulator
> before leaving it, and union those names into the temporary-variable check
> for persistent functions and into the temporary-function properties.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
