[
https://issues.apache.org/jira/browse/SPARK-59672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59672:
-----------------------------------
Labels: pull-request-available (was: )
> SQL parameter binding FAILS with UNBOUND_SQL_PARAMETER when a session
> extension injects a parser (4.1 regression)
> -----------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-59672
> URL: https://issues.apache.org/jira/browse/SPARK-59672
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.1.0, 4.1.1, 4.1.2, 4.1.3
> Reporter: Francisco A. B. Sampaio
> Priority: Major
> Labels: pull-request-available
>
> Since 4.1.0, {{spark.sql(query, args)}} fails to bind parameters whenever a
> session extension is configured via {{spark.sql.extensions}}. Both positional
> ({{?}}) and named ({{:name}}) parameters fail with {{UNBOUND_SQL_PARAMETER}}.
> Unsetting {{spark.sql.extensions}} on the same build makes them work. 4.0.x
> is unaffected.
> Reproduced with two unrelated extensions, Delta Lake 4.1.0
> ({{io.delta.sql.DeltaSparkSessionExtension}}) and Apache Iceberg 1.11.0, so
> it is not specific to either project. Both inject a parser through
> {{SparkSessionExtensions.injectParser}}.
> h3. Root cause
> SPARK-53573 moved parameter handling into a parser pre-processor, and
> {{SparkSession.sql}} now calls {{ParserInterface.parsePlanWithParameters}}.
> That method has a default implementation that discards the parameters:
> {code:scala}
> def parsePlanWithParameters(sqlText: String, parameterContext:
> ParameterContext): LogicalPlan = {
> // Default implementation falls back to regular parsePlan
> // Concrete implementations can override this for parameter support
> parsePlan(sqlText)
> }
> {code}
> Only {{SparkSqlParser}} overrides it. A parser injected by an extension
> inherits the default, so the values never reach the parser and the markers
> arrive at the analyzer unbound.
> h3. Reproduction (spark-shell, no Spark Connect involved)
> {code:scala}
> // spark-shell --conf
> spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension
> spark.sql("SELECT ? AS id", ArrayAny).collect()
> // org.apache.spark.sql.AnalysisException: [UNBOUND_SQL_PARAMETER] Found the
> unbound parameter: _7.
> spark.sql("SELECT :id AS id", Map("id" -> 42)).collect()
> // [UNBOUND_SQL_PARAMETER] Found the unbound parameter: id.
> // same build, extension unset: both return [42]
> {code}
> h3. Results
> ||Spark||spark.sql.extensions||Parameter binding||
> |4.1.0 - 4.1.3|Delta 4.1.0|fails|
> |4.1.3|Iceberg 1.11.0|fails|
> |4.1.3|unset|works|
> |4.0.4|Delta 4.0.1|works|
> |4.0.4|Iceberg 1.10.2|works|
> Setting only {{spark.sql.catalog.spark_catalog}} (Delta's catalog, without
> the extension) does not trigger it.
> Also reproducible through Spark Connect, for both the {{SQL}} relation and
> the {{SqlCommand}} command, using {{pos_arguments}}, {{named_arguments}} and
> the deprecated {{pos_args}}.
> h3. Workaround
> Setting {{spark.sql.legacy.parameterSubstitution.constantsOnly=true}}
> restores binding for classic {{spark.sql(query, args)}} and for the Connect
> {{SqlCommand}} path, because the analyzer binds the markers instead. It does
> not help the Connect {{SQL}} relation path, which never applies the legacy
> fallback - that looks like a separate defect and I can file it separately.
> h3. Impact
> Parameterized SQL is the standard defence against SQL injection. On 4.1.x it
> fails outright for anyone using Delta, Iceberg or any other session
> extension, pushing users toward string interpolation.
> A PR is ready: the default implementation binds the parameters during
> analysis so parsers that only override {{parsePlan}} keep working.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
