j1wonpark opened a new issue, #16625:
URL: https://github.com/apache/iceberg/issues/16625
### Apache Iceberg version
1.11.0 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
**Summary**
On Spark 4.1, when the Iceberg SQL extensions are enabled, parameterized
queries fail with `UNBOUND_SQL_PARAMETER`. Any parameterized statement
submitted via PySpark's `spark.sql(sqlText, args=...)` (both classic PySpark
and Spark Connect) leaves its positional/named parameter markers unbound. The
Iceberg extensions parser doesn't forward the parameter context to Spark;
details and fix are in the linked PR.
**How to reproduce**
Start Spark 4.1 with the Iceberg SQL extensions enabled, then run a
parameterized query from PySpark:
```python
from pyspark.sql import SparkSession
spark = (
SparkSession.builder
.config(
"spark.sql.extensions",
"org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions",
)
.getOrCreate()
)
# No catalog or table is needed — enabling the extension alone triggers the
bug,
# since the parser wrapper intercepts all SQL.
# positional marker
spark.sql("SELECT ? AS id", args=[42]).show()
# named marker
spark.sql("SELECT :id AS id", args={"id": 42}).show()
```
**Actual result**
Both calls raise `AnalysisException`. The unresolved plan still contains
`posparameter(7)` / `namedparameter(id)`, confirming the parameter context was
dropped during parsing (`id#N` is a session-dependent attribute id):
```
# positional — spark.sql("SELECT ? AS id", args=[42])
[UNBOUND_SQL_PARAMETER] Found the unbound parameter: _7. Please, fix `args`
and provide a mapping of the parameter to either a SQL literal or collection
constructor functions such as `map()`, `array()`, `struct()`. SQLSTATE: 42P02;
line 1 pos 7;
'Project [posparameter(7) AS id#N]
+- OneRowRelation
```
```
# named — spark.sql("SELECT :id AS id", args={"id": 42})
[UNBOUND_SQL_PARAMETER] Found the unbound parameter: id. Please, fix `args`
and provide a mapping of the parameter to either a SQL literal or collection
constructor functions such as `map()`, `array()`, `struct()`. SQLSTATE: 42P02;
line 1 pos 7;
'Project [namedparameter(id) AS id#N]
+- OneRowRelation
```
The identical queries bind correctly when the Iceberg extensions are **not**
installed, which pins the wrapper parser as the regression point.
**Expected result**
Parameter markers are bound and the query returns the supplied arguments
(`42`), exactly as without the Iceberg extensions.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
