kevinjqliu commented on code in PR #15736:
URL: https://github.com/apache/iceberg/pull/15736#discussion_r2978763821
##########
docs/docs/spark-queries.md:
##########
@@ -51,6 +51,14 @@ writing filters that match Iceberg partition transforms.
These functions are ava
[Iceberg catalog](spark-configuration.md#catalog-configuration); they are not
registered in Spark's
built-in catalog.
+!!! note
+ In Spark versions before 4.2.0, `SparkSessionCatalog` does not expose
Iceberg's `system`
+ namespace (see SPARK-54760). Queries such as `SELECT
spark_catalog.system.bucket(16, id)`
Review Comment:
```suggestion
namespace (see
[SPARK-54760](https://issues.apache.org/jira/browse/SPARK-54760)). Queries such
as `SELECT spark_catalog.system.bucket(16, id)`
```
##########
docs/docs/spark-configuration.md:
##########
@@ -112,6 +112,13 @@ Spark's built-in catalog supports existing v1 and v2
tables tracked in a Hive Me
This configuration can use same Hive Metastore for both Iceberg and
non-Iceberg tables.
+`SparkSessionCatalog` is useful when you want `spark_catalog` to work with
both Iceberg and non-Iceberg
+tables in the same metastore. It is not a full replacement for a dedicated
Iceberg catalog, though.
+In Spark versions before 4.2.0, `SparkSessionCatalog` does not expose
Iceberg's `system` namespace
+(see SPARK-54760), so catalog-scoped SQL functions such as `system.bucket`,
`system.days`, and
Review Comment:
```suggestion
(see [SPARK-54760](https://issues.apache.org/jira/browse/SPARK-54760)), so
catalog-scoped SQL functions such as `system.bucket`, `system.days`, and
```
nit: add hyperlink, but https://issues.apache.org/jira/browse/SPARK-54760
doesnt really describe much
##########
docs/docs/spark-queries.md:
##########
@@ -51,6 +51,14 @@ writing filters that match Iceberg partition transforms.
These functions are ava
[Iceberg catalog](spark-configuration.md#catalog-configuration); they are not
registered in Spark's
built-in catalog.
+!!! note
+ In Spark versions before 4.2.0, `SparkSessionCatalog` does not expose
Iceberg's `system`
+ namespace (see SPARK-54760). Queries such as `SELECT
spark_catalog.system.bucket(16, id)`
Review Comment:
hey @pan3793 what do you think of the documentation change here?
Is there a better link to use as reference? Or can we add a more details to
SPARK-54760 so theres context on why it doesnt work with `SparkSessionCatalog`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
