wombatu-kun commented on issue #14424:
URL: https://github.com/apache/iceberg/issues/14424#issuecomment-4562573065
Picking this up. I prototyped the natural code fix in `SparkSessionCatalog`
(parse `default-namespace` in `initialize()`, return it from
`defaultNamespace()`, mirroring what `SparkCatalog` already does). A unit test
confirmed the method then honored the option, but an end-to-end test failed:
```java
spark.conf().set("spark.sql.catalog.spark_catalog.default-namespace",
"my_ns");
spark.sql("CREATE NAMESPACE IF NOT EXISTS my_ns");
spark.sessionState().catalogManager().reset();
spark.sql("USE spark_catalog");
// Expected: ["my_ns"]
// Actual: ["default"]
assertThat(spark.sessionState().catalogManager().currentNamespace())
.containsExactly("my_ns");
```
Tracing why: Spark's `V2SessionCatalog.defaultNamespace()` also hard-codes
`["default"]`, and for `spark_catalog` the current namespace flows through V1
`SessionCatalog.currentDatabase`, which is not influenced by the V2 catalog's
`defaultNamespace()`. The `spark.sql.catalog.spark_catalog.defaultDatabase`
workaround works precisely because it feeds into V1 initialization directly.
Propagating `default-namespace` into V1 from our `initialize()` would mean
issuing side effects on Spark's session catalog, which feels out of scope for a
documentation gap.
Given that, I opened #16591 with a docs-only change pointing users to
`defaultDatabase`. Happy to revisit a code fix if maintainers prefer that path
- would just want guidance on whether V1 side effects from `initialize()` are
acceptable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
