LamiumAmplexicaule commented on PR #7451:
URL: https://github.com/apache/kyuubi/pull/7451#issuecomment-4950485708
Thank you for testing this end-to-end.
In my test environment, `hive.metastore.uris` is defined in hive-site.xml,
so I didn’t notice this issue.
My understanding is that among the native prefixes covered by
ConfigAudience, only `hive.*` may actually be required by the Spark SQL Engine.
* Spark
* Consumes `hive.*` configurations such as `hive.metastore.uris`, as you
pointed out.
* Trino
* Uses `trino.*` only to pass session properties to the engine and does
not reference other native prefixes.
* Flink
* Although it may use Hive configurations, Flink SQL specifies the
location of hive-site.xml through the catalog, so it does not rely on
forwarding `hive.*` configurations.
* JDBC / Data Agent
* Do not reference native prefixes.
With that in mind, instead of treating all unregistered native
configurations as ANY, what do you think about introducing a
backward-compatible mechanism for selecting the engine audience, such as the
following configuration?
```scala
val ENGINE_SPARK_EXTRA_CONF_PREFIXES: ConfigEntry[Seq[String]] =
buildConf("kyuubi.config.engine.spark.extra.prefixes")
.doc("A comma-separated list of config key prefixes that should be
forwarded to the " +
"Spark Engine in addition to spark.* and kyuubi.engine.spark.*
prefixes. " +
"This is useful for native engine configs like hive.* that Spark also
consumes.")
.audience(SERVER)
.immutable
.version("1.12.0")
.stringConf
.toSequence()
.createWithDefault(Seq("hive."))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
