yew1eb opened a new pull request, #3871: URL: https://github.com/apache/celeborn/pull/3871
## What changes were proposed in this pull request? Gate the Celeborn Spark UI extension on the new `celeborn.client.spark.ui.enabled` config so the documented "zero-overhead when off" actually holds. Today `CelebornPlugin` registers `CelebornListener` and attaches the Celeborn tab unconditionally: setting `spark.celeborn.client.spark.ui.enabled=false` never disables anything. And a bare-key `SparkConf.getBoolean/getInt` lookup would not help either, because SparkConf matches keys exactly while user-set values carry the `spark.celeborn.` prefix — so the flag must be resolved through `CelebornConf`. This PR: - adds the `celeborn.client.spark.ui.enabled` entry (`categories: client`, default `true`) with its accessor and the regenerated `docs/configuration/client.md` row; - adds `CelebornUIUtils.celebornConf`, a local `spark.celeborn.* -> CelebornConf` conversion, and reads the flag through the `CelebornConf` entry (never a bare-key lookup). The conversion is local to the `spark-3-ui` module because `SparkUtils.fromSparkConf`'s class init drags in spark-sql's `UnsafeRowSerializer`, which is not on this module's classpath (spark-core only) and is absent in RDD-only applications; - gates `CelebornDriverPlugin.init` (listener registration) and `registerMetrics` (tab attachment) on the flag, together with Spark's own `spark.ui.enabled`. ## Why are the changes needed? The extension could not be turned off, so the listener stayed on the status queue and metrics kept being collected even when the user disabled the UI extension. Fixes CELEBORN-2477. ## Does this PR introduce _any_ user-facing change? Yes: `celeborn.client.spark.ui.enabled=false` now disables the Celeborn Spark UI extension (listener + tab). Default behavior is unchanged (enabled). ## How was this patch tested? - New `uiGatingHonorsSparkCelebornPrefix` case in `CelebornListenerSuite`: default true, `spark.celeborn.client.spark.ui.enabled=false` disables, `spark.ui.enabled=false` disables. - `mvn -Pspark-3.5 -pl client-spark/spark-3-ui -am clean test` passes (incl. the `ConfigurationSuite` golden-file check for the new doc row). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
