ashokkumar-allu commented on PR #18585: URL: https://github.com/apache/hudi/pull/18585#issuecomment-4684189654
Thanks for the review @nsivabalan! Addressed your feedback in the latest push — adopted your suggestion exactly: - **New class `HoodiePreCommitValidatorEngineContext`** extends `HoodieLocalEngineContext` and overrides only `map()`, backed by a classloader-aware `ExecutorService` (fixed thread pool, daemon threads, lazy-initialized via a `PoolHolder` inner class per JLS 12.4.2). This keeps the pre-commit-specific parallelism concern completely isolated from `HoodieLocalEngineContext`. - **`HoodieLocalEngineContext` is clean again** — `mapParallel()`, `PoolHolder`, `detectJavaVersion()`, `initForkJoinPool()`, and `makeWorkerThreadFactory()` are all removed. The class retains its original `stream().parallel()` behavior in `map()` with no behavioral change for any existing caller. - **`SparkValidatorUtils`** now instantiates `new HoodiePreCommitValidatorEngineContext(context.getStorageConf()).map(validators, ...)` at the call site. - **Tests** moved to a new `TestHoodiePreCommitValidatorEngineContext` covering happy-path, exception propagation, and the classloader regression guard. The `ExecutorService` approach works well here — pre-commit validators are independent, non-recursive tasks so there is no benefit from ForkJoin work-stealing, and a plain fixed thread pool with an explicit classloader on each worker is simpler and cleaner. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
