deniskuzZ opened a new pull request, #6883: URL: https://github.com/apache/hive/pull/6883
### What changes were proposed in this pull request? `LlapDecider.adjustAutoParallelism` applies the minimum reducers per executor to the reduce works of a reduce-side merge join (`MergeJoinWork`): its main work and each merged work. ### Why are the changes needed? The floor (`hive.tez.llap.min.reducer.per.executor` x executors) keeps auto-reduce from shrinking a reducer to a few tasks on a large cluster, but it was applied only to a `ReduceWork`, so a merge join vertex was skipped. TPC-DS q51 at 10 TB on 24 LLAP daemons (288 executors): the FULL OUTER merge join ran 13 tasks while the reducers around it ran 253. The merge join's vertex takes its parallelism from the edges of its reduce works, so each of them gets the floor; their edge properties are the ones `GenTezWork` reconnects to the merge join. ### Does this PR introduce _any_ user-facing change? Reduce-side merge joins under LLAP get at least `hive.tez.llap.min.reducer.per.executor` x executors tasks, as other auto-reduced reducers do. ### How was this patch tested? `TestLlapDecider`: on a 288-executor cluster, the main and merged reduce works of a merge join and their edges get the floor of 96, as a lone reduce work does; it fails without the change (13). 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
