Re: why advisoryPartitionSize <= maxShuffledHashJoinLocalMapThreshold

2023-08-15 Thread XiDuo You
CoalesceShufflePartitions will merge small partitions into bigger ones. Say, if you set maxShuffledHashJoinLocalMapThreshold to 32MB, but the advisoryPartitionSize is 64MB, then the final each reducer partition size will be close to 64MB. It breaks the maxShuffledHashJoinLocalMapThreshold. So we

[ANNOUNCE] Release Apache Kyuubi (Incubating) 1.3.1-incubating

2021-10-25 Thread XiDuo You
Hi Spark Community, The Apache Kyuubi (Incubating) community is pleased to announce that Apache Kyuubi (Incubating) 1.3.1-incubating has been released! Apache Kyuubi (Incubating) is a distributed multi-tenant JDBC server for large-scale data processing and analytics, built on top of Apache