Thanks Zhentao for the update! Regards Venkata krishnan
On Mon, Aug 10, 2026 at 12:22 AM Zhentao Shuai <[email protected]> wrote: > Hi Venkata, > > Thank you for reporting this concurrency bug and filing the fix. > > PR #3776 has been reviewed, merged to both main and branch-0.7, > and is included in the new release candidate v0.7.0-rc1. > > Regarding PR #3745 (scope reducer metadata by partition range), > we plan to include it in the next 0.7.x release. Please continue > driving the review forward — it's a solid improvement and we'd > love to see it land soon. > > I'll send out the [VOTE] for 0.7.0-rc1 shortly with the updated > artifacts. > > Regards, > Zhentao Shuai > > > On 2026/08/05 18:32:50 Venkatakrishnan Sowrirajan wrote: > > (Cross posting this in this thread as well) > > > > Hi Celeborn devs, > > > > We found a concurrency bug with GetReducerFileGroup in ShuffleClientImpl > in > > our internal workloads. The blocking RPC runs inside > > ConcurrentHashMap.compute, so it holds the per-key bin lock across the > > whole round-trip and queues behind every reduce task of the shuffle. At > > scale this wedged a stage for over 20 minutes with no fetch activity. I > > filed https://github.com/apache/celeborn/pull/3776 to fix it (isolated > > change, has a regression test). > > > > After filing it, I noticed @Chao Sun <[email protected]> already had > > https://github.com/apache/celeborn/pull/3687 open for the same issue, > and > > later superseded it with https://github.com/apache/celeborn/pull/3745, > > which scopes the reduceFileGroups fetch to the reader's partition range > > instead of shipping the full map to every executor. > > > > Both are solid improvements: the concurrency fix and the fetch > > optimization. I would like to get both into 0.7.0. #3776 is small and > ready > > for review; happy to rebase it onto the 0.7.0 branch. Would appreciate > > thoughts on whether #3745 can make 0.7.0 as well. > > > > Regards > > Venkata krishnan > > > > > > On Wed, Aug 5, 2026 at 8:34 AM kerwin zhang <[email protected]> > > wrote: > > > > > +1 (binding) > > > I checked > > > - Download links, checksums and PGP signatures are valid. > > > - Successfully built the binary from the source on MacOs with Command: > > > build/make-distribution.sh --sbt-enabled --release > > > > > > Regards, > > > Kerwin Zhang > > > > > > Zhentao Shuai <[email protected]> 于2026年8月3日周一 11:06写道： > > > > > > > > Hello, Celeborn community, > > > > > > > > This is a call for a vote to release Apache Celeborn > > > > 0.7.0-rc0 > > > > > > > > The git tag to be voted upon: > > > > https://github.com/apache/celeborn/releases/tag/v0.7.0-rc0 > > > > > > > > Source and binary artifacts can be found at: > > > > https://dist.apache.org/repos/dist/dev/celeborn/v0.7.0-rc0/ > > > > > > > > The git commit hash: > > > > d6005dbe17d57ba3790435b2dc4976d644772419 > > > > > > > > The full changelog since the previous release (0.6.3): > > > > https://github.com/apache/celeborn/compare/v0.6.3...v0.7.0-rc0 > > > > > > > > The staging repo: > > > > > > > > https://repository.apache.org/content/repositories/orgapacheceleborn-1135 > > > > > > > > The fingerprint of the PGP key release artifacts is signed with: > > > > 34D5A9C7996B94968C0C2626208CE878499FD25A > > > > > > > > My public key to verify signatures can be found in: > > > > https://dist.apache.org/repos/dist/release/celeborn/KEYS > > > > > > > > The vote will be open for at least 72 hours or until the necessary > > > > number of votes are reached. > > > > > > > > Please vote accordingly: > > > > > > > > [ ] +1 approve > > > > [ ] +0 no opinion > > > > [ ] -1 disapprove (and the reason) > > > > > > > > Steps to validate the release: > > > > https://www.apache.org/info/verification.html > > > > > > > > - Download links, checksums, and PGP signatures are valid. > > > > - Source code distributions have correct names matching the current > > > release. > > > > - LICENSE and NOTICE files are correct. > > > > - All files have license headers if necessary. > > > > - No unlicensed compiled archives bundled in the source archive. > > > > - The source tarball matches the git tag. > > > > - Build from source is successful. > > > > > > > > Regards, > > > > Zaynt Shuai > > > > > >
