Hi Mridul, Thanks for the update.
The web module will not be shipped in Celeborn-0.5.0 so that will not be a trouble. The JVMQuakeSuite runs smoothly, as Maven can not run a single test suite, I tried that with SBT build tools. The JVMQuakeSuite looks like depending on the GC activities. The following command is used to verify the JVMQuakeSuite. `./build/sbt testOnly org.apache.celeborn.service.deploy.worker.monitor.JVMQuakeSuite` Do those two issues block the release process? Thanks, Ethan Mridul Muralidharan <mri...@gmail.com> 于2025年1月1日周三 18:35写道： > > Hi Ethan, > > Thanks, that helped ! > I did also find a few other issues when running tests: > > a) lint task in web submodule's pom.xml runs into failures - I disabled > this to make progress. > b) I found JVMQuakeSuite to be flakey - and ended up disabling it locally > as well. > > With these two changes (which impact tests primarily) - in addition to the > version bump you identified, the rest of the build went through. > > Thanks ! > Mridul > > > On Mon, Dec 30, 2024 at 11:54 PM Ethan Feng <ethanf...@apache.org> wrote: > > > Hi Mridul, > > > > I wanted to share the results of my recent test regarding the scenario > > you provided. > > > > Upon investigation, I discovered that the issue was associated with > > the zinc plugin used for incremental Scala compilation. I upgraded the > > scala-maven-plugin from version 4.7.2 to 4.9.2, which resolved the > > problem. > > > > Based on my tests, I suspect that there may be underlying bugs in the > > zinc compilation process. However, it appears that upgrading the > > scala-maven-plugin resolves these issues. > > > > Could you please verify if this solution works in your environment as well? > > > > Thanks, > > Ethan > > > > Mridul Muralidharan <mri...@gmail.com> 于2024年12月30日周一 05:31写道： > > > > > > Hi Ethan, > > > > > > I typically do something like: > > > ARGS="-Pspark-3.1 -Pjdk-11"; ./build/mvn $ARGS clean && ./build/mvn > > > -DskipTests $ARGS package && ./build/mvn $ARGS package > > > > > > I am able to successfully build both branch-0.4 and main, so I will need > > to > > > investigate why branch-0.5 is having issues. > > > Thanks for the pointer about github workflow - let me take a look at why > > it > > > is working there. > > > > > > Note that even in a previous vote thread for 0.5, I had faced issues with > > > service module [1] in the past - which is why I was trying to see if it > > was > > > fixed this time around. > > > > > > > > > Regards, > > > Mridul > > > > > > [1] https://lists.apache.org/thread/4z56v0g7c1n2dphgncg6c7mkr0fnhj7f > > > > > > > > > > > > > > > On Sun, Dec 29, 2024 at 7:17 AM Ethan Feng <ethanf...@apache.org> wrote: > > > > > > > Hi Mridul, > > > > > > > > Thanks for your check. I tried that on my environment, and the > > > > service module tests are smooth, according to the CI result[0] of the > > > > latest commit in branch-0.5. > > > > There might be an issue with your environment. > > > > > > > > > > > > [0] > > > > > > https://github.com/apache/celeborn/commit/5571aec14133e90b7664d3f686d6fd803e9df4f4 > > > > > > > > Thanks, > > > > Ethan Feng > > > > > > > > Mridul Muralidharan <mri...@gmail.com> 于2024年12月29日周日 08:06写道： > > > > > > > > > > Hi, > > > > > > > > > > Do we still have the issue compiling with maven in this branch ? > > Or is > > > > > this an issue with my env perhaps ? > > > > > I see 'service' spinning endless when running tests. > > > > > > > > > > Regards, > > > > > Mridul > > > > > > > > > > > > > > > On Fri, Dec 27, 2024 at 11:41 PM Fei Wang <feiw...@apache.org> > > wrote: > > > > > > > > > > > +1, > > > > > > I checked > > > > > > - the git commit hash is correct > > > > > > - download links are valid > > > > > > - checksums are good > > > > > > ``` > > > > > > shasum --check apache-celeborn-0.5.3-bin.tgz.sha512 > > > > > > shasum --check apache-celeborn-0.5.3-source.tgz.sha512 > > > > > > ``` > > > > > > - signatures are good > > > > > > ``` > > > > > > gpg --verify apache-celeborn-0.5.3-bin.tgz.asc > > > > > > apache-celeborn-0.5.3-bin.tgz > > > > > > gpg --verify apache-celeborn-0.5.3-source.tgz.asc > > > > > > apache-celeborn-0.5.3-source.tgz > > > > > > ``` > > > > > > - LICENSE and NOTICE look good. > > > > > > - Build from source is successful. > > > > > > ``` > > > > > > ./build/make-distribution.sh --release > > > > > > ``` > > > > > > Regards, > > > > > > Fei Wang > > > > > > > > > > > > On 2024/12/27 12:07:16 Ethan Feng wrote: > > > > > > > Hi, Celeborn community, > > > > > > > > > > > > > > This is a call for a vote to release Apache Celeborn > > > > > > > 0.5.3-rc1 > > > > > > > > > > > > > > The git tag to be voted upon: > > > > > > > https://github.com/apache/celeborn/releases/tag/v0.5.3-rc1 > > > > > > > > > > > > > > The git commit hash: > > > > > > > ea8811e3ad06527e1c7014b0b79ffdc3519cf5f2 > > > > > > > > > > > > > > Source and binary artifacts can be found at: > > > > > > > https://dist.apache.org/repos/dist/dev/celeborn/v0.5.3-rc1 > > > > > > > > > > > > > > The staging repo: > > > > > > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/orgapacheceleborn-1094 > > > > > > > > > > > > > > The fingerprint of the PGP key release artifacts is signed with > > the > > > > > > following: > > > > > > > FCF2 0BB2 9C7B EFDF 58F9 98F7 6392 F71F 3735 6FA0 > > > > > > > > > > > > > > My public key to verify signatures can be found in: > > > > > > > https://dist.apache.org/repos/dist/release/celeborn/KEYS > > > > > > > > > > > > > > The vote will be open for at least 72 hours or until the > > necessary > > > > > > > number of votes are reached. > > > > > > > > > > > > > > Please vote accordingly: > > > > > > > > > > > > > > [ ] +1 approve > > > > > > > [ ] +0 no opinion > > > > > > > [ ] -1 disapprove (and the reason) > > > > > > > > > > > > > > Steps to validate the release: > > > > > > > https://www.apache.org/info/verification.html > > > > > > > > > > > > > > * Download links, checksums, and PGP signatures are valid. > > > > > > > * Source code distributions have correct names matching the > > current > > > > > > release. > > > > > > > * LICENSE and NOTICE files are correct. > > > > > > > * All files have license headers if necessary. > > > > > > > * No unlicensed compiled archives bundled in the source archive. > > > > > > > * The source tarball matches the git tag. > > > > > > > * Build from source is successful. > > > > > > > > > > > > > > The following scenarios were tested: > > > > > > > 1. TPC-DS with data replication on, random worker kill, verified > > data > > > > > > > correctness. > > > > > > > 2. TPC-DS with random worker kill, verified data correctness. > > > > > > > 3. TPC-DS with stage rerun, verified data correctness. > > > > > > > > > > > > > > Regards, > > > > > > > Ethan Feng > > > > > > > > > > > > > > > > > > >
