Good catch Dongjoon :) This release candidate fails, but feel free to keep testing for any other potential blockers.
I’ll roll RC4 next week with the older release scripts (but the more modern image since the legacy image didn’t have a good time with the R doc packaging). On Thu, Feb 3, 2022 at 3:53 PM Dongjoon Hyun <dongjoon.h...@gmail.com> wrote: > Unfortunately, -1 for 3.1.3 RC3 due to the packaging issue. > > It seems that the master branch release script didn't work properly for > Hadoop 2 binary distribution, Holden. > > $ curl -s > https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/spark-3.1.3-bin-hadoop2.tgz > | tar tz | grep hadoop-common > spark-3.1.3-bin-hadoop2/jars/hadoop-common-3.2.0.jar > > Apache Spark didn't drop Apache Hadoop 2 based binary distribution yet. > > Dongjoon > > > On Wed, Feb 2, 2022 at 3:38 PM Mridul Muralidharan <mri...@gmail.com> > wrote: > >> Hi, >> >> Minor nit: the tag mentioned under [1] looks like a typo - I used >> "v3.1.3-rc3" for my vote (3.2.1 is mentioned in a couple of places, treat >> them as 3.1.3 instead) >> >> +1 >> Signatures, digests, etc check out fine. >> Checked out tag and build/tested with -Pyarn -Pmesos -Pkubernetes >> >> Regards, >> Mridul >> >> [1] "The tag to be voted on is v3.2.1-rc1" - the commit hash and git url >> are correct. >> >> >> On Wed, Feb 2, 2022 at 9:30 AM Mridul Muralidharan <mri...@gmail.com> >> wrote: >> >>> >>> Thanks Tom ! >>> I missed [1] (or probably forgot) the 3.1 part of the discussion given >>> it centered around 3.2 ... >>> >>> >>> Regards, >>> Mridul >>> >>> [1] https://www.mail-archive.com/dev@spark.apache.org/msg28484.html >>> >>> On Wed, Feb 2, 2022 at 8:55 AM Thomas Graves <tgraves...@gmail.com> >>> wrote: >>> >>>> It was discussed doing all the maintenance lines back at beginning of >>>> December (Dec 6) when we were talking about release 3.2.1. >>>> >>>> Tom >>>> >>>> On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan <mri...@gmail.com> >>>> wrote: >>>> > >>>> > Hi Holden, >>>> > >>>> > Not that I am against releasing 3.1.3 (given the fixes that have >>>> already gone in), but did we discuss releasing it ? I might have missed the >>>> thread ... >>>> > >>>> > Regards, >>>> > Mridul >>>> > >>>> > On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <hol...@pigscanfly.ca> >>>> wrote: >>>> >> >>>> >> Please vote on releasing the following candidate as Apache Spark >>>> version 3.1.3. >>>> >> >>>> >> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and >>>> passes if a majority >>>> >> +1 PMC votes are cast, with a minimum of 3 + 1 votes. >>>> >> >>>> >> [ ] +1 Release this package as Apache Spark 3.1.3 >>>> >> [ ] -1 Do not release this package because ... >>>> >> >>>> >> To learn more about Apache Spark, please see >>>> http://spark.apache.org/ >>>> >> >>>> >> There are currently no open issues targeting 3.1.3 in Spark's JIRA >>>> https://issues.apache.org/jira/browse >>>> >> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in >>>> (Open, Reopened, "In Progress")) >>>> >> at https://s.apache.org/n79dw >>>> >> >>>> >> >>>> >> >>>> >> The tag to be voted on is v3.2.1-rc1 (commit >>>> >> b8c0799a8cef22c56132d94033759c9f82b0cc86): >>>> >> https://github.com/apache/spark/tree/v3.1.3-rc3 >>>> >> >>>> >> The release files, including signatures, digests, etc. can be found >>>> at: >>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/ >>>> >> >>>> >> Signatures used for Spark RCs can be found in this file: >>>> >> https://dist.apache.org/repos/dist/dev/spark/KEYS >>>> >> >>>> >> The staging repository for this release can be found at >>>> >> : >>>> https://repository.apache.org/content/repositories/orgapachespark-1400/ >>>> >> >>>> >> The documentation corresponding to this release can be found at: >>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/ >>>> >> >>>> >> The list of bug fixes going into 3.1.3 can be found at the following >>>> URL: >>>> >> https://s.apache.org/x0q9b >>>> >> >>>> >> This release is using the release script in master as of >>>> ddc77fb906cb3ce1567d277c2d0850104c89ac25 >>>> >> The release docker container was rebuilt since the previous version >>>> didn't have the necessary components to build the R documentation. >>>> >> >>>> >> FAQ >>>> >> >>>> >> >>>> >> ========================= >>>> >> How can I help test this release? >>>> >> ========================= >>>> >> >>>> >> If you are a Spark user, you can help us test this release by taking >>>> >> an existing Spark workload and running on this release candidate, >>>> then >>>> >> reporting any regressions. >>>> >> >>>> >> If you're working in PySpark you can set up a virtual env and install >>>> >> the current RC and see if anything important breaks, in the >>>> Java/Scala >>>> >> you can add the staging repository to your projects resolvers and >>>> test >>>> >> with the RC (make sure to clean up the artifact cache before/after so >>>> >> you don't end up building with an out of date RC going forward). >>>> >> >>>> >> =========================================== >>>> >> What should happen to JIRA tickets still targeting 3.1.3? >>>> >> =========================================== >>>> >> >>>> >> The current list of open tickets targeted at 3.2.1 can be found at: >>>> >> https://issues.apache.org/jira/projects/SPARK and search for "Target >>>> >> Version/s" = 3.1.3 >>>> >> >>>> >> Committers should look at those and triage. Extremely important bug >>>> >> fixes, documentation, and API tweaks that impact compatibility should >>>> >> be worked on immediately. Everything else please retarget to an >>>> >> appropriate release. >>>> >> >>>> >> ================== >>>> >> But my bug isn't fixed? >>>> >> ================== >>>> >> >>>> >> In order to make timely releases, we will typically not hold the >>>> >> release unless the bug in question is a regression from the previous >>>> >> release. That being said, if there is something that is a regression >>>> >> that has not been correctly targeted please ping me or a committer to >>>> >> help target the issue. >>>> >> >>>> >> ================== >>>> >> What happened to RC1 & RC2? >>>> >> ================== >>>> >> >>>> >> When I first went to build RC1 the build process failed due to the >>>> >> lack of the R markdown package in my local rm container. By the time >>>> >> I had time to debug and rebuild there was already another bug fix >>>> commit in >>>> >> branch-3.1 so I decided to skip ahead to RC2 and pick it up directly. >>>> >> When I went to go send the RC2 vote e-mail I noticed a correctness >>>> issue had >>>> >> been fixed in branch-3.1 so I rolled RC3 to contain the correctness >>>> fix. >>>> >> >>>> >> -- >>>> >> Twitter: https://twitter.com/holdenkarau >>>> >> Books (Learning Spark, High Performance Spark, etc.): >>>> https://amzn.to/2MaRAG9 >>>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>> >>> -- Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
