Shall we use the release scripts of branch 3.1 to release 3.1? On Fri, Feb 4, 2022 at 4:57 AM Holden Karau <hol...@pigscanfly.ca> wrote:
> Good catch Dongjoon :) > > This release candidate fails, but feel free to keep testing for any other > potential blockers. > > I’ll roll RC4 next week with the older release scripts (but the more > modern image since the legacy image didn’t have a good time with the R doc > packaging). > > On Thu, Feb 3, 2022 at 3:53 PM Dongjoon Hyun <dongjoon.h...@gmail.com> > wrote: > >> Unfortunately, -1 for 3.1.3 RC3 due to the packaging issue. >> >> It seems that the master branch release script didn't work properly for >> Hadoop 2 binary distribution, Holden. >> >> $ curl -s >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/spark-3.1.3-bin-hadoop2.tgz >> | tar tz | grep hadoop-common >> spark-3.1.3-bin-hadoop2/jars/hadoop-common-3.2.0.jar >> >> Apache Spark didn't drop Apache Hadoop 2 based binary distribution yet. >> >> Dongjoon >> >> >> On Wed, Feb 2, 2022 at 3:38 PM Mridul Muralidharan <mri...@gmail.com> >> wrote: >> >>> Hi, >>> >>> Minor nit: the tag mentioned under [1] looks like a typo - I used >>> "v3.1.3-rc3" for my vote (3.2.1 is mentioned in a couple of places, treat >>> them as 3.1.3 instead) >>> >>> +1 >>> Signatures, digests, etc check out fine. >>> Checked out tag and build/tested with -Pyarn -Pmesos -Pkubernetes >>> >>> Regards, >>> Mridul >>> >>> [1] "The tag to be voted on is v3.2.1-rc1" - the commit hash and git url >>> are correct. >>> >>> >>> On Wed, Feb 2, 2022 at 9:30 AM Mridul Muralidharan <mri...@gmail.com> >>> wrote: >>> >>>> >>>> Thanks Tom ! >>>> I missed [1] (or probably forgot) the 3.1 part of the discussion given >>>> it centered around 3.2 ... >>>> >>>> >>>> Regards, >>>> Mridul >>>> >>>> [1] https://www.mail-archive.com/dev@spark.apache.org/msg28484.html >>>> >>>> On Wed, Feb 2, 2022 at 8:55 AM Thomas Graves <tgraves...@gmail.com> >>>> wrote: >>>> >>>>> It was discussed doing all the maintenance lines back at beginning of >>>>> December (Dec 6) when we were talking about release 3.2.1. >>>>> >>>>> Tom >>>>> >>>>> On Wed, Feb 2, 2022 at 2:07 AM Mridul Muralidharan <mri...@gmail.com> >>>>> wrote: >>>>> > >>>>> > Hi Holden, >>>>> > >>>>> > Not that I am against releasing 3.1.3 (given the fixes that have >>>>> already gone in), but did we discuss releasing it ? I might have missed >>>>> the >>>>> thread ... >>>>> > >>>>> > Regards, >>>>> > Mridul >>>>> > >>>>> > On Tue, Feb 1, 2022 at 7:12 PM Holden Karau <hol...@pigscanfly.ca> >>>>> wrote: >>>>> >> >>>>> >> Please vote on releasing the following candidate as Apache Spark >>>>> version 3.1.3. >>>>> >> >>>>> >> The vote is open until Feb. 4th at 5 PM PST (1 AM UTC + 1 day) and >>>>> passes if a majority >>>>> >> +1 PMC votes are cast, with a minimum of 3 + 1 votes. >>>>> >> >>>>> >> [ ] +1 Release this package as Apache Spark 3.1.3 >>>>> >> [ ] -1 Do not release this package because ... >>>>> >> >>>>> >> To learn more about Apache Spark, please see >>>>> http://spark.apache.org/ >>>>> >> >>>>> >> There are currently no open issues targeting 3.1.3 in Spark's JIRA >>>>> https://issues.apache.org/jira/browse >>>>> >> (try project = SPARK AND "Target Version/s" = "3.1.3" AND status in >>>>> (Open, Reopened, "In Progress")) >>>>> >> at https://s.apache.org/n79dw >>>>> >> >>>>> >> >>>>> >> >>>>> >> The tag to be voted on is v3.2.1-rc1 (commit >>>>> >> b8c0799a8cef22c56132d94033759c9f82b0cc86): >>>>> >> https://github.com/apache/spark/tree/v3.1.3-rc3 >>>>> >> >>>>> >> The release files, including signatures, digests, etc. can be found >>>>> at: >>>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-bin/ >>>>> >> >>>>> >> Signatures used for Spark RCs can be found in this file: >>>>> >> https://dist.apache.org/repos/dist/dev/spark/KEYS >>>>> >> >>>>> >> The staging repository for this release can be found at >>>>> >> : >>>>> https://repository.apache.org/content/repositories/orgapachespark-1400/ >>>>> >> >>>>> >> The documentation corresponding to this release can be found at: >>>>> >> https://dist.apache.org/repos/dist/dev/spark/v3.1.3-rc3-docs/ >>>>> >> >>>>> >> The list of bug fixes going into 3.1.3 can be found at the >>>>> following URL: >>>>> >> https://s.apache.org/x0q9b >>>>> >> >>>>> >> This release is using the release script in master as of >>>>> ddc77fb906cb3ce1567d277c2d0850104c89ac25 >>>>> >> The release docker container was rebuilt since the previous version >>>>> didn't have the necessary components to build the R documentation. >>>>> >> >>>>> >> FAQ >>>>> >> >>>>> >> >>>>> >> ========================= >>>>> >> How can I help test this release? >>>>> >> ========================= >>>>> >> >>>>> >> If you are a Spark user, you can help us test this release by taking >>>>> >> an existing Spark workload and running on this release candidate, >>>>> then >>>>> >> reporting any regressions. >>>>> >> >>>>> >> If you're working in PySpark you can set up a virtual env and >>>>> install >>>>> >> the current RC and see if anything important breaks, in the >>>>> Java/Scala >>>>> >> you can add the staging repository to your projects resolvers and >>>>> test >>>>> >> with the RC (make sure to clean up the artifact cache before/after >>>>> so >>>>> >> you don't end up building with an out of date RC going forward). >>>>> >> >>>>> >> =========================================== >>>>> >> What should happen to JIRA tickets still targeting 3.1.3? >>>>> >> =========================================== >>>>> >> >>>>> >> The current list of open tickets targeted at 3.2.1 can be found at: >>>>> >> https://issues.apache.org/jira/projects/SPARK and search for >>>>> "Target >>>>> >> Version/s" = 3.1.3 >>>>> >> >>>>> >> Committers should look at those and triage. Extremely important bug >>>>> >> fixes, documentation, and API tweaks that impact compatibility >>>>> should >>>>> >> be worked on immediately. Everything else please retarget to an >>>>> >> appropriate release. >>>>> >> >>>>> >> ================== >>>>> >> But my bug isn't fixed? >>>>> >> ================== >>>>> >> >>>>> >> In order to make timely releases, we will typically not hold the >>>>> >> release unless the bug in question is a regression from the previous >>>>> >> release. That being said, if there is something that is a regression >>>>> >> that has not been correctly targeted please ping me or a committer >>>>> to >>>>> >> help target the issue. >>>>> >> >>>>> >> ================== >>>>> >> What happened to RC1 & RC2? >>>>> >> ================== >>>>> >> >>>>> >> When I first went to build RC1 the build process failed due to the >>>>> >> lack of the R markdown package in my local rm container. By the time >>>>> >> I had time to debug and rebuild there was already another bug fix >>>>> commit in >>>>> >> branch-3.1 so I decided to skip ahead to RC2 and pick it up >>>>> directly. >>>>> >> When I went to go send the RC2 vote e-mail I noticed a correctness >>>>> issue had >>>>> >> been fixed in branch-3.1 so I rolled RC3 to contain the correctness >>>>> fix. >>>>> >> >>>>> >> -- >>>>> >> Twitter: https://twitter.com/holdenkarau >>>>> >> Books (Learning Spark, High Performance Spark, etc.): >>>>> https://amzn.to/2MaRAG9 >>>>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>> >>>> -- > Twitter: https://twitter.com/holdenkarau > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >
