Re: [VOTE] Release Spark 3.1.0 (RC1)

Hi, I'm curious why Spark 3.1.0 is already available in repo1.maven.org? https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.12/3.1.0/ Pozdrawiam, Jacek Laskowski https://about.me/JacekLaskowski "The Internals Of" Online Books Follow me on https://twitter

Re: [VOTE] Release Spark 3.1.0 (RC1)

Er, yeah uh oh. Did the staging repo accidentally get closed/released? Maybe I'm also missing something. If so then one way or the other we can't undo that as the 3.1.0 release in Maven, as far as I know. We can make a 3.1.1, but then it's kind of weird there was never any 3.1.0 source release. We

Re: [VOTE] Release Spark 3.1.0 (RC1)

Yes, it was my mistake. I faced the same issue as INFRA-20651 , and it is worse in my case because I misunderstood that RC and releases are separately released out. Right after this, I filed an INFRA JIRA to revert this at INFRA-21266

Re: [VOTE] Release Spark 3.1.0 (RC1)

Thank you, Jacek, Sean, and Hyukjin. The release is a human-driven process. Everyone can make mistakes. For example, I released Apache Spark 2.2.3 with a missing pandoc, but we didn't touch it because it's a community-blessed official version. https://pypi.org/project/pyspark/2.2.3/ For

Re: [VOTE] Release Spark 3.1.0 (RC1)

OK, we'll have to update the release page to clarify there was never a real 3.1.0 release then. But I'm not suggesting releasing 3.1.0 _because_ it was published accidentally. I'm suggesting we figure out normally whether we would have released it, and if so, great. If not, fine we must skip the v

Re: [VOTE] Release Spark 3.1.0 (RC1)

I think it makes sense to wait and see what they say on INFRA-21266.   In the mean time hopefully people can start testing it and if no other problems found and vote passes can stay published.  It seems like the 2 issues above wouldn't be blockers in my opinion and could be handled in a 3.1.1 bu

Re: [VOTE] Release Spark 3.1.0 (RC1)

Thanks Dongjoon, Sean and Tom. I just thought that we could have some more bug fixes or some changes if RC1 passes as a regular release due to the relatively fewer RCs. I agree that if this RC passes, it's just that an RC passed normally per the regular process, and there's nothing wrong here. By r

Re: [VOTE] Release Spark 3.1.0 (RC1)

No worries about the accident. We're human beings, and everyone can make a mistake. Let's wait and see the response of INFRA-21266. Just a 2 cents, I'm actually leaning toward to skip 3.1.0 and start the release process for 3.1.1, as anyone could be some sort of "rushing" on verification on 3.1.0.

Re: [VOTE] Release Spark 3.1.0 (RC1)

I just don't see a reason to believe there's a rush? just test it as normal? I did, you can too, etc. Or specifically what blocks the current RC? On Wed, Jan 6, 2021 at 5:46 PM Jungtaek Lim wrote: > No worries about the accident. We're human beings, and everyone can make a > mistake. Let's wait

Re: [VOTE] Release Spark 3.1.0 (RC1)

Before we discover the pre-uploaded artifacts, both Jungtaek and Hyukjin already made two blockers shared here. IIUC, it meant implicitly RC1 failure at that time. In addition to that, there are two correctness issues. So, I made up my mind to cast -1 for this RC1 before joining this thread. SPAR

Re: [VOTE] Release Spark 3.1.0 (RC1)

I don't agree the first two are blockers for reasons I gave earlier. Those two do look like important issues - are they regressions from 3.0.1? I do agree we'd probably cut a new RC for those in any event, so agree with the plan to drop 3.1.0 (if the Maven release can't be overwritten) On Wed, Jan

Re: [VOTE] Release Spark 3.1.0 (RC1)

I think that It would be great though if we have a clear blocker that makes the release pointless if we want to drop this RC practically given that we will schedule 3.1.1 faster - non-regression bug fixes will be delivered to end users relatively fast. That would make it clear which option we shoul

Re: [VOTE] Release Spark 3.1.0 (RC1)

Okay, let me just start to prepare 3.1.1. I think that will address all concerns except that 3.1.0 will remain in Maven as incomplete. By right, removal in the Maven repo is disallowed. Overwrite is possible as far as I know but other mirrors that maintain cache will get affected. Maven is one of t

Re: [VOTE] Release Spark 3.1.0 (RC1)

I agree with Jungtaek that people are likely to be biased when testing 3.1.0. At least this will not be the same community-blessed release as previous ones, because the voting is already affected by the fact that 3.1.0 is already in maven central. Skipping 3.1.0 sounds better to me. On Thu, Jan 7,

Re: [VOTE] Release Spark 3.1.0 (RC1)

I think that posting the 3.1.0 maven release was an accident and we're going to 3.1.1 RCs is the right step forward. I'd ask for maybe a day before cutting the 3.1.1 release, I think https://issues.apache.org/jira/browse/SPARK-34018 is also a blocker (at first I thought it was just a test issue, bu

Re: [VOTE] Release Spark 3.1.0 (RC1)

Thank you Holden and Wenchen! Let me: - prepare a PR for news in spark-website first about 3.1.0 accident late tonight (in KST) - and start to prepare 3.1.1 probably in few more days like next monday in case other people have different thoughts 2021년 1월 7일 (목) 오후 2:04, Holden Karau 님이 작성: > I