"branch-4.0" has been cut. Please merge commits to this branch as well if you think it's necessary before Feb 1.
On Wed, Jan 15, 2025 at 4:37 PM Wenchen Fan <cloud0...@gmail.com> wrote: > Yes, I'll start a new thread this week to collect the ongoing projects and > track the progress. We can discuss there and see if the code freeze date > (Feb 1) is reasonable or not. But in any case, we should try our best to > stick with the schedule, or we need to vote to change the date. > > On Wed, Jan 15, 2025 at 4:29 PM Jungtaek Lim <kabhwan.opensou...@gmail.com> > wrote: > >> Shall we not focus on the date itself but focus on the delivery? >> >> I don't have any concern about branch cut, but the starting date of the >> QA period sounds really tight. We have two weeks remaining, and one week >> will be a national holiday of both China and South Korea. Shall we collect >> and check the plan of ongoing projects to see whether they can make it in >> time in advance, so that we could determine the realistic date of the QA >> period rather than an arbitrary one? >> >> On Wed, Jan 15, 2025 at 5:11 PM Wenchen Fan <cloud0...@gmail.com> wrote: >> >>> Dongjoon, let's take it easy. I'll cut the branch today to stick with >>> the committed schedule (2025-01-15), and I'll pick a time that is >>> convenient for me (likely 9 PM UTC+8). As I mentioned earlier, people can >>> keep backporting commits to the 4.0 branch before February 1, so we don't >>> have any blocker for the branch cut (as long as the CI is green). And I >>> don't think the branch cut is an urgent blocker to anything either, so no >>> one needs to wait for it today. >>> >>> For committers, just remember that starting from tomorrow you need to do >>> two additional keyboard actions when merging PRs if you want to backport it >>> to 4.0. >>> >>> Thanks, >>> Wenchen >>> >>> On Wed, Jan 15, 2025 at 3:44 PM Dongjoon Hyun <dongjoon.h...@gmail.com> >>> wrote: >>> >>>> Let me enumerate some questions. >>>> >>>> 1. Since you have no open list, do you have some internal PRs which are >>>> planned to be open the rest of Today? >>>> >>>> 2. For the timezone, the following timezone was originally proposed by >>>> Hyukjin and agreed by me. >>>> Apparently, you disagree with this because you don't want me to >>>> create it at midnight in PST, right? >>>> >>>> > one alternative would be to cut the branch at midnight in PST >>>> > as relatively more commits are coming in daytime in PST. >>>> >>>> 3. Since it's already 3:44 PM (GMT+8) in China, >>>> how many hours do we need to wait until you finish your day? >>>> Could you be more specific? >>>> >>>> > Since it's still within my working hours today, ... after I finish my >>>> day. >>>> >>>> 4. As we know, Apache Spark merge script allows `backporting` with just >>>> two additional keyboard actions. >>>> Even if I create branch-4.0 now, you can use that feature after >>>> having our branch-4.0. >>>> It's easy, isn't it? Or, does it still bother you for some reason? >>>> Are you going to delete `branch-4.0` created by me? >>>> >>>> Sincerely, >>>> Dongjoon. >>>> >>>> >>>> >>>> On Tue, Jan 14, 2025 at 10:55 PM Wenchen Fan <cloud0...@gmail.com> >>>> wrote: >>>> >>>>> I don't have any specific PRs in mind, as I review PRs almost daily: PRs >>>>> I've commented on >>>>> <https://github.com/apache/spark/pulls?q=commenter%3Acloud-fan+sort%3Acreated-desc> >>>>> . >>>>> >>>>> Since it's still within my working hours today, I’d prefer to do the >>>>> branch cut after I finish my day. I hope this is a reasonable request. >>>>> >>>>> On Wed, Jan 15, 2025 at 2:42 PM Dongjoon Hyun <dongjoon.h...@gmail.com> >>>>> wrote: >>>>> >>>>>> Well, previously, you didn't have specific PRs in your very previous >>>>>> email. >>>>>> >>>>>> > I don't have specific PRs in mind, >>>>>> >>>>>> Now, it seems that you made up your mind to tell what is going on >>>>>> there. Which PRs are those? >>>>>> >>>>>> > I'm still reviewing and may merge PRs during the daytime. >>>>>> >>>>>> Let's see what your real concern is and let us help you. >>>>>> >>>>>> I really want to understand how those PRs are blocked or interrupted >>>>>> by `branch cut`. >>>>>> >>>>>> Dongjoon. >>>>>> >>>>>> On Tue, Jan 14, 2025 at 10:25 PM Wenchen Fan <cloud0...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> OK nice joke, I should have said 2025-01-15 23:59:59 PST :) >>>>>>> >>>>>>> Anyway, I'll cut the branch tonight in my timezone (UTC+8) as I'm >>>>>>> still reviewing and may merge PRs during the daytime. The branch cut >>>>>>> will >>>>>>> still be on 2025-01-15 and please let me know if the policy requires the >>>>>>> time to be midnight PST. >>>>>>> >>>>>>> >>>>>>> On Wed, Jan 15, 2025 at 1:57 PM Dongjoon Hyun < >>>>>>> dongjoon.h...@gmail.com> wrote: >>>>>>> >>>>>>>> We provide sufficient information in advance on our website and >>>>>>>> mailing list. >>>>>>>> >>>>>>>> Exactly one week ago, we also had the dev mailing threads about >>>>>>>> `Spark 4.0.0 timeline`. >>>>>>>> >>>>>>>> https://lists.apache.org/thread/vq00tmwz6x4ddjypk2rmw8ktotty0pb6 >>>>>>>> (2025-01-07) >>>>>>>> >>>>>>>> In addition, Apache Spark also complains about the proposed date, >>>>>>>> `2025-01-15 24:00:00 PST`, is invalid. >>>>>>>> >>>>>>>> ``` >>>>>>>> spark-sql (default)> SELECT CAST('2025-01-15 23:59:59' AS >>>>>>>> TIMESTAMP); >>>>>>>> 2025-01-15 23:59:59 >>>>>>>> Time taken: 0.045 seconds, Fetched 1 row(s) >>>>>>>> spark-sql (default)> SELECT CAST('2025-01-15 24:00:00' AS >>>>>>>> TIMESTAMP); >>>>>>>> [CAST_INVALID_INPUT] The value '2025-01-15 24:00:00' of the >>>>>>>> type "STRING" cannot be cast to "TIMESTAMP" because it is malformed. >>>>>>>> ``` >>>>>>>> >>>>>>>> BTW, after creating the `branch-4.0`, the following PR will be >>>>>>>> merged >>>>>>>> in order to differentiate `master` from `branch-4.0`. >>>>>>>> >>>>>>>> [SPARK-50822][BUILD] Setting version to 4.1.0-SNAPSHOT >>>>>>>> https://github.com/apache/spark/pull/49495 >>>>>>>> >>>>>>>> Best regards, >>>>>>>> Dongjoon. >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jan 14, 2025 at 9:03 PM Wenchen Fan <cloud0...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I don't have specific PRs in mind, I'm just referring to the huge >>>>>>>>> amount of commits we merge every day: >>>>>>>>> https://github.com/apache/spark/commits/master >>>>>>>>> >>>>>>>>> BTW, 4 hours seems too short for a reminder, why not 2025-01-15 >>>>>>>>> 24:00:00 PST? I can do it as it will be my afternoon at my timezone. >>>>>>>>> >>>>>>>>> On Wed, Jan 15, 2025 at 11:56 AM Hyukjin Kwon < >>>>>>>>> gurwls...@apache.org> wrote: >>>>>>>>> >>>>>>>>>> I don't have any examples :-). Just wanted to make sure. +1 for >>>>>>>>>> cutting soon as scheduled. >>>>>>>>>> >>>>>>>>>> On Wed, 15 Jan 2025 at 12:55, Dongjoon Hyun < >>>>>>>>>> dongjoon.h...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Thank you, Wenchen and Hyukjin. >>>>>>>>>>> >>>>>>>>>>> BTW, if you can, please provide specific PRs which you are >>>>>>>>>>> referring to as active. >>>>>>>>>>> >>>>>>>>>>> I can help those PRs as much as possible if I can. >>>>>>>>>>> >>>>>>>>>>> For `branch-4.0`, as Hyukjin mentioned, I'm going to create it >>>>>>>>>>> at midnight PST (Today, 2025-01-15 00:00:00 PST). >>>>>>>>>>> >>>>>>>>>>> In other words, we have about 4 hours from now. >>>>>>>>>>> >>>>>>>>>>> Dongjoon. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Jan 14, 2025 at 7:15 PM Hyukjin Kwon < >>>>>>>>>>> gurwls...@apache.org> wrote: >>>>>>>>>>> >>>>>>>>>>>> one alternative would be to cut the branch at midnight in PST >>>>>>>>>>>> as relatively more commits are coming in daytime in PST. >>>>>>>>>>>> >>>>>>>>>>>> On Wed, 15 Jan 2025 at 12:12, Hyukjin Kwon < >>>>>>>>>>>> gurwls...@apache.org> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Yeah, I agree that cutting in the middle of week, e.g., >>>>>>>>>>>>> commits are being merged, it might cause some confusions which >>>>>>>>>>>>> branch >>>>>>>>>>>>> committers should target for a while. >>>>>>>>>>>>> >>>>>>>>>>>>> If we're all happy with that, I am fine going ahead as >>>>>>>>>>>>> scheduled. >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, 15 Jan 2025 at 12:03, Wenchen Fan <cloud0...@gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I don’t have a strong preference, as people can backport >>>>>>>>>>>>>> whatever they need to the 4.0 branch before February 1. My >>>>>>>>>>>>>> intention was to >>>>>>>>>>>>>> avoid interrupting others’ work in the middle of a week when PRs >>>>>>>>>>>>>> are >>>>>>>>>>>>>> actively being merged, and I thought the weekend would be a >>>>>>>>>>>>>> better time. >>>>>>>>>>>>>> However, I’m fine with either way. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Jan 15, 2025 at 10:40 AM Dongjoon Hyun < >>>>>>>>>>>>>> dongjoon.h...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, Wenchen. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'd not delay the branch cut. We had better stick to our >>>>>>>>>>>>>>> plan and those week days to stablize branch-4.0. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If you are busy, I'm going to cut the branch on schedule >>>>>>>>>>>>>>> (2015-01-15) and help the on-going PR reviews. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> As you mentioned, we don't block feature backportings. There >>>>>>>>>>>>>>> is no reason to break the community schedule. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sincerely, >>>>>>>>>>>>>>> Dongjoon >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, Jan 14, 2025 at 17:52 Wenchen Fan < >>>>>>>>>>>>>>> cloud0...@gmail.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> According to the schedule >>>>>>>>>>>>>>>> <https://spark.apache.org/versioning-policy.html>, the 4.0 >>>>>>>>>>>>>>>> branch cut is set for January 15. I plan to cut the branch at >>>>>>>>>>>>>>>> the end of >>>>>>>>>>>>>>>> this week so that folks can have a few more working days to >>>>>>>>>>>>>>>> complete the >>>>>>>>>>>>>>>> ongoing work. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Note: While we can still merge PRs to the 4.0 branch after >>>>>>>>>>>>>>>> the cut, code freeze will begin on February 1. After that >>>>>>>>>>>>>>>> date, only bug >>>>>>>>>>>>>>>> fixes can be backported. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Wenchen >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>
