Re: [DISCUSS] Make a release to be announced at ApacheCon
I'll post the checklist along with the preview release candidate (RC0). I actually wanted to create it today (as announced three days ago) but apache's git repositories are down :( I'll try again tomorrow. On Tue, Mar 31, 2015 at 11:35 AM, Maximilian Michels m...@apache.org wrote: +1 Would be great if we created a checklist with all the things to watch out for when testing the release candidate. On Mon, Mar 30, 2015 at 9:00 PM, Henry Saputra henry.sapu...@gmail.com wrote: +1 Would be good to have well documented release process with all the black magic scripts we have =) Thanks for driving this, Robert. - Henry On Mon, Mar 30, 2015 at 11:15 AM, Robert Metzger rmetz...@apache.org wrote: Okay, I think we have reached consensus on this. I'll create a RC0 non-voting, preview release candidate for 0.9.0- milestone-1 on Thursday (April 2) this week so that we have version to tests against. Once all issues of RC0 have been resolved, we'll start voting in the week of April 6. (The vote needs to start at the latest of April 7, so that we have time on Friday to update the website, send the final release files to the mirrors (they need 24 hrs) and mvn central (24 hrs as well). Monday after that ApacheCon will start.) I'll be on vacation at the beginning of next week, but I'm sure Marton or Ufuk can also create RC1 and the VOTE. I'll start documenting the release process in the Wiki, including a list required verification steps during the VOTE process. On Fri, Mar 27, 2015 at 4:06 PM, Till Rohrmann trohrm...@apache.org wrote: +1 for 0.9.0-milestone-1 On Fri, Mar 27, 2015 at 3:52 PM, Kostas Tzoumas ktzou...@apache.org wrote: +1 On Fri, Mar 27, 2015 at 3:44 PM, Aljoscha Krettek aljos...@apache.org wrote: +1 for 0.9.0-M1 (or milestone-1) On Mar 27, 2015 2:45 PM, Ufuk Celebi u...@apache.org wrote: On Friday, March 27, 2015, Maximilian Michels m...@apache.org wrote: +1 for 0.9.0-milestone-1 +1
Re: [DISCUSS] Make a release to be announced at ApacheCon
Okay, I think we have reached consensus on this. I'll create a RC0 non-voting, preview release candidate for 0.9.0- milestone-1 on Thursday (April 2) this week so that we have version to tests against. Once all issues of RC0 have been resolved, we'll start voting in the week of April 6. (The vote needs to start at the latest of April 7, so that we have time on Friday to update the website, send the final release files to the mirrors (they need 24 hrs) and mvn central (24 hrs as well). Monday after that ApacheCon will start.) I'll be on vacation at the beginning of next week, but I'm sure Marton or Ufuk can also create RC1 and the VOTE. I'll start documenting the release process in the Wiki, including a list required verification steps during the VOTE process. On Fri, Mar 27, 2015 at 4:06 PM, Till Rohrmann trohrm...@apache.org wrote: +1 for 0.9.0-milestone-1 On Fri, Mar 27, 2015 at 3:52 PM, Kostas Tzoumas ktzou...@apache.org wrote: +1 On Fri, Mar 27, 2015 at 3:44 PM, Aljoscha Krettek aljos...@apache.org wrote: +1 for 0.9.0-M1 (or milestone-1) On Mar 27, 2015 2:45 PM, Ufuk Celebi u...@apache.org wrote: On Friday, March 27, 2015, Maximilian Michels m...@apache.org wrote: +1 for 0.9.0-milestone-1 +1
Re: [DISCUSS] Make a release to be announced at ApacheCon
+1 Would be good to have well documented release process with all the black magic scripts we have =) Thanks for driving this, Robert. - Henry On Mon, Mar 30, 2015 at 11:15 AM, Robert Metzger rmetz...@apache.org wrote: Okay, I think we have reached consensus on this. I'll create a RC0 non-voting, preview release candidate for 0.9.0- milestone-1 on Thursday (April 2) this week so that we have version to tests against. Once all issues of RC0 have been resolved, we'll start voting in the week of April 6. (The vote needs to start at the latest of April 7, so that we have time on Friday to update the website, send the final release files to the mirrors (they need 24 hrs) and mvn central (24 hrs as well). Monday after that ApacheCon will start.) I'll be on vacation at the beginning of next week, but I'm sure Marton or Ufuk can also create RC1 and the VOTE. I'll start documenting the release process in the Wiki, including a list required verification steps during the VOTE process. On Fri, Mar 27, 2015 at 4:06 PM, Till Rohrmann trohrm...@apache.org wrote: +1 for 0.9.0-milestone-1 On Fri, Mar 27, 2015 at 3:52 PM, Kostas Tzoumas ktzou...@apache.org wrote: +1 On Fri, Mar 27, 2015 at 3:44 PM, Aljoscha Krettek aljos...@apache.org wrote: +1 for 0.9.0-M1 (or milestone-1) On Mar 27, 2015 2:45 PM, Ufuk Celebi u...@apache.org wrote: On Friday, March 27, 2015, Maximilian Michels m...@apache.org wrote: +1 for 0.9.0-milestone-1 +1
Re: [DISCUSS] Make a release to be announced at ApacheCon
+1 for 0.9.0-milestone-1
Re: [DISCUSS] Make a release to be announced at ApacheCon
+1 for 0.9.0-milestone-1. On Fri, Mar 27, 2015 at 12:47 PM, Stephan Ewen se...@apache.org wrote: Okay, to how about we make this dependency groupIdorg.apache.flink/groupId artifactIdflink-core/artifactId version0.9.0-milestone-1/version /dependency I think it is common that milestones have numbers. There is no such thing as the milestone. On Thu, Mar 26, 2015 at 10:02 PM, Henry Saputra henry.sapu...@gmail.com wrote: Yeah, always prefer to get it with consensus that VOTE I am fine with either. - Henry On Thu, Mar 26, 2015 at 11:03 AM, Kostas Tzoumas ktzou...@apache.org wrote: The ASF press team wants to announce next week, so a 3-day vote right now might cancel the subject line of this thread :-) Perhaps we can reach consensus in the DISCUSS thread or have a 24-hour vote? I agree with Stephan on 0.9.0.M1 (or 0.9.0-m1 or whatever), as it seems that other open source projects are using this naming scheme. Kostas On Thu, Mar 26, 2015 at 6:10 PM, Stephan Ewen se...@apache.org wrote: I think Milestone pretty much says that we have some crucial things in there, but not all. Beta in comparison, has an immature early version connotation. We are, for example, using a milestone 1 version of Jetty for the Web Frontend, so that is a pretty standard thing, in my opinion: dependency groupIdorg.eclipse.jetty/groupId artifactIdjetty-server/artifactId version8.0.0.M1/version /dependency On Thu, Mar 26, 2015 at 3:44 PM, Robert Metzger rmetz...@apache.org wrote: Looks like we need to vote on 0.9-beta or 0.9-milestone. Can we find consensus whether to add a 1 after the name? -beta1 or -milestone1. Adding a 1 allows us to create a second beta/milestone release. I'm against adding a 1. On Thu, Mar 26, 2015 at 3:40 PM, Ufuk Celebi u...@apache.org wrote: On 26 Mar 2015, at 11:01, Robert Metzger rmetz...@apache.org wrote: Two weeks have passed since we've discussed the 0.9 release the last time. The ApacheCon is in 18 days from now. If we want, we can also release a 0.9.0-beta release that contains known bugs, but allows our users to try out the new features easily (because they are part of a release). The vote for such a release would be mainly about the legal aspects of the release rather than the stability. So I suspect that the vote will go through much quicker. +1 for 0.9-beta
Re: [DISCUSS] Make a release to be announced at ApacheCon
Two weeks have passed since we've discussed the 0.9 release the last time. The ApacheCon is in 18 days from now. If we want, we can also release a 0.9.0-beta release that contains known bugs, but allows our users to try out the new features easily (because they are part of a release). The vote for such a release would be mainly about the legal aspects of the release rather than the stability. So I suspect that the vote will go through much quicker. On Fri, Mar 13, 2015 at 12:01 PM, Robert Metzger rmetz...@apache.org wrote: I've reopened https://issues.apache.org/jira/browse/FLINK-1650 because the issue is still occurring. On Thu, Mar 12, 2015 at 7:05 PM, Ufuk Celebi u...@apache.org wrote: On Thursday, March 12, 2015, Till Rohrmann till.rohrm...@gmail.com wrote: Have you run the 20 builds with the new shading code? With new shading the TaskManagerFailsITCase should no longer fail. If it still does, then we have to look into it again. No, rebased on Monday before shading. Let me rebase and rerun tonight.
Re: [DISCUSS] Make a release to be announced at ApacheCon
+1 for an early milestone release. Perhaps we can call it 0.9-milestone or so? On Thu, Mar 26, 2015 at 11:01 AM, Robert Metzger rmetz...@apache.org wrote: Two weeks have passed since we've discussed the 0.9 release the last time. The ApacheCon is in 18 days from now. If we want, we can also release a 0.9.0-beta release that contains known bugs, but allows our users to try out the new features easily (because they are part of a release). The vote for such a release would be mainly about the legal aspects of the release rather than the stability. So I suspect that the vote will go through much quicker. On Fri, Mar 13, 2015 at 12:01 PM, Robert Metzger rmetz...@apache.org wrote: I've reopened https://issues.apache.org/jira/browse/FLINK-1650 because the issue is still occurring. On Thu, Mar 12, 2015 at 7:05 PM, Ufuk Celebi u...@apache.org wrote: On Thursday, March 12, 2015, Till Rohrmann till.rohrm...@gmail.com wrote: Have you run the 20 builds with the new shading code? With new shading the TaskManagerFailsITCase should no longer fail. If it still does, then we have to look into it again. No, rebased on Monday before shading. Let me rebase and rerun tonight.
Re: [DISCUSS] Make a release to be announced at ApacheCon
+1 for an early release. It will help unblock the samoa PR that has 0.9 dependencies. On 26 Mar 2015, at 11:44, Kostas Tzoumas ktzou...@apache.org wrote: +1 for an early milestone release. Perhaps we can call it 0.9-milestone or so? On Thu, Mar 26, 2015 at 11:01 AM, Robert Metzger rmetz...@apache.org wrote: Two weeks have passed since we've discussed the 0.9 release the last time. The ApacheCon is in 18 days from now. If we want, we can also release a 0.9.0-beta release that contains known bugs, but allows our users to try out the new features easily (because they are part of a release). The vote for such a release would be mainly about the legal aspects of the release rather than the stability. So I suspect that the vote will go through much quicker. On Fri, Mar 13, 2015 at 12:01 PM, Robert Metzger rmetz...@apache.org wrote: I've reopened https://issues.apache.org/jira/browse/FLINK-1650 because the issue is still occurring. On Thu, Mar 12, 2015 at 7:05 PM, Ufuk Celebi u...@apache.org wrote: On Thursday, March 12, 2015, Till Rohrmann till.rohrm...@gmail.com wrote: Have you run the 20 builds with the new shading code? With new shading the TaskManagerFailsITCase should no longer fail. If it still does, then we have to look into it again. No, rebased on Monday before shading. Let me rebase and rerun tonight.
Re: [DISCUSS] Make a release to be announced at ApacheCon
The ASF press team wants to announce next week, so a 3-day vote right now might cancel the subject line of this thread :-) Perhaps we can reach consensus in the DISCUSS thread or have a 24-hour vote? I agree with Stephan on 0.9.0.M1 (or 0.9.0-m1 or whatever), as it seems that other open source projects are using this naming scheme. Kostas On Thu, Mar 26, 2015 at 6:10 PM, Stephan Ewen se...@apache.org wrote: I think Milestone pretty much says that we have some crucial things in there, but not all. Beta in comparison, has an immature early version connotation. We are, for example, using a milestone 1 version of Jetty for the Web Frontend, so that is a pretty standard thing, in my opinion: dependency groupIdorg.eclipse.jetty/groupId artifactIdjetty-server/artifactId version8.0.0.M1/version /dependency On Thu, Mar 26, 2015 at 3:44 PM, Robert Metzger rmetz...@apache.org wrote: Looks like we need to vote on 0.9-beta or 0.9-milestone. Can we find consensus whether to add a 1 after the name? -beta1 or -milestone1. Adding a 1 allows us to create a second beta/milestone release. I'm against adding a 1. On Thu, Mar 26, 2015 at 3:40 PM, Ufuk Celebi u...@apache.org wrote: On 26 Mar 2015, at 11:01, Robert Metzger rmetz...@apache.org wrote: Two weeks have passed since we've discussed the 0.9 release the last time. The ApacheCon is in 18 days from now. If we want, we can also release a 0.9.0-beta release that contains known bugs, but allows our users to try out the new features easily (because they are part of a release). The vote for such a release would be mainly about the legal aspects of the release rather than the stability. So I suspect that the vote will go through much quicker. +1 for 0.9-beta
Re: [DISCUSS] Make a release to be announced at ApacheCon
Yeah, always prefer to get it with consensus that VOTE I am fine with either. - Henry On Thu, Mar 26, 2015 at 11:03 AM, Kostas Tzoumas ktzou...@apache.org wrote: The ASF press team wants to announce next week, so a 3-day vote right now might cancel the subject line of this thread :-) Perhaps we can reach consensus in the DISCUSS thread or have a 24-hour vote? I agree with Stephan on 0.9.0.M1 (or 0.9.0-m1 or whatever), as it seems that other open source projects are using this naming scheme. Kostas On Thu, Mar 26, 2015 at 6:10 PM, Stephan Ewen se...@apache.org wrote: I think Milestone pretty much says that we have some crucial things in there, but not all. Beta in comparison, has an immature early version connotation. We are, for example, using a milestone 1 version of Jetty for the Web Frontend, so that is a pretty standard thing, in my opinion: dependency groupIdorg.eclipse.jetty/groupId artifactIdjetty-server/artifactId version8.0.0.M1/version /dependency On Thu, Mar 26, 2015 at 3:44 PM, Robert Metzger rmetz...@apache.org wrote: Looks like we need to vote on 0.9-beta or 0.9-milestone. Can we find consensus whether to add a 1 after the name? -beta1 or -milestone1. Adding a 1 allows us to create a second beta/milestone release. I'm against adding a 1. On Thu, Mar 26, 2015 at 3:40 PM, Ufuk Celebi u...@apache.org wrote: On 26 Mar 2015, at 11:01, Robert Metzger rmetz...@apache.org wrote: Two weeks have passed since we've discussed the 0.9 release the last time. The ApacheCon is in 18 days from now. If we want, we can also release a 0.9.0-beta release that contains known bugs, but allows our users to try out the new features easily (because they are part of a release). The vote for such a release would be mainly about the legal aspects of the release rather than the stability. So I suspect that the vote will go through much quicker. +1 for 0.9-beta
Re: [DISCUSS] Make a release to be announced at ApacheCon
+1 for a beta release. So there is no feature-freeze until the RC right? On 26.03.2015 15:32, Márton Balassi wrote: +1 for the early release. I'd call it 0.9-milestone1. On Thu, Mar 26, 2015 at 1:37 PM, Maximilian Michels m...@apache.org wrote: +1 for a beta release: 0.9-beta. On Thu, Mar 26, 2015 at 12:09 PM, Paris Carbone par...@kth.se wrote: +1 for an early release. It will help unblock the samoa PR that has 0.9 dependencies. On 26 Mar 2015, at 11:44, Kostas Tzoumas ktzou...@apache.org wrote: +1 for an early milestone release. Perhaps we can call it 0.9-milestone or so? On Thu, Mar 26, 2015 at 11:01 AM, Robert Metzger rmetz...@apache.org wrote: Two weeks have passed since we've discussed the 0.9 release the last time. The ApacheCon is in 18 days from now. If we want, we can also release a 0.9.0-beta release that contains known bugs, but allows our users to try out the new features easily (because they are part of a release). The vote for such a release would be mainly about the legal aspects of the release rather than the stability. So I suspect that the vote will go through much quicker. On Fri, Mar 13, 2015 at 12:01 PM, Robert Metzger rmetz...@apache.org wrote: I've reopened https://issues.apache.org/jira/browse/FLINK-1650 because the issue is still occurring. On Thu, Mar 12, 2015 at 7:05 PM, Ufuk Celebi u...@apache.org wrote: On Thursday, March 12, 2015, Till Rohrmann till.rohrm...@gmail.com wrote: Have you run the 20 builds with the new shading code? With new shading the TaskManagerFailsITCase should no longer fail. If it still does, then we have to look into it again. No, rebased on Monday before shading. Let me rebase and rerun tonight.
Re: [DISCUSS] Make a release to be announced at ApacheCon
@Timo: No feature freeze for this, yes. On Thu, Mar 26, 2015 at 3:36 PM, Timo Walther twal...@apache.org wrote: +1 for a beta release. So there is no feature-freeze until the RC right? On 26.03.2015 15:32, Márton Balassi wrote: +1 for the early release. I'd call it 0.9-milestone1. On Thu, Mar 26, 2015 at 1:37 PM, Maximilian Michels m...@apache.org wrote: +1 for a beta release: 0.9-beta. On Thu, Mar 26, 2015 at 12:09 PM, Paris Carbone par...@kth.se wrote: +1 for an early release. It will help unblock the samoa PR that has 0.9 dependencies. On 26 Mar 2015, at 11:44, Kostas Tzoumas ktzou...@apache.org wrote: +1 for an early milestone release. Perhaps we can call it 0.9-milestone or so? On Thu, Mar 26, 2015 at 11:01 AM, Robert Metzger rmetz...@apache.org wrote: Two weeks have passed since we've discussed the 0.9 release the last time. The ApacheCon is in 18 days from now. If we want, we can also release a 0.9.0-beta release that contains known bugs, but allows our users to try out the new features easily (because they are part of a release). The vote for such a release would be mainly about the legal aspects of the release rather than the stability. So I suspect that the vote will go through much quicker. On Fri, Mar 13, 2015 at 12:01 PM, Robert Metzger rmetz...@apache.org wrote: I've reopened https://issues.apache.org/jira/browse/FLINK-1650 because the issue is still occurring. On Thu, Mar 12, 2015 at 7:05 PM, Ufuk Celebi u...@apache.org wrote: On Thursday, March 12, 2015, Till Rohrmann till.rohrm...@gmail.com wrote: Have you run the 20 builds with the new shading code? With new shading the TaskManagerFailsITCase should no longer fail. If it still does, then we have to look into it again. No, rebased on Monday before shading. Let me rebase and rerun tonight.
Re: [DISCUSS] Make a release to be announced at ApacheCon
On 26 Mar 2015, at 11:01, Robert Metzger rmetz...@apache.org wrote: Two weeks have passed since we've discussed the 0.9 release the last time. The ApacheCon is in 18 days from now. If we want, we can also release a 0.9.0-beta release that contains known bugs, but allows our users to try out the new features easily (because they are part of a release). The vote for such a release would be mainly about the legal aspects of the release rather than the stability. So I suspect that the vote will go through much quicker. +1 for 0.9-beta
Re: [DISCUSS] Make a release to be announced at ApacheCon
I've reopened https://issues.apache.org/jira/browse/FLINK-1650 because the issue is still occurring. On Thu, Mar 12, 2015 at 7:05 PM, Ufuk Celebi u...@apache.org wrote: On Thursday, March 12, 2015, Till Rohrmann till.rohrm...@gmail.com wrote: Have you run the 20 builds with the new shading code? With new shading the TaskManagerFailsITCase should no longer fail. If it still does, then we have to look into it again. No, rebased on Monday before shading. Let me rebase and rerun tonight.
Re: [DISCUSS] Make a release to be announced at ApacheCon
Have you run the 20 builds with the new shading code? With new shading the TaskManagerFailsITCase should no longer fail. If it still does, then we have to look into it again. On Thu, Mar 12, 2015 at 2:01 PM, Stephan Ewen se...@apache.org wrote: I am also big time skeptical. There are some remaining stability issues with 0.9 - Apparently a bug in the task canceling - Blocking Data Exchange is premature at this point - TaskManager startup is not robust - TaskManager / JobManager registration is not robust - Streaming fault tolerance needs more testing before we can make an assessment I think this needs a few more weeks... On Thu, Mar 12, 2015 at 1:57 PM, Aljoscha Krettek aljos...@apache.org wrote: I would like to get the Expression API for Java in there, as well. On Thu, Mar 12, 2015 at 11:59 AM, Ufuk Celebi u...@apache.org wrote: On Thu, Mar 12, 2015 at 10:11 AM, Robert Metzger rmetz...@apache.org wrote: So you're saying regarding the release you don't feel very confident that we manage to fork off release-0.9 next week? Yes. At the moment I would be uncomfortable with forking off. Regarding the failing tests: I thought that some failings jobs were related to my changes, but after looking into it, it was a false alarm. See comments here: https://github.com/apache/flink/pull/475
Re: [DISCUSS] Make a release to be announced at ApacheCon
On Thursday, March 12, 2015, Till Rohrmann till.rohrm...@gmail.com wrote: Have you run the 20 builds with the new shading code? With new shading the TaskManagerFailsITCase should no longer fail. If it still does, then we have to look into it again. No, rebased on Monday before shading. Let me rebase and rerun tonight.
Re: [DISCUSS] Make a release to be announced at ApacheCon
I would like to get the Expression API for Java in there, as well. On Thu, Mar 12, 2015 at 11:59 AM, Ufuk Celebi u...@apache.org wrote: On Thu, Mar 12, 2015 at 10:11 AM, Robert Metzger rmetz...@apache.org wrote: So you're saying regarding the release you don't feel very confident that we manage to fork off release-0.9 next week? Yes. At the moment I would be uncomfortable with forking off. Regarding the failing tests: I thought that some failings jobs were related to my changes, but after looking into it, it was a false alarm. See comments here: https://github.com/apache/flink/pull/475
Re: [DISCUSS] Make a release to be announced at ApacheCon
I am also big time skeptical. There are some remaining stability issues with 0.9 - Apparently a bug in the task canceling - Blocking Data Exchange is premature at this point - TaskManager startup is not robust - TaskManager / JobManager registration is not robust - Streaming fault tolerance needs more testing before we can make an assessment I think this needs a few more weeks... On Thu, Mar 12, 2015 at 1:57 PM, Aljoscha Krettek aljos...@apache.org wrote: I would like to get the Expression API for Java in there, as well. On Thu, Mar 12, 2015 at 11:59 AM, Ufuk Celebi u...@apache.org wrote: On Thu, Mar 12, 2015 at 10:11 AM, Robert Metzger rmetz...@apache.org wrote: So you're saying regarding the release you don't feel very confident that we manage to fork off release-0.9 next week? Yes. At the moment I would be uncomfortable with forking off. Regarding the failing tests: I thought that some failings jobs were related to my changes, but after looking into it, it was a false alarm. See comments here: https://github.com/apache/flink/pull/475
Re: [DISCUSS] Make a release to be announced at ApacheCon
On Thu, Mar 12, 2015 at 10:11 AM, Robert Metzger rmetz...@apache.org wrote: So you're saying regarding the release you don't feel very confident that we manage to fork off release-0.9 next week? Yes. At the moment I would be uncomfortable with forking off. Regarding the failing tests: I thought that some failings jobs were related to my changes, but after looking into it, it was a false alarm. See comments here: https://github.com/apache/flink/pull/475
Re: [DISCUSS] Make a release to be announced at ApacheCon
I will follow up again with Sally this week if there any special messaging or communications needed to do for the Apache Con from our side. - Henry On Tue, Mar 10, 2015 at 3:20 AM, Robert Metzger rmetz...@apache.org wrote: Hey, whats the status on this? There is one week left until we are going to fork off a branch for 0.9 .. if we stick to the suggested timeline. The initial email said I am very much in favor of doing this, under the strong condition that we are very confident that the master has grown to be stable enough. I think it is time to evaluate whether we are confident that the master is stable. Best Robert On Wed, Mar 4, 2015 at 9:42 AM, Robert Metzger rmetz...@apache.org wrote: +1 for Marton as a release manager. Thank you! On Tue, Mar 3, 2015 at 7:56 PM, Henry Saputra henry.sapu...@gmail.com wrote: Ah, thanks Márton. So we are chartering to the similar concept of Spark RRD staging execution =P I suppose there will be a runtime configuration or hint to tell the Flink Job manager to indicate which execution is preferred? - Henry On Tue, Mar 3, 2015 at 2:09 AM, Márton Balassi balassi.mar...@gmail.com wrote: Hi Henry, Batch mode is a new execution mode for batch Flink jobs where instead of pipelining the whole execution the job is scheduled in stages, thus materializing the intermediate result before continuing to the next operators. For implications see [1]. [1] http://www.slideshare.net/KostasTzoumas/flink-internals, page 18-21. On Mon, Mar 2, 2015 at 11:39 PM, Henry Saputra henry.sapu...@gmail.com wrote: HI Stephan, What is Batch mode feature in the list? - Henry On Mon, Mar 2, 2015 at 5:03 AM, Stephan Ewen se...@apache.org wrote: Hi all! ApacheCon is coming up and it is the 15th anniversary of the Apache Software Foundation. In the course of the conference, Apache would like to make a series of announcements. If we manage to make a release during (or shortly before) ApacheCon, they will announce it through their channels. I am very much in favor of doing this, under the strong condition that we are very confident that the master has grown to be stable enough (there are major changes in the distributed runtime since version 0.8 that we are still stabilizing). No use in a widely announced build that does not have the quality. Flink has now many new features that warrant a release soon (once we fixed the last quirks in the new distributed runtime). Notable new features are: - Gelly - Streaming windows - Flink on Tez - Expression API - Distributed Runtime on Akka - Batch mode - Maybe even a first ML library version - Some streaming fault tolerance Robert proposed to have a feature freeze mid Match for that. His cornerpoints were: Feature freeze (forking off release-0.9): March 17 RC1 vote: March 24 The RC1 vote is 20 days before the ApacheCon (13. April). For the last three releases, the average voting time was 20 days: R 0.8.0 -- 14 days R 0.7.0 -- 22 days R 0.6 -- 26 days Please share your opinion on this! Greetings, Stephan
Re: [DISCUSS] Make a release to be announced at ApacheCon
On the streaming side: Must have: * Tests for the fault tolerance (My first priority this week) * Merging Gyula's recent windowing PR [1] Really needed: * Self-join for DataStreams (Gabor has a prototype, PR coming today) [1] * ITCase tests for streaming examples (Peter myself, review and clean up pending) [3] * Different streaming/batch cluster memory settings (Stephan) [4] * Make projection operator chainable (Gabor Gevay - a wannabe GSoC student, PR coming soon) [5] * Parallel time discretization (Gyula, PR coming tomorrow) [6] Would be nice to have: * Complex integration test for streaming (Peter) [7] * Extend streaming aggregation tests to include POJOs [8] * Iteration bug for large input [9] We would also need a general pass over the streaming API for javadocs. This is not one week but we can hopefully fit into two weeks. [1] https://github.com/apache/flink/pull/465 [2] https://issues.apache.org/jira/browse/FLINK-1594 [3] https://issues.apache.org/jira/browse/FLINK-1560 [4] https://issues.apache.org/jira/browse/FLINK-1368 [5] https://issues.apache.org/jira/browse/FLINK-1641 [6] https://issues.apache.org/jira/browse/FLINK-1618 [7] https://issues.apache.org/jira/browse/FLINK-1595 [8] https://issues.apache.org/jira/browse/FLINK-1544 [9] https://issues.apache.org/jira/browse/FLINK-1239 On Tue, Mar 10, 2015 at 11:20 AM, Robert Metzger rmetz...@apache.org wrote: Hey, whats the status on this? There is one week left until we are going to fork off a branch for 0.9 .. if we stick to the suggested timeline. The initial email said I am very much in favor of doing this, under the strong condition that we are very confident that the master has grown to be stable enough. I think it is time to evaluate whether we are confident that the master is stable. Best Robert On Wed, Mar 4, 2015 at 9:42 AM, Robert Metzger rmetz...@apache.org wrote: +1 for Marton as a release manager. Thank you! On Tue, Mar 3, 2015 at 7:56 PM, Henry Saputra henry.sapu...@gmail.com wrote: Ah, thanks Márton. So we are chartering to the similar concept of Spark RRD staging execution =P I suppose there will be a runtime configuration or hint to tell the Flink Job manager to indicate which execution is preferred? - Henry On Tue, Mar 3, 2015 at 2:09 AM, Márton Balassi balassi.mar...@gmail.com wrote: Hi Henry, Batch mode is a new execution mode for batch Flink jobs where instead of pipelining the whole execution the job is scheduled in stages, thus materializing the intermediate result before continuing to the next operators. For implications see [1]. [1] http://www.slideshare.net/KostasTzoumas/flink-internals, page 18-21. On Mon, Mar 2, 2015 at 11:39 PM, Henry Saputra henry.sapu...@gmail.com wrote: HI Stephan, What is Batch mode feature in the list? - Henry On Mon, Mar 2, 2015 at 5:03 AM, Stephan Ewen se...@apache.org wrote: Hi all! ApacheCon is coming up and it is the 15th anniversary of the Apache Software Foundation. In the course of the conference, Apache would like to make a series of announcements. If we manage to make a release during (or shortly before) ApacheCon, they will announce it through their channels. I am very much in favor of doing this, under the strong condition that we are very confident that the master has grown to be stable enough (there are major changes in the distributed runtime since version 0.8 that we are still stabilizing). No use in a widely announced build that does not have the quality. Flink has now many new features that warrant a release soon (once we fixed the last quirks in the new distributed runtime). Notable new features are: - Gelly - Streaming windows - Flink on Tez - Expression API - Distributed Runtime on Akka - Batch mode - Maybe even a first ML library version - Some streaming fault tolerance Robert proposed to have a feature freeze mid Match for that. His cornerpoints were: Feature freeze (forking off release-0.9): March 17 RC1 vote: March 24 The RC1 vote is 20 days before the ApacheCon (13. April). For the last three releases, the average voting time was 20 days: R 0.8.0 -- 14 days R 0.7.0 -- 22 days R 0.6 -- 26 days Please share your opinion on this! Greetings, Stephan
Re: [DISCUSS] Make a release to be announced at ApacheCon
+1 for Marton as a release manager. Thank you! On Tue, Mar 3, 2015 at 7:56 PM, Henry Saputra henry.sapu...@gmail.com wrote: Ah, thanks Márton. So we are chartering to the similar concept of Spark RRD staging execution =P I suppose there will be a runtime configuration or hint to tell the Flink Job manager to indicate which execution is preferred? - Henry On Tue, Mar 3, 2015 at 2:09 AM, Márton Balassi balassi.mar...@gmail.com wrote: Hi Henry, Batch mode is a new execution mode for batch Flink jobs where instead of pipelining the whole execution the job is scheduled in stages, thus materializing the intermediate result before continuing to the next operators. For implications see [1]. [1] http://www.slideshare.net/KostasTzoumas/flink-internals, page 18-21. On Mon, Mar 2, 2015 at 11:39 PM, Henry Saputra henry.sapu...@gmail.com wrote: HI Stephan, What is Batch mode feature in the list? - Henry On Mon, Mar 2, 2015 at 5:03 AM, Stephan Ewen se...@apache.org wrote: Hi all! ApacheCon is coming up and it is the 15th anniversary of the Apache Software Foundation. In the course of the conference, Apache would like to make a series of announcements. If we manage to make a release during (or shortly before) ApacheCon, they will announce it through their channels. I am very much in favor of doing this, under the strong condition that we are very confident that the master has grown to be stable enough (there are major changes in the distributed runtime since version 0.8 that we are still stabilizing). No use in a widely announced build that does not have the quality. Flink has now many new features that warrant a release soon (once we fixed the last quirks in the new distributed runtime). Notable new features are: - Gelly - Streaming windows - Flink on Tez - Expression API - Distributed Runtime on Akka - Batch mode - Maybe even a first ML library version - Some streaming fault tolerance Robert proposed to have a feature freeze mid Match for that. His cornerpoints were: Feature freeze (forking off release-0.9): March 17 RC1 vote: March 24 The RC1 vote is 20 days before the ApacheCon (13. April). For the last three releases, the average voting time was 20 days: R 0.8.0 -- 14 days R 0.7.0 -- 22 days R 0.6 -- 26 days Please share your opinion on this! Greetings, Stephan
Re: [DISCUSS] Make a release to be announced at ApacheCon
Hi Henry, Batch mode is a new execution mode for batch Flink jobs where instead of pipelining the whole execution the job is scheduled in stages, thus materializing the intermediate result before continuing to the next operators. For implications see [1]. [1] http://www.slideshare.net/KostasTzoumas/flink-internals, page 18-21. On Mon, Mar 2, 2015 at 11:39 PM, Henry Saputra henry.sapu...@gmail.com wrote: HI Stephan, What is Batch mode feature in the list? - Henry On Mon, Mar 2, 2015 at 5:03 AM, Stephan Ewen se...@apache.org wrote: Hi all! ApacheCon is coming up and it is the 15th anniversary of the Apache Software Foundation. In the course of the conference, Apache would like to make a series of announcements. If we manage to make a release during (or shortly before) ApacheCon, they will announce it through their channels. I am very much in favor of doing this, under the strong condition that we are very confident that the master has grown to be stable enough (there are major changes in the distributed runtime since version 0.8 that we are still stabilizing). No use in a widely announced build that does not have the quality. Flink has now many new features that warrant a release soon (once we fixed the last quirks in the new distributed runtime). Notable new features are: - Gelly - Streaming windows - Flink on Tez - Expression API - Distributed Runtime on Akka - Batch mode - Maybe even a first ML library version - Some streaming fault tolerance Robert proposed to have a feature freeze mid Match for that. His cornerpoints were: Feature freeze (forking off release-0.9): March 17 RC1 vote: March 24 The RC1 vote is 20 days before the ApacheCon (13. April). For the last three releases, the average voting time was 20 days: R 0.8.0 -- 14 days R 0.7.0 -- 22 days R 0.6 -- 26 days Please share your opinion on this! Greetings, Stephan
[DISCUSS] Make a release to be announced at ApacheCon
Hi all! ApacheCon is coming up and it is the 15th anniversary of the Apache Software Foundation. In the course of the conference, Apache would like to make a series of announcements. If we manage to make a release during (or shortly before) ApacheCon, they will announce it through their channels. I am very much in favor of doing this, under the strong condition that we are very confident that the master has grown to be stable enough (there are major changes in the distributed runtime since version 0.8 that we are still stabilizing). No use in a widely announced build that does not have the quality. Flink has now many new features that warrant a release soon (once we fixed the last quirks in the new distributed runtime). Notable new features are: - Gelly - Streaming windows - Flink on Tez - Expression API - Distributed Runtime on Akka - Batch mode - Maybe even a first ML library version - Some streaming fault tolerance Robert proposed to have a feature freeze mid Match for that. His cornerpoints were: Feature freeze (forking off release-0.9): March 17 RC1 vote: March 24 The RC1 vote is 20 days before the ApacheCon (13. April). For the last three releases, the average voting time was 20 days: R 0.8.0 -- 14 days R 0.7.0 -- 22 days R 0.6 -- 26 days Please share your opinion on this! Greetings, Stephan
Re: [DISCUSS] Make a release to be announced at ApacheCon
Hey, We have a nice list of new features - it definitely makes sense to have that as a release. On my side I really want to have a first limited version of streaming fault tolerance in it. +1 for Robert's proposal for the deadlines. I'm also volunteering for release manager. Best, Marton On Mon, Mar 2, 2015 at 2:03 PM, Stephan Ewen se...@apache.org wrote: Hi all! ApacheCon is coming up and it is the 15th anniversary of the Apache Software Foundation. In the course of the conference, Apache would like to make a series of announcements. If we manage to make a release during (or shortly before) ApacheCon, they will announce it through their channels. I am very much in favor of doing this, under the strong condition that we are very confident that the master has grown to be stable enough (there are major changes in the distributed runtime since version 0.8 that we are still stabilizing). No use in a widely announced build that does not have the quality. Flink has now many new features that warrant a release soon (once we fixed the last quirks in the new distributed runtime). Notable new features are: - Gelly - Streaming windows - Flink on Tez - Expression API - Distributed Runtime on Akka - Batch mode - Maybe even a first ML library version - Some streaming fault tolerance Robert proposed to have a feature freeze mid Match for that. His cornerpoints were: Feature freeze (forking off release-0.9): March 17 RC1 vote: March 24 The RC1 vote is 20 days before the ApacheCon (13. April). For the last three releases, the average voting time was 20 days: R 0.8.0 -- 14 days R 0.7.0 -- 22 days R 0.6 -- 26 days Please share your opinion on this! Greetings, Stephan
