回复：[DISCUSS] Releasing Flink 1.4

[Zhijiang(wangzhijiang999)]

   Hi Stephan!
   Thanks for replying.
   I am clearly now, all the big new features would be covered in 1.5. And 
it is very necessary to fully test big changes before release.
   Zhijiang
--发件人：Stephan 
Ewen 发送时间：2017年10月18日(星期三) 22:54收件人：Till Rohrmann 
抄　送：dev ; Zhijiang(wangzhijiang999) 
主　题：Re: [DISCUSS] Releasing Flink 1.4
Hi Zhijiang!
The proposal is to release 1.4 asap and have the master switch so 1.5 soon 
(early November or so). Merge the network code directly after that - the 1.5 
release would come after flip-6, network stack, and local state caching is 
merged and tested.
That gives us a better QA window.
The reason is that we got feedback from Netflix and some other users that 1.3 
had too many bugs, because we merged big changes directly before the release. 
We are worried that users lose trust in Flink of we release these big new 
features too fast and without intensively testing them.
What do you think?
Stephan
On Oct 18, 2017 09:27, "Till Rohrmann"  wrote:
Thanks for driving this effort Aljoscha. I would be in favour of quick feature 
freeze (31st of October). However, looking at the list of open and in progress 
blockers we still have 53 blocker issues. I think it is unrealistic to complete 
all of them within the next two weeks. I would therefore propose to go again 
over the list of blockers to decide what is really a blocker for 1.4 and what 
not.
@Zhijiang, my gut feeling is that we will have this feature in Flink 1.5. There 
are simply too many other open issues which we have to address first in order 
to finish the 1.4 release. Moreover, merging it early in the release cycle of 
1.5 will give the barrier alignment improvements a good amount of exposure to 
spot potential problems before releasing it officially.
Cheers,Till
On Wed, Oct 18, 2017 at 10:00 AM, Zhijiang(wangzhijiang999) 
 wrote:
@Stephan Ewen , do you think the barrier alignment improvement should be 
covered in release-1.4 or release-1.5?It can avoid spilling data on receiver 
side for exactly-once and speed the barrier alignment.

We have implemented this feature based on new credit-based flow control and 
verified the good performance.If needed, I will try to submit the related PRs 
after network stack merged into 1.4 ASAP.

--发件人：Aljoscha 
Krettek 发送时间：2017年10月18日(星期三) 03:04收件人：dev 
主　题：Re: [DISCUSS] Releasing Flink 1.4
It seems we all agree on the general plan. Now we have to talk actual dates. I 
propose an aggressive feature-freeze date of October 31st (that would be in 2 
weeks) so that we can release 1.4 early and can go full-speed with the features 
moved to 1.5.

We can still merge in new features during the next two weeks but I propose that 
we then cut a release branch, start testing, and only merge bug fixes until the 
release. What do you think?


> On 17. Oct 2017, at 08:37, Piotr Nowojski  wrote:

> 
> +1

> 

> I would only try to merge as many of the smaller network stack improvements 
>as possible for 1.4, since they give quite big performance improvement.

> 

> Piotrek

> 

>> On 16 Oct 2017, at 17:42, Eron Wright  wrote:

>> 

>> +1 from our side on this plan.

>> 

>> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske  wrote:

>> 

>>> OK, sounds good to me.

>>> 

>>> We have a couple of bugs to fix for the Table API / SQL but have PRs for

>>> most of them.

>>> 

>>> There's only one major issue that I'd like to include in 1.4.0 which is a

>>> refactoring of the TableSource interface.

>>> This effort has already started and is currently waiting for reviews /

>>> comments.

>>> I'm quite confident that we can get it in within the next two weeks.

>>> 

>>> Cheers, Fabian

>>> 

>>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek :

>>> 

 @Bowen I started marking essential stuff as blocking (with fixVersion

 1.4.0). You're right, that we should start moving things to 1.5.0 that

>>> are

 not blocking and that we don't think will make it into 1.4.0. I think we

 can only release 1.4.0 if there are 0 (zero) unresolved issues with

 fixVersion 1.4.0.

 

> On 14. Oct 2017, at 07:34, Alexandru Gutan 

 wrote:

> 

> great

> 

> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <

> wangzhijiang...@aliyun.com> wrote:

> 

>> totally agree with the way.--

>> 发件人：Stephan Ewen <

 se...@apache.org

>>> 发送时间：2017年10月13日(星期五) 21:29收件人：dev@flink.apache.org <

 dev@flink.apache.org>主

>> 题：Re: [DISCUSS] Releasing Flink 

回复：[DISCUSS] Releasing Flink 1.4

[Zhijiang(wangzhijiang999)]

Thanks for replying @Till Rohrmann , and I agree with your suggestion.
Cheers,Zhijiang
--发件人：Till 
Rohrmann 发送时间：2017年10月18日(星期三) 16:27收件人：dev 
; Zhijiang(wangzhijiang999) 抄　
送：Stephan Ewen 主　题：Re: [DISCUSS] Releasing Flink 1.4
Thanks for driving this effort Aljoscha. I would be in favour of quick
feature freeze (31st of October). However, looking at the list of open and
in progress blockers we still have 53 blocker issues. I think it is
unrealistic to complete all of them within the next two weeks. I would
therefore propose to go again over the list of blockers to decide what is
really a blocker for 1.4 and what not.

@Zhijiang, my gut feeling is that we will have this feature in Flink 1.5.
There are simply too many other open issues which we have to address first
in order to finish the 1.4 release. Moreover, merging it early in the
release cycle of 1.5 will give the barrier alignment improvements a good
amount of exposure to spot potential problems before releasing it
officially.

Cheers,
Till

On Wed, Oct 18, 2017 at 10:00 AM, Zhijiang(wangzhijiang999) <
wangzhijiang...@aliyun.com> wrote:

> @Stephan Ewen , do you think the barrier alignment improvement should be
> covered in release-1.4 or release-1.5?It can avoid spilling data on
> receiver side for exactly-once and speed the barrier alignment.
> We have implemented this feature based on new credit-based flow control
> and verified the good performance.If needed, I will try to submit the
> related PRs after network stack merged into 1.4 ASAP.
> --发件人：Aljoscha
> Krettek 发送时间：2017年10月18日(星期三) 03:04收件人：dev <
> dev@flink.apache.org>主 题：Re: [DISCUSS] Releasing Flink 1.4
> It seems we all agree on the general plan. Now we have to
> talk actual dates. I propose an aggressive feature-freeze
> date of October 31st (that would be in 2 weeks) so that
> we can release 1.4 early and can go full-speed with the
> features moved to 1.5.
>
> We can still merge in new features during the next two
> weeks but I propose that we then cut a release branch,
> start testing, and only merge bug fixes until the release.
> What do you think?
>
> > On 17. Oct 2017, at 08:37, Piotr Nowojski  > wrote:
> >
> > +1
> >
> > I would only try to merge as many of the smaller
> network stack improvements as possible for 1.4, since they
> give quite big performance improvement.
> >
> > Piotrek
> >
> >> On 16 Oct 2017, at 17:42, Eron Wright  wrote:
> >>
> >> +1 from our side on this plan.
> >>
> >> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske  > wrote:
> >>
> >>> OK, sounds good to me.
> >>>
> >>> We have a couple of bugs to fix for the Table API /
> SQL but have PRs for
> >>> most of them.
> >>>
> >>> There's only one major issue that I'd like to
> include in 1.4.0 which is a
> >>> refactoring of the TableSource interface.
> >>> This effort has already started and is currently waiting for reviews /
> >>> comments.
> >>> I'm quite confident that we can get it in within the next two weeks.
> >>>
> >>> Cheers, Fabian
> >>>
> >>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek :
> >>>
>  @Bowen I started marking essential stuff as blocking (with fixVersion
>  1.4.0). You're right, that we should start moving
> things to 1.5.0 that
> >>> are
>  not blocking and that we don't think will make it into
> 1.4.0. I think we
>  can only release 1.4.0 if there are 0 (zero) unresolved issues with
>  fixVersion 1.4.0.
> 
> > On 14. Oct 2017, at 07:34, Alexandru Gutan 
>  wrote:
> >
> > great
> >
> > On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
> > wangzhijiang...@aliyun.com> wrote:
> >
> >> totally agree with the way.--
> >> 发件人：Stephan Ewen <
>  se...@apache.org
> >>> 发送时间：2017年10月13日(星期五) 21:29收件人：dev@flink.apache.org <
>  dev@flink.apache.org>主
> >> 题：Re: [DISCUSS] Releasing Flink 1.4
> >> I am in favor of doing this, if we can set it up in
> the following way.
> >>
> >> - We put out the 1.4 release now, as Till and Aljoscha suggested. A
> >> stable cut before the fundamental changes go in.
> >>
> >> - We merge the very big changes (FLIP-6, Network stack, localized
> >>> state
> >> restore, etc). directly (or very soon) after.
> >> - We try to stabilize these changes and release 1.5
> asap after that.
> >> Ideally Around end of year or so.
> >>
> >> The reason I am bringing this up is that I
> know various users waiting
>  very
> >> much for FLIP-6 and Network Stack enhancements. Given that these
> >>> issues
> >> 

回复：[DISCUSS] Releasing Flink 1.4

[Zhijiang(wangzhijiang999)]

@Stephan Ewen , do you think the barrier alignment improvement should be 
covered in release-1.4 or release-1.5?It can avoid spilling data on receiver 
side for exactly-once and speed the barrier alignment.
We have implemented this feature based on new credit-based flow control and 
verified the good performance.If needed, I will try to submit the related PRs 
after network stack merged into 1.4 ASAP.
--发件人：Aljoscha 
Krettek 发送时间：2017年10月18日(星期三) 03:04收件人：dev 
主　题：Re: [DISCUSS] Releasing Flink 1.4
It seems we all agree on the general plan. Now we have to talk actual dates. I 
propose an aggressive feature-freeze date of October 31st (that would be in 2 
weeks) so that we can release 1.4 early and can go full-speed with the features 
moved to 1.5.

We can still merge in new features during the next two weeks but I propose that 
we then cut a release branch, start testing, and only merge bug fixes until the 
release. What do you think?

> On 17. Oct 2017, at 08:37, Piotr Nowojski  wrote:
> 
> +1
> 
> I would only try to merge as many of the smaller network stack improvements 
>as possible for 1.4, since they give quite big performance improvement.
> 
> Piotrek
> 
>> On 16 Oct 2017, at 17:42, Eron Wright  wrote:
>> 
>> +1 from our side on this plan.
>> 
>> On Mon, Oct 16, 2017 at 3:33 AM, Fabian Hueske  wrote:
>> 
>>> OK, sounds good to me.
>>> 
>>> We have a couple of bugs to fix for the Table API / SQL but have PRs for
>>> most of them.
>>> 
>>> There's only one major issue that I'd like to include in 1.4.0 which is a
>>> refactoring of the TableSource interface.
>>> This effort has already started and is currently waiting for reviews /
>>> comments.
>>> I'm quite confident that we can get it in within the next two weeks.
>>> 
>>> Cheers, Fabian
>>> 
>>> 2017-10-16 10:22 GMT+02:00 Aljoscha Krettek :
>>> 
 @Bowen I started marking essential stuff as blocking (with fixVersion
 1.4.0). You're right, that we should start moving things to 1.5.0 that
>>> are
 not blocking and that we don't think will make it into 1.4.0. I think we
 can only release 1.4.0 if there are 0 (zero) unresolved issues with
 fixVersion 1.4.0.
 
> On 14. Oct 2017, at 07:34, Alexandru Gutan 
 wrote:
> 
> great
> 
> On 13 October 2017 at 18:02, Zhijiang(wangzhijiang999) <
> wangzhijiang...@aliyun.com> wrote:
> 
>> totally agree with the way.--
>> 发件人：Stephan Ewen <
 se...@apache.org
>>> 发送时间：2017年10月13日(星期五) 21:29收件人：dev@flink.apache.org <
 dev@flink.apache.org>主
>> 题：Re: [DISCUSS] Releasing Flink 1.4
>> I am in favor of doing this, if we can set it up in the following way.
>> 
>> - We put out the 1.4 release now, as Till and Aljoscha suggested. A
>> stable cut before the fundamental changes go in.
>> 
>> - We merge the very big changes (FLIP-6, Network stack, localized
>>> state
>> restore, etc). directly (or very soon) after.
>> - We try to stabilize these changes and release 1.5 asap after that.
>> Ideally Around end of year or so.
>> 
>> The reason I am bringing this up is that I know various users waiting
 very
>> much for FLIP-6 and Network Stack enhancements. Given that these
>>> issues
>> were flagged for release 1.4, the users were planning to have them
 rather
>> soon.
>> 
>> Stephan
>> 
>> 
>> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek <
>>> aljos...@apache.org>
>> wrote:
>> 
>>> +1 Excellent
>>> 
>>> I'd like to volunteer as release manager. I already set
>> up a Kanban board
>>> to monitor the open blocking (and non-blocking) issues
>> for 1.4, though this
>>> is independent of me volunteering as release manager. We
>> should all go over
>>> these issues and see which ones should actually be blockin
>> g and which ones
>>> are not yet on that list.
>>> 
 On 13. Oct 2017, at 12:24, Renjie Liu 
 wrote:
 
 Cool!!!
 
 On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann >> wrote:
 
> Hi all,
> 
> I want to revive the discussion about releasing Flink 1.4 [1] and
>>> the
>>> set
> of features to include.
> 
> The gist of the previous discussion was that we postponed the
>>> feature
> freeze for 1.4 in order to include some more features w
>> hich were being
> developed. By now, we have completed a good set of features such as
>>> exactly
> once Kafka producer, reduced dependency footprint, Hado
>> op-free Flink and
> many bug fixes. I 

Re: 回复：[DISCUSS] Releasing Flink 1.4

[Bowen Li]

+1.

To give users a heads up, might be good to start to change the ‘target release’ 
of some Jira tickets from 1.4 to 1.5.

Sent from my iPhone

> On Oct 13, 2017, at 11:02 PM, Zhijiang(wangzhijiang999) 
>  wrote:
> 
> totally agree with the 
> way.--发件人：Stephan
>  Ewen 发送时间：2017年10月13日(星期五) 21:29收件人：dev@flink.apache.org 
> 主　题：Re: [DISCUSS] Releasing Flink 1.4
> I am in favor of doing this, if we can set it up in the following way.
> 
>  - We put out the 1.4 release now, as Till and Aljoscha suggested. A
> stable cut before the fundamental changes go in.
> 
>  - We merge the very big changes (FLIP-6, Network stack, localized state
> restore, etc). directly (or very soon) after.
>  - We try to stabilize these changes and release 1.5 asap after that.
> Ideally Around end of year or so.
> 
> The reason I am bringing this up is that I know various users waiting very
> much for FLIP-6 and Network Stack enhancements. Given that these issues
> were flagged for release 1.4, the users were planning to have them rather
> soon.
> 
> Stephan
> 
> 
> On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek 
> wrote:
> 
>> +1 Excellent
>> 
>> I'd like to volunteer as release manager. I already set up a Kanban board
>> to monitor the open blocking (and non-blocking) issues for 1.4, though this
>> is independent of me volunteering as release manager. We should all go over
>> these issues and see which ones should actually be blocking and which ones
>> are not yet on that list.
>> 
>>> On 13. Oct 2017, at 12:24, Renjie Liu  wrote:
>>> 
>>> Cool!!!
>>> 
 On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann 
>>> wrote:
>>> 
 Hi all,
 
 I want to revive the discussion about releasing Flink 1.4 [1] and the
>> set
 of features to include.
 
 The gist of the previous discussion was that we postponed the feature
 freeze for 1.4 in order to include some more features which were being
 developed. By now, we have completed a good set of features such as
>> exactly
 once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
 many bug fixes. I believe that these features will make good release and
 users are already waiting for them.
 
 Some of the other features which we wanted to include, mainly Flip-6, to
 some extent the network stack enhancements and the state decoupling
>> still
 need some more time. Since these features are major changes to Flink's
 runtime, it would be in my opinion a good idea to cut a stable release
>> with
 the above-mentioned feature set now and give the engine features a bit
>> more
 time to ripen and be properly tested.
 
 Therefore, I would actually be in favour of aiming for a quick release
 meaning that we now concentrate mainly on fixing bugs and critical
>> issues.
 Moreover, I'm optimistic that the delayed features will be completed
>> soon
 such that we can deliver them with the next release. What do you think?
 
 [1]
 
 http://apache-flink-mailing-list-archive.1008284.n3.
>> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
 
 Cheers,
 Till
>>> --
>>> Liu, Renjie
>>> Software Engineer, MVAD
> 

回复：[DISCUSS] Releasing Flink 1.4

[Zhijiang(wangzhijiang999)]

totally agree with the 
way.--发件人：Stephan
 Ewen 发送时间：2017年10月13日(星期五) 21:29收件人：dev@flink.apache.org 
主　题：Re: [DISCUSS] Releasing Flink 1.4
I am in favor of doing this, if we can set it up in the following way.

  - We put out the 1.4 release now, as Till and Aljoscha suggested. A
stable cut before the fundamental changes go in.

  - We merge the very big changes (FLIP-6, Network stack, localized state
restore, etc). directly (or very soon) after.
  - We try to stabilize these changes and release 1.5 asap after that.
Ideally Around end of year or so.

The reason I am bringing this up is that I know various users waiting very
much for FLIP-6 and Network Stack enhancements. Given that these issues
were flagged for release 1.4, the users were planning to have them rather
soon.

Stephan


On Fri, Oct 13, 2017 at 2:35 PM, Aljoscha Krettek 
wrote:

> +1 Excellent
>
> I'd like to volunteer as release manager. I already set up a Kanban board
> to monitor the open blocking (and non-blocking) issues for 1.4, though this
> is independent of me volunteering as release manager. We should all go over
> these issues and see which ones should actually be blocking and which ones
> are not yet on that list.
>
> > On 13. Oct 2017, at 12:24, Renjie Liu  wrote:
> >
> > Cool!!!
> >
> > On Fri, Oct 13, 2017 at 5:49 PM Till Rohrmann 
> wrote:
> >
> >> Hi all,
> >>
> >> I want to revive the discussion about releasing Flink 1.4 [1] and the
> set
> >> of features to include.
> >>
> >> The gist of the previous discussion was that we postponed the feature
> >> freeze for 1.4 in order to include some more features which were being
> >> developed. By now, we have completed a good set of features such as
> exactly
> >> once Kafka producer, reduced dependency footprint, Hadoop-free Flink and
> >> many bug fixes. I believe that these features will make good release and
> >> users are already waiting for them.
> >>
> >> Some of the other features which we wanted to include, mainly Flip-6, to
> >> some extent the network stack enhancements and the state decoupling
> still
> >> need some more time. Since these features are major changes to Flink's
> >> runtime, it would be in my opinion a good idea to cut a stable release
> with
> >> the above-mentioned feature set now and give the engine features a bit
> more
> >> time to ripen and be properly tested.
> >>
> >> Therefore, I would actually be in favour of aiming for a quick release
> >> meaning that we now concentrate mainly on fixing bugs and critical
> issues.
> >> Moreover, I'm optimistic that the delayed features will be completed
> soon
> >> such that we can deliver them with the next release. What do you think?
> >>
> >> [1]
> >>
> >> http://apache-flink-mailing-list-archive.1008284.n3.
> nabble.com/DISCUSS-Flink-1-4-and-time-based-release-td19331.html
> >>
> >> Cheers,
> >> Till
> >>
> > --
> > Liu, Renjie
> > Software Engineer, MVAD
>
>