Re: [DISCUSS] Schedule and Scope for Flink 1.2

t; > > wrote: > >> > >> > > > >> > >> > > > Hi, > >> > >> > > > we're still working on making the backwards compatibility > from > >> 1.1 > >>

Re: [DISCUSS] Schedule and Scope for Flink 1.2

e tests >> now >> > >> but >> > >> > it >> > >> > > > still needs some work. This is the issue that tracks the >> progress >> > on >> > >> > the >> > >> > > > op

Re: [DISCUSS] Schedule and Scope for Flink 1.2

>> but > > >> > it > > >> > > > still needs some work. This is the issue that tracks the > progress > > on > > >> > the > > >> > > > operators that we would like to make backwards

Re: [DISCUSS] Schedule and Scope for Flink 1.2

we would like to make backwards compatible: > >> > > > https://issues.apache.org/jira/browse/FLINK-5292 > >> > > > > >> > > > Cheers, > >> > > > Aljoscha > >> > > > > >> > > > On Tue, 13 Dec 2016 at 11:22 Feng Wang

Re: [DISCUSS] Schedule and Scope for Flink 1.2

s, >> > > > Aljoscha >> > > > >> > > > On Tue, 13 Dec 2016 at 11:22 Feng Wang >> wrote: >> > > > >> > > > > It will be pretty good if 1.2 branch could be forked off within >> this >> > > > week, >> > > > > and our guys w

Re: [DISCUSS] Schedule and Scope for Flink 1.2

branch could be merged > > into > > > > > master as soon as possible. > > > > > > > > > > Best Regards, > > > > > > > > > > Feng Wang > > > > > > > > > > Alibaba > > > > >

Re: [DISCUSS] Schedule and Scope for Flink 1.2

gards, > > > > > > > > Feng Wang > > > > > > > > Alibaba > > > > > > > > > > > > From: Robert Metzger > > > > Sent: Tuesday, December 13, 2016 4:58 AM > > > &g

Re: [DISCUSS] Schedule and Scope for Flink 1.2

this > > week, > > > and our guys working on FLIP-6 hope FLIP-6 branch could be merged into > > > master as soon as possible. > > > > > > Best Regards, > > > > > > Feng Wang > > > > > > Alibaba > > > > >

Re: [DISCUSS] Schedule and Scope for Flink 1.2

> > > > > From: Robert Metzger > > Sent: Tuesday, December 13, 2016 4:58 AM > > To: dev@flink.apache.org > > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2 > > > > Thank you all for figuring out a solution for

Re: [DISCUSS] Schedule and Scope for Flink 1.2

t; > Best Regards, > > Feng Wang > > Alibaba > > > From: Robert Metzger > Sent: Tuesday, December 13, 2016 4:58 AM > To: dev@flink.apache.org > Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2 > > Thank you all for figuring out

Re: [DISCUSS] Schedule and Scope for Flink 1.2

, 2016 4:58 AM To: dev@flink.apache.org Subject: Re: [DISCUSS] Schedule and Scope for Flink 1.2 Thank you all for figuring out a solution for the security pull request. Lets try to get 1.2 feature freezed as fast as possible so that we can "unblock" waiting features like FLIP-6 and the

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Thank you all for figuring out a solution for the security pull request. Lets try to get 1.2 feature freezed as fast as possible so that we can "unblock" waiting features like FLIP-6 and the remaining security changes. *What do you think about Friday evening (6pm Berlin, 9am US west coast) for f

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi Stephan, The new handler changes for Netty layer is straightforward and it is not really an workaround. Moreover if we think there is a better way to handle it in future we can easily unwind it as it is just a pluggable handler. Pushing these changes before FLIP-6 merge certainly avoids lot

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi Vijay! The workaround you suggest may be doable, but I am wondering how much that helps, because the authorization feature would be incomplete like that and thus of limited use. I would also assume that merging it properly and in full use after the 1.2 release would be a bit better - in genera

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Max and Ufuk, I respect your concerns and fully understand the importance of the network layer stack in Flink code base. Will you be comfortable to merge the code if I remove the Netty layer changes and leave the rest of the code. We can address the Netty code changes post 1.2 release? Regards,

Re: [DISCUSS] Schedule and Scope for Flink 1.2

On 12 December 2016 at 12:30:31, Maximilian Michels (m...@apache.org) wrote: > > It seems like we lack the resources for now to properly to take > care > of your pull request before the release. Unless someone from > the > community is really eager to help out here, I would be in favor > of >

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi Vijay, Thank you for updating the pull request. I appreciate your work in the security realm of Flink and value your contributions so far. It would be great to merge the authorization pull request for the release. However, I don't feel comfortable about the network stack (i.e. Netty) related ch

Re: [DISCUSS] Schedule and Scope for Flink 1.2

On FLINK-3930, almost all of the feedback has been addressed. The only pending review is Netty cookie authorization part which I have moved the cookie validation from message level to a separate channel handler. I have just rebased the code with master for final review. Regards, Vijay Sent fro

Re: [DISCUSS] Schedule and Scope for Flink 1.2

We might also think about addressing: Relocate Flink's Hadoop dependency and its transitive dependencies (FLINK-5297), because a user reported that they cannot use the system due to a dependency issue. Cheers, Till On Thu, Dec 8, 2016 at 10:17 AM, Robert Metzger wrote: > Thank you for your re

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Thank you for your responses Max and Vijay. So I understand that Mesos is basically ready for the 1.2 release. Regarding the security changes: Having Hadoop, Kafka and Zookeeper integration is a big improvement and a much requested feature. I'm super excited to have that in :) Are all the other se

Re: [DISCUSS] Schedule and Scope for Flink 1.2

> - UNRESOLVED Integrate Flink with Apache Mesos (FLINK-1984) The initial integration is already completed with the last issues being resolved in the Mesos component: https://issues.apache.org/jira/browse/FLINK/component/12331068/ The implementation will be further refined after the next release a

Re: [DISCUSS] Schedule and Scope for Flink 1.2

>>Secure Data Access (FLINK-3930) The PR for the work is still under review and I hope this could be included in the release. Regards, Vijay Sent from my iPhone > On Dec 6, 2016, at 11:51 AM, Robert Metzger wrote: > > UNRESOLVED Secure Data Access (FLINK-3930)

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Thank you all for your responses so far. The situation is better than I expected :) Let me summarize, and please object if you disagree: - RESOLVED dynamic Scaling / Key Groups (FLINK-3755) - RESOLVED Add Rescalable Non-Partitioned State (FLINK-4379) - RESOLVED [Split for 1.3] Add Flink 1.1 savep

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi Robert, I agree with Stefan on the estimation about 4. As for the testing, this can already start, as some basic operators have already been ported. Kostas > On Dec 6, 2016, at 5:50 PM, Stefan Richter > wrote: > > e already restored a job with general window operator and kafka source in a

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi Robert, The state of (FLINK-4797) Add Flink 1.1 savepoint backwards compatibility is as follows: We have identified 5 major steps: 1) Convert savepoint format, task state, state handles, etc. 2) Convert binary format of keyed state backends (Rocks, Heap, Fs) 3) Introduce a first step for for

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Thanks Robert for bringing this up again. Would be great to come to that @use: I'll take a look at https://github.com/apache/flink/pull/2752 Considering "UNRESOLVED Provide support for asynchronous operations over streams (FLINK-4391)" - that is currently in the hands of the contributor. I would

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hey Robert, thanks for bringing the discussion up again. On Mon, Dec 5, 2016 at 6:28 PM, Robert Metzger wrote: > - UNRESOLVED Unify Savepoints and Checkpoints (FLINK-4484) This issue/FLIP contained two parts: 1. Write periodic checkpoints out to external storage 2. Allow periodic scheduling

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Thanks Robert for reactivating the discussion. There are a few Table API related refactorings that I would like to have in 1.2 because they are API breaking: - FLINK-4704 Clean up the packages of the Table API and move it to the org.apache.flink.table package space. This is an API breaking change

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi Robert, our pull request was closed in favor of https://github.com/apache/flink/pull/2861. However that PR still need some review I think (I also added some comment in the JIRA issue[1]). it will be great to have that fix in the next release :) [1] https://issues.apache.org/jira/browse/FLINK-51

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi, I would like to reactivate this old email thread. The initial plan was to release 1.2 before Christmas, have RC1 by end of Nov / December and a feature freeze mid-November. Today (December 5), we did not yet do the feature freeze. I would like to get an overview again of the remaining blockers

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi to all, we've completed the PR #2790 to implement FLINK-4491 : we've added the comment that @zentol asked for a week ago and we also fixed a potential annoying problem we faced during indexing (if a mal

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi to all, for the 1.2 release do you think it will be possible to include also a PR to implement FLINK-4491 ? The PR is waiting for a review at https://github.com/apache/flink/pull/2790. Best, Flavio On Fri, Oct 28, 2016 at 5:59 PM, Fabian Hueske

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hi Jark, FLINK-4469: I agree, UDTFs would be a great feature. I don't think we should make it a blocker but I will help to get it the master before the feature freeze. FLINK-4704: Regarding the renaming of the namespace, we have two positive votes in JIRA, but we should also start a discussion on

Re: [DISCUSS] Schedule and Scope for Flink 1.2

Hello, As Aljoshca said, the TriggerDSL depends on another PR, before it can be merged. In addition, this will change also the whole structure of how we test window/trigger related functionality, so some refactoring will be required. Kostas > On Oct 28, 2016, at 3:06 PM, Aljoscha Krettek wrote

Re: [DISCUSS] Schedule and Scope for Flink 1.2

@Till: For the TriggerDSL we have to merge my open PR [1] for refactoring the WindowOperator/Trigger tests and then refactor the tests of the TriggerDSL to use the new test facilities introduced there. Plus then maybe some fixups. On Fri, 28 Oct 2016 at 11:17 Jark Wu wrote: > +1 to the schedule

Re: [DISCUSS] Schedule and Scope for Flink 1.2

+1 to the schedule I hope to add 1) Table API: User defined table function (FLINK-4469 ) 2) Table API: Move Table API to org.apache.flink.table (FLINK-4704 ) The 1) has been discussed for a l

Re: [DISCUSS] Schedule and Scope for Flink 1.2

+1 for the features to include. What is the state of the Trigger DSL? How much is left to be done before merging? Cheers, Till On Tue, Oct 25, 2016 at 4:32 PM, Aljoscha Krettek wrote: > +1 the schedule proposed so far. > > Do we also want to get in the "Trigger DSL" that we've had brewing for

Re: [DISCUSS] Schedule and Scope for Flink 1.2

+1 the schedule proposed so far. Do we also want to get in the "Trigger DSL" that we've had brewing for a while now? On Mon, 17 Oct 2016 at 16:17 Stephan Ewen wrote: > I think this sounds very reasonable, +1 to the schedule. > > I would definitely add > - FLIP-10 (unify checkpoints and savepo

Re: [DISCUSS] Schedule and Scope for Flink 1.2

I think this sounds very reasonable, +1 to the schedule. I would definitely add - FLIP-10 (unify checkpoints and savepoints) - FLIP-7 (metrics in web UI) - FLIP-12 (async request operators) Those should be all safe bets, as they are basically done. Let's see what else is in shape until the

[DISCUSS] Schedule and Scope for Flink 1.2

Hi everybody, Flink 1.1.0 was released in August and I think it is time to kick off a discussion about the schedule and scope of Flink 1.2.0. == Scope We started to collect features for Flink 1.2.0 in the Flink Release wiki page [1]. I copy the feature list for convenience: - Dynamic Scaling