Re: Add a (temporary) Portable Flink branch to the ASF repo?

+1 I would merge on master asap. Regards JB Le 18 avr. 2018 à 20:09, à 20:09, "Ismaël Mejía" a écrit: >+1 to bring this to ASF repo ASAP. Also +1 to Aljoscha's idea if we >can guarantee no interference. An extra branch has the issue of >rebasing that will be 'automatically'

Re: Add a (temporary) Portable Flink branch to the ASF repo?

+1 to bring this to ASF repo ASAP. Also +1 to Aljoscha's idea if we can guarantee no interference. An extra branch has the issue of rebasing that will be 'automatically' solved if the development continues in master. Also other it will be easier to more contributors to jump in. On Wed, Apr 18,

Re: Add a (temporary) Portable Flink branch to the ASF repo?

I think we're actually quite safe in merging this into master because it doesn't really touch the currently existing Flink Runner. The "Portable Flink Runner" is essentially a new Runner that reuses some of the existing code and runtime components but doesn't modify them. What do you think? >

Re: Add a (temporary) Portable Flink branch to the ASF repo?

We now have 290 commits in (bsidhom/beam/hacking-job-server - apache/beam/master). To better get a handle on this, I created https://docs.google.com/spreadsheets/d/1KF5n5eTq0neIqwhIkFbvRTbp0G5mqmJ9O4ywSMWtfq0/edit#gid=1955143518 ; I'd ask everyone to help fill this out (creating and/or assigning

Re: Add a (temporary) Portable Flink branch to the ASF repo?

Yea, that's a nice idea Romain. I would support trying to move code to master behind a flag ASAP. The thing to remember is this: if/when a feature branch moves to master it is too large to review all together. So the reviews to get things onto a feature branch need to be the same quality and

Re: Add a (temporary) Portable Flink branch to the ASF repo?

Maybe add a toggle in flinkoptions to activate it until it is tested and you are happy with it? Kind of --flinkExperimental=sdf,log,... (random values). This allows to hit master and continue to work on it. Le 13 avr. 2018 03:07, "Robert Bradshaw" a écrit : Thomas captures

Re: Add a (temporary) Portable Flink branch to the ASF repo?

Thomas captures exactly my concerns. I think we should look at getting everything we can into master (at least filing JIRAs, the work may take longer) and move what development we can there. What remains would be a collection of hacks (mostly in the SDKs, but it's not a feature branch 'cause we'd

Re: Add a (temporary) Portable Flink branch to the ASF repo?

+ 1 to capture in JIRA what needs to be done. The simplest path forward might be to reimplement/cherrypick'n'modify the changes onto master directly. We would then effectively just abandon the hacking branch and treat code there as a prototype. Although we would add components without end2end

Re: Add a (temporary) Portable Flink branch to the ASF repo?

Strong +1 on transitioning all development to the ASF repo. I think a straight move of the hacking branch may still be problematic though, because it sets the path to continue hacking vs. working towards a viable milestone that other contributors can base their work off. I would prefer a state

Re: Add a (temporary) Portable Flink branch to the ASF repo?

So I would be strongly in favour of adding it as a branch on the Apache repo. This way other folks are more likely to be able to help with the splitting up and merging process and also while Flink forward is behind us getting in the practice of doing feature branches on the ASF repo for

Re: Add a (temporary) Portable Flink branch to the ASF repo?

I suppose with the hackathon and flink forward behind us, I'm thinking we should start shifting gears more getting what we have into master in production state and less on continuing working on a hacking branch. If we think it'll fairly quick there's no big need to create an official branch, and

Re: Add a (temporary) Portable Flink branch to the ASF repo?

I would also be in favour of adding a branch to our main repo. A random branch on some personal GitHub account can seem a bit sketchy and adding a branch to our repo could make it more visible for people that are interested. > On 12. Apr 2018, at 15:29, Ben Sidhom wrote: >

Re: Add a (temporary) Portable Flink branch to the ASF repo?

I would say that *most* of it is not suitable for direct merging. There are several reasons for this: - Most changes are built on upstream PRs that are either not submitted or have been rebased before submission. - There are some very hacky changes in the Python and Java SDKs to get

Re: Add a (temporary) Portable Flink branch to the ASF repo?

How much of this is not suitable to merging into master directly (not as is, but as separate PRs)? On Thu, Apr 12, 2018 at 3:10 PM Ben Sidhom wrote: > Hey all, > I've been working on a proof-of-concept portable Flink runner with some other Beam contributors. We would like to

Add a (temporary) Portable Flink branch to the ASF repo?

Hey all, I've been working on a proof-of-concept portable Flink runner with some other Beam contributors. We would like to have a point of reference for the rest of the Beam community as we integrate this work into master. It currently lives under https://github.com/bsidhom/beam/tree/hacking-job-