Re: Add a (temporary) Portable Flink branch to the ASF repo?

refactoring before actually going in. On Thu, Apr 12, 2018 at 3:16 PM, Robert Bradshaw <rober...@google.com> wrote: > How much of this is not suitable to merging into master directly (not as > is, but as separate PRs)? > On Thu, Apr 12, 2018 at 3:10 PM Ben Sidhom <sid...@google.com

Add a (temporary) Portable Flink branch to the ASF repo?

Hey all, I've been working on a proof-of-concept portable Flink runner with some other Beam contributors. We would like to have a point of reference for the rest of the Beam community as we integrate this work into master. It currently lives under https://github.com/bsidhom/beam/tree/hacking-job-

Re: Portable Flink Runner plan

With respect to sharing code for rewriting pipelines: we've already written a few utilities for pipeline fusion and rewriting transforms to work with portable runners. Fusion functions the same way as in the ULR and is as simple as a single method call. However, two things prevent us from

Re: Portable Flink Runner plan

; > Btw, I feel that the thing currently called Runner, i.e. FlinkRunner will > go way in the long run and we will have FlinkJobService, SparkJobService > and whatnot, what do you think? > > Aljoscha > > > On 9. Feb 2018, at 01:31, Ben Sidhom <sid...@google.com> wrote: >

Re: [VOTE] Release 2.3.0, release candidate #3

gt; > Thanks for the update, I will close the vote then. > > Regards > JB > > On 02/15/2018 11:45 PM, Ben Sidhom wrote: > > I just successfully ran the quickstart on Flink 1.4.0 on Dataproc. > Should be > > good to go. > > > > > > On Thu, Feb 15, 2018 a

Re: [VOTE] Release 2.3.0, release candidate #3

I just successfully ran the quickstart on Flink 1.4.0 on Dataproc. Should be good to go. On Thu, Feb 15, 2018 at 9:21 AM Jean-Baptiste Onofré wrote: > Luke said he tested Flink locally. So, we have to test on a Yarn cluster. > > Regards > JB > > On 02/15/2018 06:16 PM,

Portable Flink Runner plan

Hey all, We're working on getting the portability framework plumbed through the Flink runner. The first iteration will likely only support batch and will be limited in its deployment flexibility, but hopefully it shouldn't be too painful to expand

Re: [POLL] Dropping support for Java 7 in Beam 2.3

Flink will be dropping Java 7 support in 1.5.0 (Java 8 is required for Scala 2.12+): https://issues.apache.org/jira/browse/FLINK-5005. This is another bump in favor of setting Java 8 as the project default. We'll eventually need to switch the Flink runner if we want it to stay up to date. On Tue,

Re: [VOTE] Fixing @yyy.com.INVALID mailing addresses

I'm not a PMC member, but this would be especially valuable if it propagated DKIM signatures properly. On Wed, Nov 22, 2017 at 3:25 PM, Lukasz Cwik wrote: > I have noticed that some e-mail addresses (notably @google.com) get > .INVALID suffixed onto it so

Re: [VOTE] Choose the "new" Spark runner

[ ] Use Spark 1 & Spark 2 Support Branch [X] Use Spark 2 Only Branch On Fri, Nov 17, 2017 at 9:46 AM, Ted Yu wrote: > [ ] Use Spark 1 & Spark 2 Support Branch > [X] Use Spark 2 Only Branch > > On Thu, Nov 16, 2017 at 5:08 AM, Jean-Baptiste Onofré

Re: New Contributor

Hey all, My name is Ben Sidhom. I'm an engineer at Google working on open source data processing on top of GCP. I hope to contribute to the runner portability effort along with Axel. On 2017-11-14 11:38, Axel Magnuson <a...@google.com.INVALID> wrote: > Hello All,> > > My nam