Re: IO configuration and @AutoValue future

Hi JB 2018-01-13 7:51 GMT+01:00 Jean-Baptiste Onofré : > Hi Romain, > > Clearly AutoValue is really convenient for developer. It reduces the > boilerplate of getter/setter for configuration. > > I'm not a big fan of an IOOptions, because it's a breaking change and I > think we

Re: IO configuration and @AutoValue future

Hi Romain, Clearly AutoValue is really convenient for developer. It reduces the boilerplate of getter/setter for configuration. I'm not a big fan of an IOOptions, because it's a breaking change and I think we gonna loose some flexibility for developers. POJO is basically what we did before

Re: [DISCUSS] Towards Beam 2.3.0

Hi Reuven, I proposed to cut on January 26th. I think it gives us enough time to: 1. stabilize the build (we have Jenkins failures) 2. do the change on the build and other parts for Java 8 3. include our current work Anyway, if we don't have time to include a change in 2.3.0, 2.4.0 will

[DISCUSS] State of the project

Hi everyone -- Apache Beam was established as a top-level project a year ago (on December 21, to be exact). This first anniversary is a great opportunity for us to look back at the past year, celebrate its successes, learn from any mistakes we have made, and plan for the next 1+ years. I’d like

Re: [DISCUSS] Towards Beam 2.3.0

When are we planning on cutting? On Tue, Jan 9, 2018 at 9:45 AM, Jean-Baptiste Onofré wrote: > Hi, > > yes, I proposed some weeks ago a best effort to have a release every two > months. > > I'm also volunteer to do those releases and iteratively improve the > process. > >

Re: IO configuration and @AutoValue future

Le 12 janv. 2018 20:49, "Kenneth Knowles" a écrit : Would you be able to put together a doc with a little more motivation and background on the feature, plus specific proposal? I think there's a lot of desire to build Beam pipelines in ways that are not programmatic and we can

Re: jackson to parse options?

Yes, we moves a few steps forward - was waiting before speaking of it but seems it led to it. Idea is to prepend a prefix per IO in the key. Currently the easiest working solution is to sanitize the transform/fn name (since we validate the unicity by default) and we are done. If not znough we can

Re: IO configuration and @AutoValue future

I know that 1) was suggested a long time ago in Google Cloud Dataflow. The concerns were of how to make it generic for any type of PTransform instead of just IO and also how to deal with the same PTransform being specified multiple times and having each option still be specified. This is why the

IO configuration and @AutoValue future

Hi guys I'd like to discuss the IO configuration. My goal is to be able to instrospect (or equivalent) the IO to instantiate them programmatically in a generic manner from a generic config - this is not yet linked to the system property topic but can benefit beam on this other topic too. Auto

Re: jackson to parse options?

2018-01-12 19:12 GMT+01:00 Lukasz Cwik : > > > On Fri, Jan 12, 2018 at 10:01 AM, Romain Manni-Bucau < > rmannibu...@gmail.com> wrote: > >> >> 2018-01-12 18:54 GMT+01:00 Lukasz Cwik : >> >>> Some concerns would be: >>> * How does a user discover what options can

Re: jackson to parse options?

On Fri, Jan 12, 2018 at 10:01 AM, Romain Manni-Bucau wrote: > > 2018-01-12 18:54 GMT+01:00 Lukasz Cwik : > >> Some concerns would be: >> * How does a user discover what options can be set and what values they >> take? >> > > Tempted to say "same as

Re: jackson to parse options?

2018-01-12 18:54 GMT+01:00 Lukasz Cwik : > Some concerns would be: > * How does a user discover what options can be set and what values they > take? > Tempted to say "same as today", what's the regression you would see? A --verbose can be nice thought to dump the DAG at startup

Re: jackson to parse options?

Some concerns would be: * How does a user discover what options can be set and what values they take? * System properties are global so how would you prevent system properties from conflicting within Apache Beam and with other non Apache Beam libraries that may rely on system properties? * There