Re: [VOTE] Release 2.3.0, release candidate #3

you can't once you closed the staging repo Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.

Re: [VOTE] Release 2.3.0, release candidate #3

oops sorry, read too fast (thanks to not align artifactId and folder names ;)): deploy#skip=true in the module :) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com&g

Re: [VOTE] Release 2.3.0, release candidate #3

it is not in the parent modules so completely skipped from the reactor Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibuc

Re: [VOTE] Release 2.3.0, release candidate #3

Ok, checked custom jobs on spark and direct runners + -parameters is usable + some advanced sdk-core integration usages (outside runners) - not sure where it fits the spreadsheet though. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.met

Re: [INFO] Gradle build is flaky on Jenkins

and never hardcoded IMHO) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in

Re: [VOTE] Release 2.3.0, release candidate #2

since it breaks only examples not sure it does worth yet another reroll (which means already a 2 weeks delay on the plan). Users will be affected the same anyway - and in an expected way until beam handles classloaders per transform. A note in the side is fine probably. Romain Manni-Bucau

Re: [VOTE] Release 2.3.0, release candidate #2

;al...@google.com > > <mailto:al...@google.com>> wrote: > > > > +1 > > > > I verified python quick start, mobile gaming examples, > streaming > > on Direct and Dataflow runner

Re: dependencies.txt in META-INF?

Was too much abused by libs and not supported everywhere :( Le 8 févr. 2018 22:39, "Lukasz Cwik" <lc...@google.com> a écrit : > It is unfortunate that setting Class-Path is so broken. > > On Wed, Feb 7, 2018 at 10:55 PM, Romain Manni-Bucau <rmannibu...@gmail.com >

Re: A 15x speed-up in local Python DirectRunner execution

Very interesting! Sounds like a sane way for beam future and I'm very happy it is consistent with the current Java experience: no need to interlace runners at the end, it makes design, code and user experience way better than trying to put everything in the direct runner :). Le 8 févr. 2018

Re: [VOTE] Release 2.3.0, release candidate #2

+1 (non-binding), thanks JB for the effort! Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.

Re: dependencies.txt in META-INF?

; > Class-Path: junit-3.8.2.jar > > > > On Wed, Feb 7, 2018 at 9:27 AM, Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> >> >> 2018-02-07 18:21 GMT+01:00 Lukasz Cwik <lc...@google.com>: >> >>> What kinds of features would this en

Re: dependencies.txt in META-INF?

his kind of thing already running a server from this kind of file typically. > > On Wed, Feb 7, 2018 at 7:23 AM, Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> Hi guys, >> >> I have a use case where I would resolve beam classpath programmatically

dependencies.txt in META-INF?

these metadata. Is it something the project could be interested in? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn &

Re: Schema-Aware PCollections revisited

Mon, Feb 5, 2018 at 1:55 PM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > None, Json-p - the spec so no strong impl requires - as record API and a > custom light wrapping for schema - like https://github.com/Talend > /component-runtime/blob/master/component-form/component- > f

Re: Schema-Aware PCollections revisited

ich json library are you thinking of? At least in Java, there's always been a problem of no good standard Json library. On Mon, Feb 5, 2018 at 12:03 PM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > > > Le 5 févr. 2018 19:54, "Reuven Lax" <re...@google.com> a écri

Re: coder evolutions?

impose that restriction as well). >> Instead, I'd keep thing the way they are, but offer a new Coder >> subclass that users can subclass if they want to write an "easy" Coder >> that does the delimiting for them (on encode and decode). We would >> point users to t

Re: Schema-Aware PCollections revisited

json friendly so you are back on json + metada so jsonschema+extension entry is strictly equivalent and as typed Reuven On Sun, Feb 4, 2018 at 9:31 AM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > You can handle integers using multipleOf: 1.0 IIRC. > Yes limitations are still he

Re: coder evolutions?

Would this work for everyone - can update the pr if so: If coder is not built in Prefix with byte size Else Current behavior ? Le 5 févr. 2018 19:21, "Romain Manni-Bucau" <rmannibu...@gmail.com> a écrit : > Answered inlined but I want to highlight beam is a

Re: coder evolutions?

but also not usable by most existing codecs out there. Even some jaxb or plain xml flavors dont work with it :(. Le 5 févr. 2018 18:46, "Robert Bradshaw" <rober...@google.com> a écrit : On Sun, Feb 4, 2018 at 6:44 AM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Hi g

Re: coder evolutions?

Thanks, created https://issues.apache.org/jira/browse/BEAM-3616 Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibuc

Re: [DISCUSS] State of the project: Culture and governance

t; the importance of community building and the Apache Way. PTAL I think > >> > this is really good and I don't see why others could disagree: > >> > > >> > https://flink.apache.org/how-to-contribute.html#how-to-becom > e-a-committer > >> > > >&g

Re: Schema-Aware PCollections revisited

esh it out. > > > > BTW in the past when I looked, Json schemas seemed to have some odd > limitations > > inherited from Javascript (e.g. no distinction between integer and > > floating-point types). Is that still true? > > > > Reuven > > > &g

Re: coder evolutions?

can't make changes to them right >> now. It would >> > > be great if in the next major version we were better prepared for >> evolution of >> > > coders, e.g. by having coders support a version marker or >> something like that, >> > > with an

Re: coder evolutions?

yep sadly :( how should we track it properly to not forget it for v3? (I dont trust jira much but if we don't have anything better...) when do we start beam 3? next week? :) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/>

Re: coder evolutions?

;). With this: no runner impact at all :). Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin

Re: Schema-Aware PCollections revisited

2018-02-04 17:53 GMT+01:00 Reuven Lax <re...@google.com>: > > > On Sun, Feb 4, 2018 at 8:42 AM, Romain Manni-Bucau <rmannibu...@gmail.com> > wrote: > >> >> 2018-02-04 17:37 GMT+01:00 Reuven Lax <re...@google.com>: >> >>> I'm not sure

Re: coder evolutions?

and this thread. We can postpone it but it would break later so for probably more users. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rm

Re: coder evolutions?

ktpub.com/application-development/java-ee-8-high-performance> 2018-02-04 17:34 GMT+01:00 Reuven Lax <re...@google.com>: > One question - does this change the actual byte encoding of elements? > We've tried hard not to do that so far for reasons of compatibility. > > Reuven

Re: Schema-Aware PCollections revisited

way to do it). So long story short the main work of this schema track is not only on using schema in runners and other ways but also starting to make beam consistent with itself which is probably the most important outcome since it is the user facing side of this work. > > On Sun, Feb 4, 2018 a

Re: Schema-Aware PCollections revisited

, it will >> be @Experimental for some time, so the API will not be fixed in stone. >> >> Any more comments on this approach before we start implementing a >> prototype? >> >> Reuven >> >> On Wed, Jan 31, 2018 at 1:12 PM, Romain Manni-Bucau < &g

Re: rename: BeamRecord -> Row

This is as true as the renaming is not needed so I guess the PR owner will decide ;). Thanks for the clarification. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com&g

Re: rename: BeamRecord -> Row

of both will be beneficial to beam in any case so better to ensure all parts of the projects move in the same direction instead of requiring yet another layer of conversion, no? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.ne

Re: rename: BeamRecord -> Row

Hi Shouldnt the discussion on schema which has a direct impact on this generic container be closed before any action on this? Le 3 févr. 2018 01:09, "Ankur Chauhan" a écrit : > ++ > > On Fri, Feb 2, 2018 at 1:33 PM Rafael Fernandez > wrote: > >> Very

Re: [DISCUSS] [Java] Private shaded dependency uber jars

well we can disagree on the code - it is fine ;), but the needed part of it by beam is not huge and in any case it can be forked without requiring 10 classes - if so we'll use another impl than the guava one ;). This is the whole point. Romain Manni-Bucau @rmannibucau <https://twitter.

Re: [DISCUSS] [Java] Private shaded dependency uber jars

thinking to coders here) but this can be another topic. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn &

Re: [DISCUSS] [Java] Private shaded dependency uber jars

idea, in particular cause the shade can be broken and requires to setup clirr or things like that and when it breaks you are done and need to fork it anyway. Limiting the dependencies for an API - as beam is - is always saner even if it requires a small fork of code. Romain Manni-Bucau @rmannibucau

Re: [DISCUSS] [Java] Private shaded dependency uber jars

more important is to get rid of it >>> for 'sdks/java/core'. We can then attack other areas afterwards. >>> >>> Other important idea would be to get rid of Protobuf in public APIs >>> like GCPIO and to better shade it from leaking into the runners. An >>> unexp

Re: [PROPOSAL] Switch from Guava futures vs Java 8 futures

+1 indeed Le 1 févr. 2018 21:34, "Eugene Kirpichov" a écrit : > Reducing dependency on Guava in favor of something Java-standard sounds > great, +1. > > On Thu, Feb 1, 2018 at 11:53 AM Reuven Lax wrote: > >> Unless there's something that doesn't work in

Re: drop scala....version from artifact ;)

Flink, Gearpump, Spark, and GCE provisioning are affected by this "issue". Dropping it if we never manage 2 versions is nicer for end users IMHO but I'm fine keeping it. Just would like to ensure it is uniform accross the whole projet. Romain Manni-Bucau @rmannibucau <https:

Re: [DISCUSS] [Java] Private shaded dependency uber jars

a map of list is fine and not a challenge we'll face long I hope ;) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibuc

Re: why org.apache.beam.sdk.util.UnownedInputStream fails on close instead of ignoring it

e are the ones owning the call since it is in a coder, no? Do >>> we assume people will badly implement coders? In this particular case we >>> can assume close() will be called by a framework I think. >>> What about swallowing one close() and fail on the second? >

Re: why org.apache.beam.sdk.util.UnownedInputStream fails on close instead of ignoring it

ng another layer and making smoothly work for most mappers. Anyway I can live with it but I'm a bit sad it closes the door to the easyness to write extensions. On Wed, Jan 31, 2018 at 12:04 PM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Hmm, here we are the ones owning the ca

Re: [VOTE] Release 2.3.0, release candidate #1

@ismael: any vote can be changes from -1 to +1 (or +-0) without additional delay Le 1 févr. 2018 03:15, "Lukasz Cwik" a écrit : > Note that a user reported TextIO being broken on Flink. > Thread is here: https://lists.apache.org/thread.html/ >

Re: [DISCUSS] [Java] Private shaded dependency uber jars

Why not dropping guava for all beam codebase? With java 8 it is quite easy to do it and avoid a bunch of conflicts. Did it in 2 projects with quite a good result. Le 1 févr. 2018 06:50, "Lukasz Cwik" a écrit : > Make sure to include the guava version in the artifact name so

Re: Tracking Sickbayed tests in Jira

@Ignore | sed 's#src/.*##' | sort -u # to count ignored tests $ find . -name '*.java' | xargs grep @Ignore | wc -l last one mixed with a loop and git allows to follow the evolution and check if it grows or decreases. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog

Re: Schema-Aware PCollections revisited

If you need help on the json part I'm happy to help. To give a few hints on what is very doable: we can add an avro module to johnzon (asf json{p,b} impl) to back jsonp by avro (guess it will be one of the first to be asked) for instance. Romain Manni-Bucau @rmannibucau <https://twitter.

Re: Schema-Aware PCollections revisited

you detail it please? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibuca

Re: why org.apache.beam.sdk.util.UnownedInputStream fails on close instead of ignoring it

Hmm, here we are the ones owning the call since it is in a coder, no? Do we assume people will badly implement coders? In this particular case we can assume close() will be called by a framework I think. What about swallowing one close() and fail on the second? Romain Manni-Bucau @rmannibucau

Re: Schema-Aware PCollections revisited

p > usage. > > Once 2.3.0 release is out, I will start to update the document with those > ideas, > and PoC. > > Thanks ! > Regards > JB > > On 01/30/2018 08:42 AM, Romain Manni-Bucau wrote: > > > > > > Le 30 janv. 2018 01:09, "Reuven Lax" &l

drop scala....version from artifact ;)

Hi guys since beam supports a single version of runners why not dropping the scala version from the artifactId? ATM upgrades are painful cause you upgrade beam version+ runner artifactIds. wdyt? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog

Re: IO plans?

Thanks JB, this is great news since they are highly used IO in the industry and really awaited now. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://g

IO plans?

Hi guys, is there a plan for future IO and some tracking somewhere? I particularly wonder if there are plans for a HTTP IO and common server IO like SFTP, SSH, etc... Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

Re: untyped pipeline API?

at that point). The SDK has no good > way of knowing what those decisions will be, so needs to conservatively > assume it could happen anywhere. > > On Tue, Jan 30, 2018 at 1:31 PM, Romain Manni-Bucau <rmannibu...@gmail.com > > wrote: > >> Hmm starts to smell like the

Re: untyped pipeline API?

> Where the split points are depends on the runner. Runners are free to > split at any point (and often do to prevent cycles from appearing in the > graph). > > On Tue, Jan 30, 2018 at 1:27 PM, Romain Manni-Bucau <rmannibu...@gmail.com > > wrote: > >> I kind of

Re: [DISCUSSION] Add hint/option on PCollection

We slightly >>>>>> break >>>>>> this already today with PCollection.setCoder, and that has caused some >>>>>> problems. Hints can be set on PTransforms though, and propagate to >>>>>> that >>

Re: untyped pipeline API?

ally only serialize when >> necessary. So the default in Beam is that the thing you want to happen is >> already done. There are some corner cases when you get to the portability >> framework but I am pretty sure it already works this way. If you show what >> is a PTransform and PCollect

Re: untyped pipeline API?

Person -> JSON -> Map<String, String> is fully enough cause there is equivalence of JSON in this example. In other words if an coder output is readable from another coder input, the java strong typing doesn't know about it and can enforce some fake steps. Romain Manni-Bucau @rmannibucau

Re: [DISCUSSION] Add hint/option on PCollection

; <mailto:j...@nanthrax.net>> wrote: >>> >>> Maybe I should have started the discussion on the user mailing list: >>> it would be great to have user feedback on this, even if I got your >>> points. >>> >>> Sometime, I have

Re: [DISCUSSION] Add hint/option on PCollection

Hmm, can work for pipeline hints but for transform hints we would need: p.apply(AddHint.of(.).wrap(originalTransform)) Would work for me too. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

untyped pipeline API?

and enforce the user to use an intermediate state to be typed. Is there already a way to avoid these useless round trips? Said otherwise: how to handle coders transitivity? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

Re: why org.apache.beam.sdk.util.UnownedInputStream fails on close instead of ignoring it

I get the issue but I don't get the last part. Concretely we can support any lib by just removing the exception in the close, no? What would be the issue? No additional wrapper, no lib integration issue. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog

Re: [DISCUSSION] Add hint/option on PCollection

aven't already covered. > There are several cases, even in the direct runner to be able to industrialize it: - use that particular executor instance - debug these infos for that transform etc... As a high level design I think it is good to bring hints to beam to avoid to add ad-hoc solution ea

why org.apache.beam.sdk.util.UnownedInputStream fails on close instead of ignoring it

Hi guys, All is in the subject ;) Rational is to support any I/O library and not fail when the close is encapsulated. Any blocker to swallow this close call? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

Re: [DISCUSSION] Add hint/option on PCollection

. It should really be about runing a runner execution (like the schema in spark). Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibuc

Re: Should we have a predictable test run order?

nd you can configure this sequence in idea. Not perfect but better than hiding the issue probably. Also running "clean" enforces inodes to change and increase the probability to reproduce it on linux. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <h

Re: Should we have a predictable test run order?

Hi Daniel, As a quick fix it sounds good but doesnt it hide a leak or issue (in test setup or in main code)? Long story short: using a random order can allow to find bugs faster instead of hiding them and discover them randomly adding a new test. That said, good point to have it configurable

Re: Schema-Aware PCollections revisited

Le 30 janv. 2018 01:09, "Reuven Lax" <re...@google.com> a écrit : On Mon, Jan 29, 2018 at 12:17 PM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Hi > > I have some questions on this: how hierarchic schemas would work? Seems it > is not really support

Re: Schema-Aware PCollections revisited

Hi I have some questions on this: how hierarchic schemas would work? Seems it is not really supported by the ecosystem (out of custom stuff) :(. How would it integrate smoothly with other generic record types - N bridges? Concretely I wonder if using json API couldnt be beneficial: json-p is a

Re: [PROPOSAL] Add a blog post for every new release

. This way we have the best of both worlds :). Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.c

Re: [HEADS UP] Preparing Beam 2.3.0

Will be online this afternoon. Ping me if you need help. Le 28 janv. 2018 11:36, "Jean-Baptiste Onofré" <j...@nanthrax.net> a écrit : > By the way, I created BEAM-3551 for the tracking. > > I will work on it today. > > Regards > JB > > On 01/28/2018 09

Re: [HEADS UP] Preparing Beam 2.3.0

Hi guys Out if curiosity, can -parameters (javac) be part of the 2.3 if not already? Le 27 janv. 2018 18:39, "Jean-Baptiste Onofré" a écrit : > Hi Reuven, > > I gonna bump 3392 and 3087 to 2.4.0. For the PR, yes Eugene did a first > round > review, I will work on it now. > >

Re: Great work closing PRs for 2.3.0 release

Cleanup done on my side - and thanks JB to have caught one in the night for me. Thanks for the heads up, it seems it is easy to forget PR on github ;). Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

Re: Gradle / Mvn diff

l like javac. The task the represents compilation would > need to be broken up into smaller tasks with smaller source sets to speed > up compilation of really large modules. > > On Wed, Jan 24, 2018 at 11:12 PM, Romain Manni-Bucau < > rmannibu...@gmail.com> wrote: &

Re: Gradle / Mvn diff

ncy-driven build (which works) and >> incremental build (which seems not to, at least right now?). >> >> On Wed, Jan 24, 2018 at 2:24 PM, Romain Manni-Bucau < >> rmannibu...@gmail.com> wrote: >> >>> Hmm, I'll try to refine it then next time we w

Re: Gradle / Mvn diff

Hmm, I'll try to refine it then next time we work with Ismael but can be a setup issue or a human (bad command) issue at the end. Thanks for the help, will make next iteration way easier probably :) Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog

Re: Gradle / Mvn diff

-> fulfilled and clarified now (just a doc issue I think), fast cycle skipping not mandatory tasks like style related ones Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.co

Re: Gradle / Mvn diff

ssed? Also not sure about the last point and how gradle helps here - it is rather the opposite due to the way it loads it model IMHO - so not sure what would be the consequence in terms of action(s) but can have missed the point. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannib

Re: Gradle / Mvn diff

2018-01-24 0:08 GMT+01:00 Lukasz Cwik <lc...@google.com>: > > > On Tue, Jan 23, 2018 at 2:51 PM, Romain Manni-Bucau <rmannibu...@gmail.com > > wrote: > >> >> >> 2018-01-23 23:44 GMT+01:00 Kenneth Knowles <k...@google.com>: >> >

Re: Gradle / Mvn diff

2018-01-23 23:44 GMT+01:00 Kenneth Knowles <k...@google.com>: > Romain - really good point about docs getting out of date. > > > On 01/23/2018 09:29 AM, Romain Manni-Bucau wrote: >> >> Yep, >> >> >> >> a compromise can be

Re: [DISCUSS] State of the project: Culture and governance

view, even if it requires a core dev commit after a merge IMHO (while it doesnt break and bring something it is good to merge kind of rule). Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordp

Re: JUnit 5 review

this extension mecanism, it just allows to instantiate it manually in a static field instead of using the meta annotation which keep the extension of this PR valuable. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Ol

Re: JUnit 5 review

Oki JB, Will implement it on my side until beam supports it then. Thanks for the feedback. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://g

JUnit 5 review

Hi guys, Anyone able to have a look to the JUnit 5 PR ( https://github.com/apache/beam/pull/4360)? Worse case a "yes we'll move this direction" or "no we don't care about JUnit 5 for now" feedback would be very valuable for me. Thanks, Romain Manni-Bucau @rmannibucau

Re: Gradle / Mvn diff

replace('Runner', '')}") { def preCommitBeamTestPipelineOptions = [ "--project=apache-beam-testing", "--tempRoot=gs://temp-storage-for-end-to-end-tests", Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rm

Re: Gradle / Mvn diff

gt; a écrit : On Mon, Jan 22, 2018 at 10:03 PM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > @Kenneth: why not dropping the doc for a script with comments in the > project? A "RUNME.sh" ;). > That's cool, too, but also any shell one liner can be a gradle one liner

Re: Gradle / Mvn diff

12:20 PM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Hi > > As mentionned in another thread, Im sending this mail to report some > differences between maven and gradle setups - percieved as regressions from > this side of the fence: > > 1. Parent versions are no

Gradle / Mvn diff

Hi As mentionned in another thread, Im sending this mail to report some differences between maven and gradle setups - percieved as regressions from this side of the fence: 1. Parent versions are not usable in children as variables - btw why not putting them in gradle.properties as ut is often

Re: IO configuration and @AutoValue future

@Lukasz: not really since if the parameters changes then the new version will get the new data so this is not a constraint on beam but on the configuration storage which must handle anyway versions compatibility somehow so not a big deal IMHO. Romain Manni-Bucau @rmannibucau <https://twitter.

Re: IO configuration and @AutoValue future

@JB: thought about another option which can be almost hurtless for beam: 1. we ensure all "config" classes are public (would avoid nasty hacks) 2. while migrating to java 8 you activate the javac -parameters flag does it sound better? Romain Manni-Bucau @rmannibucau <https:

Re: IO configuration and @AutoValue future

gt; specific annotation processor that create another POJO class, not actually > used in the IO code, but "describing" the configuration for automation > projects. This POJO will be public, no final. > > WDYT ? > > Regards > JB > > On 12/01/2018 19:26, Romain Man

Re: IO configuration and @AutoValue future

nting to chain them can be nice too, so would be enforcing a single ConfigPojo for IO configuration. Would that work? > > Regards > JB > > On 12/01/2018 19:26, Romain Manni-Bucau wrote: > >> Hi guys >> >> I'd like to discuss the IO configuration. >>

Re: IO configuration and @AutoValue future

on friendly unless we know this will never need to grow into more. Saw some working like xbean/tomee and others failing due to too much complexity for tye config. Im sure well make it if we want . This is about config, not yet about generic runtimes ;) Kenn On Fri, Jan 12, 2018 at 10:26 AM, Ro

Re: jackson to parse options?

gnificantly more detailed in what your suggesting. With the system property approach, how would a user use the same IO multiple times? On Fri, Jan 12, 2018 at 10:18 AM, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > > 2018-01-12 19:12 GMT+01:00 Lukasz Cwik <lc...@google.com>

IO configuration and @AutoValue future

much as 1 without getting all the benefit to have an agnostic config and the tooling we can build on top of it 5. probably others Wdyt? Personally I really like 1 cause it starts to create a clean programming model we can then build other features on. Romain Manni-Bucau @rmannibucau <https:/

Re: jackson to parse options?

2018-01-12 19:12 GMT+01:00 Lukasz Cwik <lc...@google.com>: > > > On Fri, Jan 12, 2018 at 10:01 AM, Romain Manni-Bucau < > rmannibu...@gmail.com> wrote: > >> >> 2018-01-12 18:54 GMT+01:00 Lukasz Cwik <lc...@google.com>: >> >>> Some concer

Re: jackson to parse options?

> > Note that the system property usage is only for integration testing and > not meant for users writing pipelines. > > On Thu, Jan 11, 2018 at 9:38 PM, Romain Manni-Bucau <rmannibu...@gmail.com > > wrote: > >> Hmm, here is my thought: allowing main options settings i

Re: jackson to parse options?

user add String[] getMainArgs to TestPipelineOptions and not do the >> convertToArgs hackiness for ITs that use main(). >> >> On Thu, Jan 11, 2018 at 1:53 PM, Romain Manni-Bucau < >> rmannibu...@gmail.com> wrote: >> >>> Some inputs - maybe comments? - would b

Re: jackson to parse options?

interface for the >> config. >> >> I'm not fully clear on all the jackson usage yet, there are some round >> trips (PO -> json -> PO) which are weird without more knowledge. >> >> >> Romain Manni-Bucau >> @rmannibucau <https://twitter.com/rmannibuc

Re: Gradle status

ess (or portion thereof) and will be removed." >> >> Once the Gradle presubmit can replace a particular Maven presubmit, >> we can >> remove the Maven version. >> >> Can you file JIRAs for the things you suspect are shaded incorrectly >> by our

Re: jackson to parse options?

son -> PO) which are weird without more knowledge. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https:

<    1   2   3   4   5   >