Re: [HEADS UP] Preparing Beam 2.3.0

2018-01-23 Thread Jean-Baptiste Onofré
blockers to me. On Tue, Jan 23, 2018 at 1:39 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: Hi guys, Some days ago, I proposed to start Beam 2.3.0 around January 26th. So, we are few days from this date. As a best effort, can you pl

Re: [HEADS UP] Preparing Beam 2.3.0

2018-01-23 Thread Jean-Baptiste Onofré
Great ! I'm starting the Jira triage ;) Thanks ! Regards JB On 23/01/2018 19:51, Reuven Lax wrote: Sounds good. Also many things that are currently flagged as 2.3.0 don't appear to be actual release blockers to me. On Tue, Jan 23, 2018 at 1:39 AM, Jean-Baptiste Onofré <j...@nanthrax.

Re: [DISCUSS] State of the project: Culture and governance

2018-01-23 Thread Jean-Baptiste Onofré
for example. > https://flink.apache.org/how-to-contribute.html#how-to-become-a-committer > > If you have some other ideas or issues that you would like to discuss > please share them in this thread. > -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: JUnit 5 review

2018-01-23 Thread Jean-Baptiste Onofré
au > @rmannibucau <https://twitter.com/rmannibucau> |  Blog > <https://rmannibucau.metawerx.net/> | Old Blog > <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | > LinkedIn <https://www.linkedin.com/in/rmannibucau> > > 2018-0

[HEADS UP] Preparing Beam 2.3.0

2018-01-23 Thread Jean-Baptiste Onofré
JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: JUnit 5 review

2018-01-23 Thread Jean-Baptiste Onofré
Blog > <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | > LinkedIn <https://www.linkedin.com/in/rmannibucau> -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Gradle / Mvn diff

2018-01-23 Thread Jean-Baptiste Onofré
mannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | > LinkedIn <https://www.linkedin.com/in/rmannibucau> > > 2018-01-23 8:45 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net > <mailto:j...@nanthrax.net>>: > > Hi Romain, >

Re: Gradle / Mvn diff

2018-01-22 Thread Jean-Baptiste Onofré
t; So I was proposing a mapping from "full sentence + description" to one > liner > to help people navigate the targets that we set up. Some web page or doc > that people can just quickly scan to find out to do common things, easier > than groovy or XML. > > Kenn > > -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [DISCUSS] State of the project: Feature roadmap for 2018

2018-01-22 Thread Jean-Baptiste Onofré
gt; > Ben > > > [1] > https://lists.apache.org/thread.html/f750f288af8dab3f468b869bf5a3f473094f4764db419567f33805d0@%3Cdev.beam.apache.org%3E > > [2] > https://lists.apache.org/thread.html/01a80d62f2df6b84bfa41f05e15fda900178f882877c294fed8be91e@%3Cdev.beam.apache.org%3E -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Switching to Java 8

2018-01-22 Thread Jean-Baptiste Onofré
uild is higher priority because it > enables > people to start modernizing the code (e.g. FileIO) and it'd be good to do > that before 2.3 cut. I wasn't able to find the PR you mentioned > in https://github.com/apache/beam/pulls , which one is it? > > On Mon, J

Re: Fwd: Google Summer of Code 2018 is coming

2018-01-22 Thread Jean-Baptiste Onofré
> > [1] https://summerofcode.withgoogle.com/ > <https://summerofcode.withgoogle.com/> > [2] http://community.apache.org/guide-to-being-a-mentor.html > <http://community.apache.org/guide-to-being-a-mentor.html> > [3] http://s.apache.org/gsoc2018ideas <http://s.apache.org/gsoc2018ideas> > [4] http://community.apache.org/gsoc.html > <http://community.apache.org/gsoc.html> -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: IO configuration and @AutoValue future

2018-01-19 Thread Jean-Baptiste Onofré
06:41 PM, Lukasz Cwik wrote: Note that using the -parameters flag in javac will require that we never change parameter names inside methods increasing the backwards compatibility burden. On Thu, Jan 18, 2018 at 12:33 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net&g

Re: IO configuration and @AutoValue future

2018-01-18 Thread Jean-Baptiste Onofré
configAsPrimitivesMap) Im happy whatever solution is used. Le 14 janv. 2018 17:19, "Jean-Baptiste Onofré" <j...@nanthrax.net <mailto:j...@nanthrax.net>> a écrit : Hi Romain, I think the missing thing for automation projects is probably mor

Re: Jenkins build is still unstable: beam_PostCommit_Java_MavenInstall #5661

2018-01-16 Thread Jean-Baptiste Onofré
t> -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Switching to Java 8

2018-01-15 Thread Jean-Baptiste Onofré
t;On Tue, Jan 9, 2018, 2:51 AM Jean-Baptiste Onofré <j...@nanthrax.net> >wrote: > >> Actually, it's part of the build and I will "expand" the java version >in >> the >> enforcer. >> >> Regards >> JB >> >> On 01/09/2018 11:46 AM, E

Re: IO configuration and @AutoValue future

2018-01-14 Thread Jean-Baptiste Onofré
Hi Romain, I think the missing thing for automation projects is probably more around "documentation" for the setters/getters. So, why not: 1. we don't change the usage and AutoValue itself 2. we can imagine to add a new set of annotations in IO Common with a specific annotation processor

Re: [DISCUSS] State of the project

2018-01-14 Thread Jean-Baptiste Onofré
Hi Davor, Thanks a lot for this e-mail. I would like to emphasize two areas where we have to improve: 1. Apache way and community. We still have to focus and being dedicated on our communities (both user & dev). Helping, encouraging, growing our communities is key for the project. Building

Re: IO configuration and @AutoValue future

2018-01-12 Thread Jean-Baptiste Onofré
Hi Romain, Clearly AutoValue is really convenient for developer. It reduces the boilerplate of getter/setter for configuration. I'm not a big fan of an IOOptions, because it's a breaking change and I think we gonna loose some flexibility for developers. POJO is basically what we did before

Re: [DISCUSS] Towards Beam 2.3.0

2018-01-12 Thread Jean-Baptiste Onofré
will happen two months later ;) Regards JB On 13/01/2018 00:08, Reuven Lax wrote: When are we planning on cutting? On Tue, Jan 9, 2018 at 9:45 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: Hi, yes, I proposed some weeks ago a best effort to ha

Re: Gradle status

2018-01-11 Thread Jean-Baptiste Onofré
<https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Gradle status

2018-01-11 Thread Jean-Baptiste Onofré
ptions I see to exit that state? wdyt? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/

Re: FileBasedSource does not match files on HDFS

2018-01-10 Thread Jean-Baptiste Onofré
No problem, happy to help ;) Regards JB On 01/10/2018 11:15 AM, Shashank Prabhakara wrote: It was the missing hdfs filesystem extension dependency. Thanks Jean-Baptiste. Much appreciated. Regards, Shashank On Wed, Jan 10, 2018 at 2:09 PM, Jean-Baptiste Onofré <j...@nanthrax.net <ma

Re: FileBasedSource does not match files on HDFS

2018-01-10 Thread Jean-Baptiste Onofré
d to exclude org.apache.hadoop:* and com.google.cloud.bigdataoss:* from shaded jar for version compat. Appreciate any help. Regards, Shashank -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Dataflow runner examples build fail

2018-01-09 Thread Jean-Baptiste Onofré
<yuzhih...@gmail.com <mailto:yuzhih...@gmail.com>> wrote: +1 ---- Original message From: Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> Date

Re: [DISCUSS] Towards Beam 2.3.0

2018-01-09 Thread Jean-Baptiste Onofré
as the need arises...) On Mon, Jan 8, 2018 at 7:04 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: > Hi Romain, > > no problem: let's try a bes

Re: [DISCUSS] Towards Beam 2.3.0

2018-01-08 Thread Jean-Baptiste Onofré
need arises...) On Mon, Jan 8, 2018 at 7:04 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: > Hi Romain, > > no problem: let's try a best effort and define the target version in the > Jira. > > Regards

Re: [DISCUSS] Towards Beam 2.3.0

2018-01-08 Thread Jean-Baptiste Onofré
u.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> 2018-01-08 15:37 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>>: Hi guys, In a previous discussion thread, we a

[DISCUSS] Towards Beam 2.3.0

2018-01-08 Thread Jean-Baptiste Onofré
to propose target Beam 2.3.0 for end January/beginning of February. I'm volunteer to do this release. Thoughts ? Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Switching to Java 8

2018-01-08 Thread Jean-Baptiste Onofré
d task from the CI - Update the docker dev build images (I will take this one since reproducible build is my pet project) - Upgrade the IOs who were still in older versions because of client compatibility. I remember SolfIO was one case but probably there are others. On

Re: Switching to Java 8

2018-01-08 Thread Jean-Baptiste Onofré
compatibility. I remember SolfIO was one case but probably there are others. On Mon, Jan 8, 2018 at 7:49 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Yes, that's the plan: build first, example "merge" after. Regards JB On 01/08/2018 07:43 AM, Eugene Kirpichov wrote: Soun

Re: Switching to Java 8

2018-01-08 Thread Jean-Baptiste Onofré
. On Mon, Jan 8, 2018 at 7:49 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Yes, that's the plan: build first, example "merge" after. Regards JB On 01/08/2018 07:43 AM, Eugene Kirpichov wrote: Sounds great, thanks! Probably best done as 2 separate steps, because after updating t

Dataflow runner examples build fail

2018-01-08 Thread Jean-Baptiste Onofré
Hi guys, The PRs and nightly builds are failing due to an issue with the dataflow platform: it seems we have a disk quota exceeded on the us-central1 region. I would like to do a clean out and increase the quota a bit. Thoughts ? Thanks Regards JB -- Jean-Baptiste Onofré jbono...@apache.org

Re: Switching to Java 8

2018-01-07 Thread Jean-Baptiste Onofré
t 10:38 PM Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: Hi Eugene, I'm taking the build update: Maven/Gradle with enforcer + merge of the examples all together. Regards JB On 01/08/2018 07:34 AM, Eugene Kirpichov wrote: >

Re: Switching to Java 8

2018-01-07 Thread Jean-Baptiste Onofré
java8 examples" and "java8 tests" into respective non-"java8" modules - Organize an effort to modernize code to use Java 8 constructs where appropriate. Especially important to modernize examples. To a large extent this can probably be automated with an IDE. Anything el

Re: jackson to parse options?

2018-01-07 Thread Jean-Baptiste Onofré
; | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: JUnit 5 TestPipeline

2018-01-06 Thread Jean-Baptiste Onofré
hub.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> 2018-01-05 15:45 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>>: It sounds good, thanks. Let me add as reviewer on the PR.

Re: JUnit 5 TestPipeline

2018-01-05 Thread Jean-Baptiste Onofré
<https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> 2018-01-05 11:43 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.n

Re: JUnit 5 TestPipeline

2018-01-05 Thread Jean-Baptiste Onofré
tps://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> -- Jean-Baptiste Onofré jbono...@apache.org http://b

Re: Euphoria Java 8 DSL - proposal

2018-01-02 Thread Jean-Baptiste Onofré
Great ! Thanks ! Regards JB On 01/03/2018 07:29 AM, David Morávek wrote: Hello JB, Perfect! I'm already on the Beam Slack workspace, I'll contact you once I get to the office. Thanks! D. On Wed, Jan 3, 2018 at 6:19 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthr

Re: [INFO] Spark runner itests fail

2018-01-02 Thread Jean-Baptiste Onofré
By the way, nightly build is not impacted, nor all PRs. I keep you posted. Regards JB On 01/02/2018 07:38 PM, Kenneth Knowles wrote: Figure anything out? If it keeps happening, maybe we could gather knowledge about it on a JIRA. Kenn On Sun, Dec 31, 2017 at 11:18 PM, Jean-Baptiste Onofré &l

Re: Testing, automation, and pipeline rollout in a CI/CD world

2018-01-02 Thread Jean-Baptiste Onofré
data is bad and try to be backwards compatible", or are there any Beam features that help with this? Thanks, Charles Allen -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [INFO] Spark runner itests fail

2018-01-02 Thread Jean-Baptiste Onofré
Hi Kenn, I will create the Jira and the corresponding PR today. Regards JB On 01/02/2018 07:38 PM, Kenneth Knowles wrote: Figure anything out? If it keeps happening, maybe we could gather knowledge about it on a JIRA. Kenn On Sun, Dec 31, 2017 at 11:18 PM, Jean-Baptiste Onofré &l

Re: Euphoria Java 8 DSL - proposal

2018-01-02 Thread Jean-Baptiste Onofré
Euphoria API with Beam's runners as much natively as possible. I hope I didn't make the subject even more unclear, if so, I'll be happy to explain anything in more detail. :-)    Jan On 12/18/2017 04:08 PM, Jean-Baptiste Onofré wrote: Hi J

[INFO] Spark runner itests fail

2017-12-31 Thread Jean-Baptiste Onofré
Hi, The Spark runner itests fails on Jenkins for some pull requests. I'm trying to reproduce locally (it seems to be random). I keep you posted. Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Happy new year

2017-12-31 Thread Jean-Baptiste Onofré
Hi beamers, I wish you a great and happy new year ! Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Python SDK build is failing

2017-12-24 Thread Jean-Baptiste Onofré
; - When this happened, Cham tried to see a way to push a different history, but it seems the only way would be to roll back which was probably messier. Happy holidays! María On Fri, Dec 15, 2017 at 7:59 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j..

Re: FileIO.write() landed on master - give it a try

2017-12-19 Thread Jean-Baptiste Onofré
have access to features like dynamic destinations, and now they do: you can use TFRecordIO.sink() and XmlIO.sink() with FileIO.write() or writeDynamic(). Thanks to +Reuven Lax <mailto:re...@google.com> and +Chamikara Jayalath <mailto:chamik...@google.com> for reviews. -- Jean-Bap

Re: remove avro from core?

2017-12-19 Thread Jean-Baptiste Onofré
the Jackson version because if it is this the case I don’t think the things will move quickly: https://issues.apache.org/jira/browse/AVRO-1126 On Tue, Dec 19, 2017 at 11:23 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Agree, it's what I meant by "core transforms". Regards J

Re: remove avro from core?

2017-12-19 Thread Jean-Baptiste Onofré
this POJO work" and Avro is the only way we give them. This was part of my argument in the schema docs. However the status quo is that they use Avro here. Reuven On Tue, Dec 19, 2017 at 1:32 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: Hi Roma

Re: [INFO] Spark runner updated to Spark 2.2.1

2017-12-18 Thread Jean-Baptiste Onofré
By the way, Flink has been updated to Flink 1.4.0 as well (as the artifacts already used Scala 2.11). Regards JB On 12/18/2017 11:50 AM, Jean-Baptiste Onofré wrote: Hi all, We are pleased to announce that Spark 2.x support in Spark runner has been merged this morning. It supports Spark

Re: Euphoria Java 8 DSL - proposal

2017-12-18 Thread Jean-Baptiste Onofré
detail. :-)   Jan On 12/18/2017 04:08 PM, Jean-Baptiste Onofré wrote: Hi Jan, Thanks for your answers. However, they confused me ;) Regarding what you replied, Euphoria seems like a programming model/SDK "close" to Beam more than a DSL on top of an existing Beam SDK. Am I wrong ? R

Re: Euphoria Java 8 DSL - proposal

2017-12-18 Thread Jean-Baptiste Onofré
on supporting the future of this DSL. Thanks and congrats again ! Ismaël On Mon, Dec 18, 2017 at 10:12 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Depending of the donation, you would need ICLA for each contributor, and CCLA in addition of SGA. We can sync with Davor and I for the lega

[INFO] Spark runner updated to Spark 2.2.1

2017-12-18 Thread Jean-Baptiste Onofré
, as planned, that Spark 2.x support will be included in next Beam 2.3.0 release. Now, we are going to work on improvements in the Spark runner. If you have any issue with the Spark runner, please let us know. Thanks ! Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net

Re: Euphoria Java 8 DSL - proposal

2017-12-18 Thread Jean-Baptiste Onofré
end can help, including JB or myself. Davor On Sun, Dec 17, 2017 at 10:13 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: Hi David, Generally speaking, having different fluent DSL on top of the Beam

Re: Euphoria Java 8 DSL - proposal

2017-12-17 Thread Jean-Baptiste Onofré
to hearing from you, David -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [DISCUSS] Migrating Apache Beam PreCommits/PostCommits to Gradle Criteria

2017-12-16 Thread Jean-Baptiste Onofré
stCommit_Java_ValidatesRunner_Spark,42,80.95,60.5 >>> beam_PostCommit_Python_ValidatesRunner_Dataflow,42,80.95,23.3 >>> beam_PostCommit_Python_Verify,42,73.81,72.2 >>> beam_PreCommit_Go_MavenInstall,140,96.43,5.1 >>> beam_PreCommit_Java_GradleBuild,139,36.69,129.6 >>> beam_PreCommit_Java_MavenInstall,136,39.71,201.2 >>> beam_PreCommit_Python_MavenInstall,140,51.43,123.8 > > -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Python SDK build is failing

2017-12-15 Thread Jean-Baptiste Onofré
ssing-time timers and clock injection That would great to provide some details and, when possible, squash with a Jira reference. It helps to understand the change and eventually track breaking changes. Thanks ! Regards JB On 12/13/2017 02:49 PM, Jean-Baptiste Onofré wr

Re: Python SDK build is failing

2017-12-15 Thread Jean-Baptiste Onofré
nd eventually track breaking changes. Thanks ! Regards JB On 12/13/2017 02:49 PM, Jean-Baptiste Onofré wrote: Hi guys, It seems that following commit breaks the Python SDK build: commit 60dd7c4035a8d320b01823cbd4c959d0ef09cf26 Author: Robert Bradshaw <rober...@gmail.com> Date:   Tue Dec 12 17:

Re: Python SDK build is failing

2017-12-13 Thread Jean-Baptiste Onofré
found anything out. On Wed, Dec 13, 2017 at 5:49 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Hi guys, It seems that following commit breaks the Python SDK build: commit 60dd7c4035a8d320b01823cbd4c959d0ef09cf26 Author: Robert Bradshaw <rober...@gmail.com> Date: Tue Dec 12 1

Re: Python SDK build is failing

2017-12-13 Thread Jean-Baptiste Onofré
basically an no-op (mostly just added logging). I'll investigate, but would be interested if you found anything out. On Wed, Dec 13, 2017 at 5:49 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Hi guys, It seems that following commit breaks the Python SDK build:

Re: Vendor files in Go SDK

2017-12-13 Thread Jean-Baptiste Onofré
support for Go, but evidently I forgot something. Sorry about the trouble. Can you clarify which files you mean? Thanks,  Henning On Tue, Dec 12, 2017 at 11:44 PM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: Hi guys, I just noticed that the

Python SDK build is failing

2017-12-13 Thread Jean-Baptiste Onofré
. I'm investigating to provide a PR to fix that. Sorry for the inconvenience. Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

[PROPOSAL] Create Beam View on Jenkins

2017-12-13 Thread Jean-Baptiste Onofré
of the Beam jobs. I did something similar for Karaf: https://builds.apache.org/view/K/view/Karaf/ It doesn't change anything on the jobs configuration by itself, it's just a Jenkins "grouping" configuration. Thoughts ? Regards JB -- Jean-Baptiste Onofré jbono...@apach

Vendor files in Go SDK

2017-12-12 Thread Jean-Baptiste Onofré
contributor update me about that ? On the other hand, the rat plugin configuration should be updated to exclude those vendor files. I'm preparing a PR about that. Thanks, Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: A personal update

2017-12-12 Thread Jean-Baptiste Onofré
more on this next year, but until then if you have a data processing problem or an Apache Beam question, I’d love to hear from you ;-). Thanks -- and so happy to be back! Davor -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Report to the Board, December 2017 edition

2017-12-12 Thread Jean-Baptiste Onofré
or contribute content, as appropriate. I'll submit the report sometime in the next 24 hours. Thanks! Davor [1] https://docs.google.com/document/d/17AFM3iLXH8YyUc7FiYZS5YFIOHymFWbbEJ5Ea2ySnkA/ -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [DISCUSS] [Java] Private shaded dependency uber jars

2017-12-11 Thread Jean-Baptiste Onofré
g as I spent another evening waiting for slow builds to try to confirm changes to brittle poms. Kenn -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [DISCUSSION] Runner agnostic metrics extractor?

2017-12-11 Thread Jean-Baptiste Onofré
., if a runner wanted to report the watermark, it could push that in directly). -- Ben On Mon, Nov 27, 2017 at 9:06 AM Jean-Baptiste Onofré<j...@nanthrax.net> wrote: Hi all, Etienne forgot to mention that we started a PoC about that. What I started is to wrap the Pipeline creation to include a

Re: Apache Beam, version 2.2.0

2017-12-04 Thread Jean-Baptiste Onofré
and, say, if we receive 5% or fewer votes for option 3 after keeping it open for 2 weeks, then adopt Java 8 without a major version change. WDYT? On Mon, Dec 4, 2017 at 8:34 PM Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wro

Re: Apache Beam, version 2.2.0

2017-12-04 Thread Jean-Baptiste Onofré
browse/BEAM-3285 On Mon, Dec 4, 2017 at 1:58 AM Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: Just an important note that we forgot to mention. !! The 2.2.0 release will be the last one supporting Spark 1.x and Java 7 !! Starting from Bea

Re: Apache Beam, version 2.2.0

2017-12-04 Thread Jean-Baptiste Onofré
Just an important note that we forgot to mention. !! The 2.2.0 release will be the last one supporting Spark 1.x and Java 7 !! Starting from Beam 2.3.0, the Spark runner will work only with Spark 2.x and we will focus only Java 8. Regards JB On 12/04/2017 10:15 AM, Jean-Baptiste Onofré

Re: JIRA spam

2017-12-02 Thread Jean-Baptiste Onofré
Is this behavior due to a recent change ? This makes JIRAs unreadable IMO (not to mention the email spam to commits@). Thanks, Cham -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: JIRA spam

2017-12-01 Thread Jean-Baptiste Onofré
://issues.apache.org/jira/browse/BEAM-1630 Is this behavior due to a recent change ? This makes JIRAs unreadable IMO (not to mention the email spam to commits@). Thanks, Cham -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [DISCUSS] [DOC] Triggering is for sinks!

2017-11-30 Thread Jean-Baptiste Onofré
) Triggering is for sinks! https://s.apache.org/beam-sink-triggers Please take a look at this "1"-pager and give feedback. Kenn -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: makes bundle concept usable?

2017-11-30 Thread Jean-Baptiste Onofré
> > > >>> > > > > I'm asking because I've reviewed almost all Beam IOs and don't >>> recall >>> > > > > seeing a similar problem. Sure, a lot of IOs do batching within a >>> > > bundle, >>> > > > > but 1) it doesn't take

Re: makes bundle concept usable?

2017-11-30 Thread Jean-Baptiste Onofré
nd up to beam to have a pluggable storage for a few concerns Short term I would be happy with 1 solved properly, long term I hope 2 will be tackled without workarounds requiring custom wrapping of IO to use a custom state persistence. Romain Manni-Bucau @rmannibucau | Blog | Old Blog |

Re: [VOTE] Use Gradle for Apache Beam developmental processes

2017-11-29 Thread Jean-Baptiste Onofré
uven On Wed, Nov 29, 2017 at 12:44 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: -0 It's not for the change itself (gradle build is interesting) but for the process used here, which, IMHO, is not clean. My major concern is the fact that

Re: [VOTE] Use Gradle for Apache Beam developmental processes

2017-11-29 Thread Jean-Baptiste Onofré
(plugin just calls Apache Maven ANT plugin) IntelliJ Integration: Mature Eclipse Integration: Mature Extensibility: Mature Number of GitHub Projects Using It: 122k Continuous build daemon: Mature Incremental build support: Mature Intra-module dependencies: Mature (via configurations) -- Jean-Ba

Re: [DISCUSS] Updating contribution guide for gitbox

2017-11-29 Thread Jean-Baptiste Onofré
have a clear merge commit, but preserves other info Kenn [1] https://apachebeam.slack.com/messages/C1AAFJYMP/ Kenn -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Apache Beam Workshop in Guadalajara Mexico

2017-11-29 Thread Jean-Baptiste Onofré
to share resources and connect with you. I'll report back with findings after the workshop. Cheers, Gris [1] https://academy.wizeline.com/about/ [2] https://academy.wizeline.com/apache-beam/ -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [DISCUSS] Updating contribution guide for gitbox

2017-11-28 Thread Jean-Baptiste Onofré
his preserves the most information (b) -1 this erases the most information (c) -0 this is just sort of a middle ground; it breaks commit hashes, does not have a clear merge commit, but preserves other info Kenn [1] https://apachebeam.slack.com/messages/C1AAFJYMP/ Kenn -- Jean-Baptiste Ono

Re: [DISCUSS] Thinking about Beam 3.x roadmap and release schedule

2017-11-28 Thread Jean-Baptiste Onofré
releases. Automation of release process where possible, better test coverage, a higher resistance to cherry-picks. Kenn On Tue, Nov 28, 2017 at 9:14 AM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: Hi Reuven, Yes, I remember th

Re: [DISCUSS] Thinking about Beam 3.x roadmap and release schedule

2017-11-28 Thread Jean-Baptiste Onofré
; to be "qualified" in a major version. I think that any major idea & feature (breaking or not the API) are valuables for Beam 3.x (and it's a good sign for our community again ;)). Thanks ! Regards JB On 11/28/2017 06:09 PM, Reuven Lax wrote: On Tue, Nov 28, 2017 at 8:55 AM, Jean-B

[DISCUSS] Thinking about Beam 3.x roadmap and release schedule

2017-11-28 Thread Jean-Baptiste Onofré
that one of interesting new feature that Beam 3.x can provide is around PCollection with Schemas. It's something that we started to discuss with Reuven and Eugene. In term of schedule, I would love to see your thoughts & ideas about releases schedule and Beam 3.x. Regards JB -- Jean-Bapt

Re: [RESULT][VOTE] Migrate to gitbox

2017-11-28 Thread Jean-Baptiste Onofré
ore granular information will make it harder to follow. On Thu, Nov 23, 2017 at 12:17 PM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Hi, following the migration to gitbox, we now have a notification e-mail (on the dev mailing list) for each action on a PR (comments, closing, etc). It cou

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Jean-Baptiste Onofré
On Mon, Nov 27, 2017 at 10:23 PM, Jean-Baptiste Onofré <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: Hi Luke, just curious (and maybe I missed it): did we do a formal vote to merge the gradle build ? Gradle is now on master, we have some Jira to update the rel

Re: [DISCUSS] Move away from Apache Maven as build tool

2017-11-27 Thread Jean-Baptiste Onofré
3dbF1M/edit?usp=sharing With this data, I feel confident that we should swap and have opened the following issue https://issues.apache.org/jira/browse/BEAM-3249 and related sub-tasks. On Sun, Nov 19, 2017 at 11:23 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Thanks for the update Luke. I'm u

Re: GitBox down

2017-11-27 Thread Jean-Baptiste Onofré
with the fact that I'm starting to get: fatal: repository 'https://git-wip-us.apache.org/repos/asf/beam.git/' not found On Sun, Nov 26, 2017 at 10:37 PM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Hi, it seems GitBox is down (timing out). I don't see anything on status.apache.org, I wil

Re: [DISCUSSION] Runner agnostic metrics extractor?

2017-11-27 Thread Jean-Baptiste Onofré
that in directly). -- Ben On Mon, Nov 27, 2017 at 9:06 AM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Hi all, Etienne forgot to mention that we started a PoC about that. What I started is to wrap the Pipeline creation to include a thread that polls periodically the metrics in the pipeline result

Re: [DISCUSSION] Runner agnostic metrics extractor?

2017-11-27 Thread Jean-Baptiste Onofré
as a beam sdk extension. Another way, to avoid polling, would be to push metrics values to a sink when they are updated but I don't know if it is feasible in a runner independent way. WDYT about the ideas in this ticket? Best, Etienne -- Jean-Baptiste Onofré jbono...@apache.org http

Re: [DISCUSS] Drop Spark 1.x support to focus on Spark 2.x

2017-11-26 Thread Jean-Baptiste Onofré
please? Is this slated for 2.3.0 at the end of the year? Thanks, Tim, Sent from my iPhone On 20 Nov 2017, at 21:21, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Hi, , it seems we have a consensus to upgrade to Spark 2.x, dropping Spark 1.x. I will upgrade the PR accordingly. Than

GitBox down

2017-11-26 Thread Jean-Baptiste Onofré
Hi, it seems GitBox is down (timing out). I don't see anything on status.apache.org, I will ping INFRA. Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [VOTE] Release 2.2.0, release candidate #4

2017-11-25 Thread Jean-Baptiste Onofré
37 -- Thanks & Regards, Nishu Tayal < https://www.avast.com/sig-email?utm_medium=email_ source=link_campaign=sig-email_content=webmail_t erm=icon Virus-free. www.avast.com < https://www.avast.com/sig-email?utm_medium=email_ source=link_campaign=sig-email_co

Re: gradle dirty files blocking maven build

2017-11-25 Thread Jean-Baptiste Onofré
I don't find any ".gogradle" files. Thanks, Manu On Fri, Nov 24, 2017 at 5:09 PM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: Let me try on my local copy. Thanks for the report. Regards JB On 11/24/2017 10:04 AM, Romain Manni-Bucau wrote: Not sure JB to be honest, my global giti

Re: [VOTE] Release 2.2.0, release candidate #4

2017-11-24 Thread Jean-Baptiste Onofré
Awesome, Thanks Reuven ! Regards JB On 11/25/2017 07:01 AM, Reuven Lax wrote: I'll go ahead and send the RESULT email right now. On Fri, Nov 24, 2017 at 9:56 PM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: It's not sync: promoting the artifacts takes some time (at least 30 m

Re: gradle dirty files blocking maven build

2017-11-24 Thread Jean-Baptiste Onofré
name '.gogradle' | xargs rm -Rf" solves it. Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github | LinkedIn -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [RESULT][VOTE] Migrate to gitbox

2017-11-23 Thread Jean-Baptiste Onofré
:19 AM, Jean-Baptiste Onofré wrote: The migration is done, you have to update your local copy with git remote set-url to use gitbox.apache.org instead of git-wip-us.apache.org. I'm checking the GitHub PRs (if we now have the merge button). Regards JB On 11/23/2017 10:55 AM, Jean-Baptiste Onofré

Re: [RESULT][VOTE] Migrate to gitbox

2017-11-23 Thread Jean-Baptiste Onofré
The migration is done, you have to update your local copy with git remote set-url to use gitbox.apache.org instead of git-wip-us.apache.org. I'm checking the GitHub PRs (if we now have the merge button). Regards JB On 11/23/2017 10:55 AM, Jean-Baptiste Onofré wrote: Hi guys, I just got

Re: [RESULT][VOTE] Migrate to gitbox

2017-11-23 Thread Jean-Baptiste Onofré
Hi guys, I just got an update from INFRA: the migration to gitbox starts now. Regards JB On 11/07/2017 05:51 PM, Jean-Baptiste Onofré wrote: Hi guys, quick update on the gitbox migration. I created a Jira for INFRA: https://issues.apache.org/jira/browse/INFRA-15456 It should be done

Re: Azure(ADLS) compatibility on Beam with Spark runner

2017-11-23 Thread Jean-Baptiste Onofré
%2Fbeam%2Fsdk%2Fio%2Fhdfs%2FHadoopFileSystemOptions.java%23L45=02%7C01%7CMilan.Chandna%40microsoft.com%7Cb7dffcc26bfe44df589a08d53201aeab%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636469905161638292=tL3UzNW4OBuFa1LMIzZsyR8eSqBoZ7hWVJipnznrQ5Q%3D=0 On Wed, Nov 22, 2017 at 1:12 AM, Jean-Baptiste Ono

<    1   2   3   4   5   6   7   8   9   10   >