Re: read source - MongoDbIO.read()

ReadAll is implemented in JDBC and Redis IOs. I will add for on other IOs. Regards JB On Oct 30, 2017, 13:39, at 13:39, Chaim Turkel wrote: >can you send me a link to the code? > >On Mon, Oct 30, 2017 at 2:21 PM, Jean-Baptiste Onofré >wrote: >> That's the evolution I&#x

Re: read source - MongoDbIO.read()

wrote: >>>any reason for this, there should be a way to run it from any point >>> >>>On Mon, Oct 30, 2017 at 11:24 AM, Jean-Baptiste Onofré >>> wrote: >>>> Hi >>>> >>>> No the pipeline starts with the read. You can always create yo

Re: read source - MongoDbIO.read()

17 at 11:24 AM, Jean-Baptiste Onofré > wrote: >> Hi >> >> No the pipeline starts with the read. You can always create your own >custom read. >> >> Regards >> JB >> >> On Oct 30, 2017, 09:33, at 09:33, Chaim Turkel >wrote: >>>Hi, >

Re: read source - MongoDbIO.read()

Hi No the pipeline starts with the read. You can always create your own custom read. Regards JB On Oct 30, 2017, 09:33, at 09:33, Chaim Turkel wrote: >Hi, > Is there a way to have some code run before the read? >I would like to check before how many records exists and based on this >have two

Re: [Proposal] Apache Beam Swag Store

It sounds good. Please let us know trademark update. Thanks Regards JB On Oct 28, 2017, 20:15, at 20:15, Griselda Cuevas wrote: >Thanks for the feedback all, I'll send this idea to the trademark@ >folks >and wait for validation. Once we have it I'll look into building the >store >possibly embed

Re: [Proposal] Beam website navigation update

; +1 >>> >>> Great job, Melissa. >>> >>> It would be nice to have left or right navigation for *blog >>> <https://apache-beam-website-pull-requests.storage.googleapi >>> s.com/332/blog/index.html>* >>> tab with recent posts or

JB offline for 5 days

Hi guys, Just to let you know that I will be offline for the next 5 days (vacation). See you end of next week ! Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Is mergebot down?

suecomment-338551253 the mergebot seems to not activate by "@asfgit merge". Jason, could you take a look? -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Update on 2.2.0 release

update about this ? Are they blocker for the 2.2.0 release ? @Reuven: I think we are good to do a 2.2.0.RC1. Thoughts ? Thanks ! Regards JB On 10/10/2017 12:53 AM, Reuven Lax wrote: I've now cut a release branch for 2.2.0. On Mon, Oct 9, 2017 at 2:01 AM, Jean-Baptiste Onofré wrote: It s

[RESULT][VOTE] Migrate to gitbox

Hi all, this vote passed with only +1. I will requuest INFRA to move the repositories to gitbox. Thanks all for your vote ! Regards JB On 10/10/2017 09:42 AM, Jean-Baptiste Onofré wrote: Hi all, following the discussion, here's the formal vote to migrate to gitbox: [ ] +1, Appro

Re: [VOTE] Migrate to gitbox

+1 (binding) Regards JB On 10/10/2017 09:42 AM, Jean-Baptiste Onofré wrote: Hi all, following the discussion, here's the formal vote to migrate to gitbox: [ ] +1, Approve to migrate to gitbox [ ] -1, Do not migrate (please provide specific comments) The vote will be open for at lea

Re: [Proposal] Sharing Neville's post and upcoming meetups in the Twitter handle

a-processing-at- spotify-the-road-to-scio-part-1/ Thanks! G -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [Proposal] Beam website navigation update

it out and see what you think -- we'd love to get your feedback. Thanks! [1] http://apache-beam-website-pull-requests.storage. googleapis.com/332/index.html [2] https://github.com/apache/beam-site/pull/332 -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Possibility of requiring Java 8 compiler for building Java 7 sources?

ote on user or something different? On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles wrote: Agree with polling Beam users as well as each runner's community in aggregate. On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré < j...@nanthrax.net wrote: Definitely agree On

Re: MongoDbIO

this is the reactive streams pattern that there always is a complete or error path chaim On Tue, Oct 17, 2017 at 2:51 PM, Jean-Baptiste Onofré wrote: Good point but I was thinking more about a PTransform with Create.of() if there's no data. Anyway, I can add a DoFn support in the write to upd

Re: MongoDbIO

apply(ParDo.of(new DoFn() { // check PCollection and set the status })) will not be called On Tue, Oct 17, 2017 at 8:33 AM, Jean-Baptiste Onofré wrote: I didn't mean on the read, I meant between the read and write. Basically, your pipeline could look like: pipeline.apply(Mong

Re: MongoDbIO

42 PM, Chaim Turkel wrote: how to i add a ParDo on the MongoDbIO.read() if there are no records read? On Mon, Oct 16, 2017 at 6:53 PM, Jean-Baptiste Onofré wrote: You can always add your own ParDo(DoFn) where you write the status. Regards JB On 10/16/2017 04:24 PM, Chaim Turkel wrote: that i

Re: Possibility of requiring Java 8 compiler for building Java 7 sources?

A vote on user or something different? On Wed, Sep 27, 2017 at 7:59 PM, Kenneth Knowles wrote: Agree with polling Beam users as well as each runner's community in aggregate. On Wed, Sep 27, 2017 at 9:44 AM, Jean-Baptiste Onofré wrote: Definitely agree On 09/27/2017 06:00 PM, Robert

Re: MongoDbIO

You can always add your own ParDo(DoFn) where you write the status. Regards JB On 10/16/2017 04:24 PM, Chaim Turkel wrote: that is the problem, i want to write a status that i tried and that there were no records On Mon, Oct 16, 2017 at 3:51 PM, Jean-Baptiste Onofré wrote: Hi Chaim, So

Re: MongoDbIO

re no records to read, is there a way to get called so that i can write the status? chaim -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: JIRA component for IOs?

Hi The component to use is actually java-sdk-extension. Regards JB On Oct 13, 2017, 20:34, at 20:34, Valentyn Tymofieiev wrote: >With IOs being a significant part of BEAM codebase&development efforts, >do >we want to have a separate component in JIRA dedicated to IOs, perhaps >broken down into

Re: 100 open pull requests

ctive. Ahmet On Thu, Oct 12, 2017 at 1:43 PM, Jean-Baptiste Onofré wrote: Agree I will start to update/review some PRs. Regards JB On Oct 12, 2017, 21:26, at 21:26, Reuven Lax wrote: In the past, Ahmet and I spent some time each week reviewing and pinging pull requests. This did not happen the

Re: 100 open pull requests

Agree I will start to update/review some PRs. Regards JB On Oct 12, 2017, 21:26, at 21:26, Reuven Lax wrote: >In the past, Ahmet and I spent some time each week reviewing and >pinging >pull requests. This did not happen the past few weeks due to some >vacations >and travel. I do think pinging i

Re: CouchDbIO connector in beam io

e: Hi All, I wanted to have inputs from community members regarding to have couchdb io connectors in our beam io module. Regards, Tarush -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [DISCUSS] Switch to gitbox

well on the website. Since gitbox enforces that the precommit checks pass, it would also be a good forcing function for the community to maintain reliably passing tests. On Fri, Oct 6, 2017 at 4:58 AM, Jean-Baptiste Onofré < j...@nanthrax.net wrote: Hi guys, We use Apache gi

[VOTE] Migrate to gitbox

votes. Thanks, Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Update on 2.2.0 release

Hi Reuven, Good one ! I plan to cherry pick RedisIO on the 2.2.0 release branch as soon as I merged on master. No objection ? Regards JB On 10/10/2017 12:53 AM, Reuven Lax wrote: I've now cut a release branch for 2.2.0. On Mon, Oct 9, 2017 at 2:01 AM, Jean-Baptiste Onofré wrote:

Re: Update on 2.2.0 release

. The remaining blocking issues will either be cherry picked into an additional release candidate or punted to 2.3.0, depending on severity. Reuven On Wed, Sep 20, 2017 at 5:34 AM, Jean-Baptiste Onofré wrote: Hi Reuven, thanks for the update. I will pick up some Jira in my bucket (to verify/d

[DISCUSS] Switch to gitbox

Hi guys, We use Apache gitbox for the website and it works fine (as soon as you linked your Apache and github with 2FA enabled). What do you think about moving to gitbox for the codebase itself ? It could speed up the review and merge for the PRs. Thoughts ? Regards JB -- Jean-Baptiste

Re: spark-submit forces jackson 2.4.4

Hi Do you start your pipeline with spark-submit ? If so you can provide the packages. You can also create a shaded jar. I have a similar issue in the spark 2 runner that I worked around by aligning the dependencies. Regards JB On Oct 2, 2017, 20:04, at 20:04, Jacob Marble wrote: >My Beam pip

Re: Contributor introduction

ssign to myself a task that I've just opened. Thanks, Uri Silberstein -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Hi

, I am a part of a Team that uses Beam and I would like to contribute to the Project. Please add me to the contributor list on JIRA. Thanks Yoni Seneor -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Possibility of requiring Java 8 compiler for building Java 7 sources?

ts. So the fix to this bug would essentially require Java 8 compilers even for compiling Java 7 code. Does anyone need to use Java 7 compilers? Because if not I would like to continue with this fix. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Possibility of requiring Java 8 compiler for building Java 7 sources?

ly require Java 8 compilers even for compiling Java 7 code. Does anyone need to use Java 7 compilers? Because if not I would like to continue with this fix. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [Proposal] Beam Newsletter

4L8b1It9Ju1JgJaSvSPlAYMlG0Q4v4hbjebkmNsnQ8/edit> so people can add their updates. 2. I'll edit it to get it ready to share 3. I'll send an email with highlights and the link to the finalized Google doc I propose to do this monthly. What do you guys think? Gris -- Jean-Baptist

Re: [VOTE] Release 2.1.1, release candidate #1

-- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: TikaIO concerns

I meant by "basically good", it's more in feature scope. Regards JB On 09/20/2017 01:18 AM, Eugene Kirpichov wrote: On Tue, Sep 19, 2017 at 5:13 AM Jean-Baptiste Onofré wrote: Hi Sergey, as discussed together during the review, I fully understand the choices you did. Your plan

Re: Proposal: Unbreak Beam Python 2.1.0 with 2.1.1 bugfix release

2.1.0 release branch (to form a new 2.1.1 release branch) and call a vote to release version 2.1.1 only for Beam Python. Alternatively, to preserve version alignment we could also re-release Beam Java 2.1.1 with the same code as 2.1.0 modulo the version bump. Thoughts? Best, Charles -- Jean

Re: Update on 2.2.0 release

fixed; please verify this and close the issue if you are the reporter. The remaining issues (BEAM-2834, BEAM-2858, BEAM-2870, BEAM-29540) all have outstanding pull requests. As soon as all of these pull requests are merged, I will start the release cut. Reuven -- Jean-Baptiste Onofré jbono...@apach

Re: [DISCUSSION] using NexMark for Beam

report any given >issue >>>> on this runner, >>>> >>>> - We also have to decide the future of some features, this is >probably >>>> independent of the current PR and part of the evolution of Nexmark >on >>>> Beam: >>>> >

Re: TikaIO concerns

O Experimental to give Tika and Beam users some time to try it with some real complex files and also decide if TikaIO can continue implemented as a BoundedSource/Reader or not Eugene, all, will it work if I start with 1) ? Thanks, Sergey [1] https://issues.apache.org/jira/browse/BEAM-2328 [2]

Re: flaky testSplit in beam-sdks-java-io-elasticsearch-tests-5

15:56, Etienne Chauchot a écrit : Yes, sure, please ignore this test, I'll fix it soon. Best, Etienne Le 15/09/2017 à 15:15, Jean-Baptiste Onofré a écrit : Thanks, in the mean time, maybe we can ignore this test (to avoid to impact the build) ? Regards JB On 09/15/2017 02:34 PM, Etienne Chau

Re: flaky testSplit in beam-sdks-java-io-elasticsearch-tests-5

this issue. Etienne -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: PBegin, PDone

chaim -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Beam 2.2.0 release

u are unavailable, we know who has these permissions. We should also make sure this is all documented on the Beam release guide. Reuven On Wed, Sep 6, 2017 at 9:25 PM, Jean-Baptiste Onofré wrote: It sounds good to me. By the way, you will need my help to complete the release process (as you need

Re: PipelineResult

the results? chaim -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Merge branch DSL_SQL to master

ql.example. BeamSqlExample. [1]. Special thanks to all contributors/reviewers: Tyler Akidau Davor Bonaci Robert Bradshaw Lukasz Cwik Tarush Grover Kai Jiang Kenneth Knowles Jingsong Lee Ismaël Mejía Jean-Baptiste Onofré James Xu Mingmin Xu [2]. https://github.com/apache

Re: Beam 2.2.0 release

e.org/jira/browse/BEAM-2833 which is a blocker to merge DSL_SQL. There may be something wrong in the back-end(maybe RunnerApi) to handle parametered CustomCoder in TestPipeline. On Thu, Aug 31, 2017 at 10:38 AM, Jean-Baptiste Onofré wrote: Fair enough. That's fine for me. Regards JB On A

Re: [PROPOSAL] FileIO.write: a modular replacement for FileBasedSink/WriteFiles

eSink { open(dest), write(input), close() }) .to(input → dest) .withFilenamePolicy(dest → prefix, shard pattern) .withEverythingElse() // like in WriteFiles -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: MapReduce Runner needs contributors

ser timers. 2. Performance improvement For examples, https://issues.apache.org/jira/browse/BEAM-2835 3. Production readiness Try run it in Hadoop cluster Thanks and looking forward to the collaboration on MapReduce Runner. -- Pei -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.

Re: Beam 2.2.0 release

s of having a >release >> published soon after the previous release is a small price to pay for >> helping those users :) >> >> On Wed, Aug 30, 2017, 11:30 PM Jean-Baptiste Onofré >> wrote: >> >> > As we released 2.1.0 couple of weeks ago, it could sound

Re: [DISCUSS] Capability Matrix revamp

o get short descriptions into the foreground to make it easier to grok. These are just a few thoughts, and not necessarily compatible with each other. What do you think? Kenn -- Thanks, Jesse -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Beam 2.2.0 release

wrote: I'd suggest to do 2.2.0 as quickly as possible, and target 2.3.0 for October. I don't see a reason to delay 2.2.0 until October: there's a huge amount of features worth releasing between when 2.1.0 was cut and the current HEAD. On Wed, Aug 30, 2017 at 10:18 PM Jean-Baptiste Ono

Re: Beam 2.2.0 release

eam 2.1.0 has finally completed, I think we should cut Beam 2.2.0 soon. I volunteer to coordinate this release. Are there any pending pull requests that people think should be merged before we cut 2.2.0? If so, please let me know soon, as I would like to cut by Wednesday of next week.

Re: Beam 2.2.0 release

2.2.0? If so, please let me know soon, as I would like to cut by Wednesday of next week. Thanks, Reuven -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Beam 2.2.0 release

pending pull requests that people think should be merged before we cut 2.2.0? If so, please let me know soon, as I would like to cut by Wednesday of next week. Thanks, Reuven -- Mingmin -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: new guy

ate to be added as a contributor on jira. Thanks! Joey -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: kafka docs

readme that points to the classe's javadocs. regards Joey Baruch -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: kafka docs

.com/apache/beam/tree/master/sdks/java/io/kafka>, but there is no docs there. I will add a simple readme that points to the classe's javadocs. regards Joey Baruch -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: "Unable to find registrar for hdfs" on Flink cluster

(Method.java:498) at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528) ... 13 more Thanks & Regards, Ramanji. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

[RESULT][VOTE] Release 2.1.0, release candidate #3

>>>> > >>> >>>> wrote: >>> >>>> >>> >>>>> On Mon, Aug 14, 2017 at 6:32 AM, Ismaël Mejía >> >>> >>> wrote: >>> >>>>> >>> >>>>>> +1 (non-binding) >>> >>>&g

Re: Beam spark 2.x runner status

k side change was introduced in >https://github.com/apache/spark/pull/12157.) > >Does anyone have ideas about this compile error? > > >On Wed, May 3, 2017 at 1:32 PM, Jean-Baptiste Onofré >wrote: > >> Hi Ted, >> >> My branch used Spark 2.1.0 and I just upda

Re: Style: how much testing for transform builder classes?

could have, but they fixed it and >their >>> work >>> > is >>> > > > not >>> > > > >> blocked. I think Robert's arguments about the cost of >trivial >>> tests >>> > > > apply. >>> > > >

Re: [VOTE] Release 2.1.0, release candidate #3

;> >>>>> On Mon, Aug 14, 2017 at 6:32 AM, Ismaël Mejía > >> >>> wrote: >> >>>>> >> >>>>>> +1 (non-binding) >> >>>>>> >> >>>>>> - Validated signatures OK >> >>>

Re: Policy for stale PRs

t; >Ahmet > >On Wed, Aug 16, 2017 at 4:32 AM, Aviem Zur wrote: > >> Makes sense to close after a long time of inactivity and no response, >and >> as Kenn mentioned they can always re-open. >> >> On Wed, Aug 16, 2017 at 12:20 AM Jean-Baptiste Onofré > >>

Re: Proposal: adding a built-in I/O source for VCF files

I will thanks ! Regards JB On Aug 16, 2017, 18:53, at 18:53, Asha Rostamianfar wrote: >Hi everyone, > >I have a proposal to add a new built-in I/O source for VCF files: >https://docs.google.com/document/d/1jsdxOPALYYlhnww2NLURS8NKXaFyRSJrcGbEDpY9Lkw/edit > >I'm planning to take on the implement

Re: [VOTE] Release 2.1.0, release candidate #3

>> > with >> > > > > the docker development images (WIP), both OK >> > > > > - Run WordCount on local Flink and Spark runners OK >> > > > > >> > > > > Everything looks nice, only one minor thing (not blocking at >all

Re: Hello from a newbie to the data world living in the city by the bay!

Welcome ! Regards JB On Aug 16, 2017, 08:54, at 08:54, "Ismaël Mejía" wrote: >Hello and welcome Griselda, Umang, Justin > >Apart of the links provided by Ahmet you might read Beam-related >material on the website (See Documentation > Programming Guide and >Documentation > Additional Resources am

Re: Policy for stale PRs

If we consider the author, it makes sense. Regards JB On Aug 15, 2017, 01:29, at 01:29, Ted Yu wrote: >The proposal makes sense. > >If the author of PR doesn't respond for 90 days, the PR is likely out >of >sync with current repo. > >Cheers > >On Mon, Aug 14, 2017 at 5:27 PM, Ahmet Altay >wrote

Re: [VOTE] Release 2.1.0, release candidate #3

+1 (binding) I do my own tests and casting my own vote ;) Regards JB On 08/09/2017 07:08 AM, Jean-Baptiste Onofré wrote: Hi everyone, Please review and vote on the release candidate #3 for the version 2.1.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release

Re: [ANNOUNCEMENT] New committers, August 2017 edition!

riven it to the point of merging to the master branch. In this effort, he accumulated 65 commits (7,812 ++ / 4,882 --) and extended the project to the new user community. Congratulations to all five! Welcome! Davor -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend -

Re: [ANNOUNCEMENT] New PMC members, August 2017 edition!

project operates, leading discussions on inclusiveness and openness. Congratulations to both! Welcome! Davor -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

JB mostly offline for the next two weeks

Hi guys FYI I'm in vacation tonight for the next two weeks. I will be online up to Sunday evening, and I will start a road trip next Monday. I will have Internet connection on the evening but very limited during days. Cheers JB ⁣​

Re: [VOTE] Release 2.1.0, release candidate #3

Gently reminder on this thread. Thanks ! Regards JB On 08/09/2017 07:08 AM, Jean-Baptiste Onofré wrote: Hi everyone, Please review and vote on the release candidate #3 for the version 2.1.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific

Re: beam-site issues with Jenkins and MergeBot

ate email about htis) The gnupg failure is not legit - I'm able to run the same command myself with no issues. 3) Suppose because of this, I'm not able to merge my PR with "@asfgit merge" command - I suppose it requires a successful test run. Would be nice if it posted a commen

[VOTE] Release 2.1.0, release candidate #3

Hi everyone, Please review and vote on the release candidate #3 for the version 2.1.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1

Re: [INFO] Build fails on master

ira/browse/BEAM-2755) to fix that. Ahmet On Mon, Aug 7, 2017 at 11:44 PM, Jean-Baptiste Onofré wrote: Hi guys, it seems we have an issue to build master on Python SDK: Test failed: error: Test failed: ERROR: InvocationError: '/home/jbonofre/Workspace/beam /sdks/python/target/.tox/py2

[INFO] Build fails on master

__ summary docs: commands succeeded lint: commands succeeded py27: commands succeeded py27cython: commands succeeded ERROR: py27gcp: commands failed [ERROR] Command execution failed. I will check the recent changes (but RC3 first ;)). Regards JB -- Jean-Baptiste Onofré jbono...@apach

Re: [VOTE] Release 2.1.0, release candidate #2

ng it drag on for too long loses that momentum. We should anyway pursue resolving the issues asap, and users who were eagerly waiting for Spark streaming to work properly can run Beam at HEAD in the window between when they are first resolved and when 2.2.0 is released. What do you think? On Sat,

Re: [VOTE] Release 2.1.0, release candidate #2

h cutting the RC, correct? Thanks. On Thu, Aug 3, 2017 at 7:04 AM Jean-Baptiste Onofré wrote: Another quick update. Regarding BEAM-2671, I asked help from Stas and Aviem on this one. It's our high priority as it's the main blocking issue before cutting RC3. At some point, if we are n

Re: [VOTE] Release 2.1.0, release candidate #2

so there are no blocking issues remaining for 2.1.0. JB - the next step is for you to proceed with cutting the RC, correct? Thanks. On Thu, Aug 3, 2017 at 7:04 AM Jean-Baptiste Onofré wrote: Another quick update. Regarding BEAM-2671, I asked help from Stas and Aviem on this one. It's our h

Re: [VOTE] Release 2.1.0, release candidate #2

2017 08:52 PM, Jean-Baptiste Onofré wrote: Hi, Thanks Eugene for the sumup. BEAM-2708 is now fixed. The last blocking issue for RC3 is BEAM-2671. I spent time today on this one, investigating the different issues. Agree that help from Aviem and Kenn would help for sure. Aviem already started

Re: Proposal : An extension for sketch-based statistics

ching> Thanks ! Arnaud -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [VOTE] Release 2.1.0, release candidate #2

/jira/browse/BEAM-2708 has a fix in review https://issues.apache.org/jira/browse/BEAM-2671 is the nasty one and we don't understand it nor have a fix. Help is needed; some people who could help are +Kenn Knowles and +Aviem Zur . On Thu, Jul 27, 2017 at 6:41 AM Jean-Baptiste Onofré wrote:

Re: [DISCUSS] Beam MapReduce Runner One-Pager

peihe/incubator-beam/tree/mr-runner I am still working on View support, and will give another update once I can get WordCount runs. On Sat, Jul 15, 2017 at 4:45 AM, Vikas RK wrote: Thanks Pei, left a few comments, but this looks exciting! -Vikas On 12 July 2017 at 21:52, Jean-Baptiste Onofré

Re: Style of messages for checkArgument/checkNotNull in IOs

Agree Regards JB On Jul 28, 2017, 20:22, at 20:22, Thomas Groh wrote: >I'm in favor of the wording in the style of the first: it's an >immediately >actionable message that will have an associated stack trace, but will >provide the parameter in plaintext so the caller doesn't have to dig >through

Re: Style of messages for checkArgument/checkNotNull in IOs

Hi I'm a bit concerned because it has been discussed in a Jira and we discussed especially about this with Kenn and I. It dates from the incubation period. I changes most of the IOs I did for style 2 whereas I started with style 1. I'm ready to change again on the IOs but we agreed that checkAr

Re: Requiring PTransform to set a coder on its resulting collections

ere. For example, in those cases you could encode in your DoFn so the type descriptor would just be byte[]. As well as being an extremely cumbersome API, this would incur the cost of coding/decoding at that DoFn boundary even if it is fused away. On Thu, Jul 27, 2017 at 1:34 AM, Jean-Bap

Re: Requiring PTransform to set a coder on its resulting collections

s fused away. On Thu, Jul 27, 2017 at 1:34 AM, Jean-Baptiste Onofré wrote: Hi, That's an interesting thread and I was wondering the relationship between type descriptor and coder for a while ;) Today, in a PCollection, we can set the coder and we also have a getTypeDescriptor().

Re: [VOTE] Release 2.1.0, release candidate #2

On 07/18/2017 06:30 PM, Jean-Baptiste Onofré wrote: Hi everyone, Please review and vote on the release candidate #2 for the version 2.1.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for

Re: Requiring PTransform to set a coder on its resulting collections

actually do this, but I'd like to know whether people agree that this is a good goal in general. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [S]FTP support as Pipeline I/O

class StringIO which means collecting the output into a string - and then I suppose uploading the string to FTP. That could work (similar stuff exists in Java library) but would limit us to files whose content fits in memory. On Mon, Jul 24, 2017, 12:31 PM Jean-Baptiste Onofré wrote: I guess

Re: [S]FTP support as Pipeline I/O

>> >> > This would require writing data to local files in order to upload >it to >> the >> > remote FTP, right? >> > >> > On Mon, Jul 24, 2017 at 12:31 AM, Jean-Baptiste Onofré > >> > wrote: >> > >> > > Hi

Re: [CANCEL][VOTE] Release 2.1.0, release candidate #2

merged for the master. Best, Aljoscha On 24. Jul 2017, at 14:09, Jean-Baptiste Onofré wrote: +1 Definitely good to have it for RC3. Regards JB On 07/24/2017 02:05 PM, Aljoscha Krettek wrote: When we're cutting a new RC anyways we could also include the fixes for https://issues.apac

Re: [CANCEL][VOTE] Release 2.1.0, release candidate #2

detailed by the following two tickets: https://issues.apache.org/jira/browse/BEAM-2670 https://issues.apache.org/jira/browse/BEAM-2671 On Mon, Jul 24, 2017 at 11:44 AM Jean-Baptiste Onofré wrote: Hi all, due to https://issues.apache.org/jira/browse/BEAM-2662, I cancel this vote. We also ha

[CANCEL][VOTE] Release 2.1.0, release candidate #2

Spark runner test fix for RC3 (BEAM-2662 is already fixed on release-2.1.0 branch). I will submit RC3 to vote as soon as Spark runner tests are fully OK. Regards JB On 07/18/2017 06:30 PM, Jean-Baptiste Onofré wrote: Hi everyone, Please review and vote on the release candidate #2 for the version

Re: [S]FTP support as Pipeline I/O

ome research and it looks like there isn't any kind of development ongoing for this (at least not on Jira). I'd like to know your thoughts and if someone would like to help/support this initiative. In case someone has started a thing already please let me know ;) Thank you, -- Jean-

Re: [VOTE] Release 2.1.0, release candidate #2

still suggest criteria to add (or remove) and sign up to validate them. Changes to the overall format are also welcome. Kenn On Sun, Jul 23, 2017 at 9:46 PM, Jean-Baptiste Onofré wrote: Hi Luke, good catch. I tested with "my" beam-samples but not with quickstart. I would consider as

Re: [VOTE] Release 2.1.0, release candidate #2

and Dataflow worked for me. On Sun, Jul 23, 2017 at 12:51 PM, Jean-Baptiste Onofré wrote: Gently reminder, the vote is still open. Here's my +1 (binding). Regards JB On 07/18/2017 06:30 PM, Jean-Baptiste Onofré wrote: Hi everyone, Please review and vote on the release candidate #2

Re: [VOTE] Release 2.1.0, release candidate #2

Gently reminder, the vote is still open. Here's my +1 (binding). Regards JB On 07/18/2017 06:30 PM, Jean-Baptiste Onofré wrote: Hi everyone, Please review and vote on the release candidate #2 for the version 2.1.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the re

<    3   4   5   6   7   8   9   10   11   12   >