Re: ElasticSearch with RestHighLevelClient

doc.getMetadata().first( Metadata.NAME ).asString() ) ) ) ) { mpr.writeValue( fos, doc ); } catch ( IOException ioe ) { log.error( ioe.getLocalizedMessage(), ioe ); } } } } On Mon, Oct 23, 2017 at 2:28 PM, Jean-Baptiste Onofré wrote: Hi Ryan, Why don't

Re: ElasticSearch with RestHighLevelClient

m never exits. If I comment out the client assignment above, the pipeline behaves normally (but obviously, I can't write anything to ES). Any advice for a dev just getting started with Apache Beam (2.0.0)? ry -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: How to use ConsoleIO SDK

nfidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [VOTE] [DISCUSSION] Remove support for Java 7

What happens for the users using spark 1.5 that run with Java 7 only ? On Oct 18, 2017, 12:06, at 12:06, "Ismaël Mejía" wrote: >+1 > >I forgot to vote yesterday, I don't really think this is a change >worth requiring a major version of Beam. Just clear information in the >site/release notes shoul

Re: [VOTE] [DISCUSSION] Remove support for Java 7

However, it's good to target this for Beam 3.0.0 as it can have an impact especially for runners. Regards JB On 10/17/2017 06:45 PM, Jean-Baptiste Onofré wrote: +1 from a general purpose. +0 from a runner perspective (as it depends of the execution engine). Regards JB On 10/17/2017

Re: [VOTE] [DISCUSSION] Remove support for Java 7

the Java 7 compatibility. So, please vote: +1 Yes, go ahead and move Beam support to Java 8. 0 Do whatever you want. I don’t have a preference. -1 Please keep Java 7 compatibility (if possible add your argument to keep supporting for Java 7). -- Jean-Baptiste Onofré jbono...@apache.org http

Re: DoFn setup/teardown sequence

Yes, no problem at all. I meant that the DoFn is "attached" to a pipeline. Regards JB On 10/16/2017 08:25 AM, Derek Hao Hu wrote: I believe a worker can execute multiple instances (i.e. threads) of a DoFn. Derek On Sun, Oct 15, 2017 at 10:46 PM, Jean-Baptiste Onofré <mailto:j...

Re: DoFn setup/teardown sequence

ank you, Jacob -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Regarding Beam Slack Channel

yNick mailto:nerdyn...@gmail.com>> wrote: Hello Can someone please add me to the Beam slack channel? Thanks. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Apache Beam meetup London 1: debrief

's co-organising these with me) for the next meetup, probably in November. Do you have any plans of being in London in that month? We would be happy to adjust the timing to your needs to be able to have you talk! Thanks :) Best, Matthias Op di 10 okt. 2017 om 14:23 schreef Jean-Baptiste Onofré

Re: Slack invitation

Done, welcome aboard ! Regards JB On 10/10/2017 07:04 PM, t osh wrote: Hello Could someone please add me to the Beam slack channel?  Thanks. Taichi Oshiumi -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Apache Beam meetup London 1: debrief

't doubt to reach out to Griselda (in cc) as well - she has been a great help for us so far! Best, Matthias -- -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [Call for Speakers] Apache Beam Meetup @ San Francisco, CA on Oct. 24th

're interested in giving any of these two talks, reply to this message and we can arrange details.* The date is a bit flexible in case someone is interested but can't make the date. Thanks! G -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Spark and Beam

i'm new at Spark and also at beam. currently i have Java code that use Spark from reading some data from DB. my Spark code using SparkSession.builder (.) and also sparkContext. how can i make beam work similar to my current code, i just want make it work for now. Thanks Tal -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Regarding Beam Slack Channel

gmail web) -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Apache Beam v2.1.0 - Spark Runner Issue

ttp://blog.nanthrax.net Talend - http://www.talend.com Hi JB, with your changes from PR #3808 https://github.com/apache/beam/pull/3808 worked with above discussed AccumulatorV2 changes in spark-core (i.e avoiding that assertion) . -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Slack invitation

Done, Welcome ! Regards JB On 09/12/2017 11:12 AM, Pawel Bartoszek wrote: Can I get an invitation to Slack? Pawel Bartoszek -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Apache Beam v2.1.0 - Spark Runner Issue

nerror-assertion-failed-copyandreset-must-return-a-zero-value-copy and https://github.com/bigdatagenomics/adam/issues/1021#issuecomment-216283222 Does spark-core needs to be changed ..? can you please confirm. Thanks in advance, Suri -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: London Apache Beam meet up

least: I'd love to hear about any *channels to share this *and get it out in the open (through social media or just your friends / colleagues that may be interested) - the more people attend, the merrier. Cheers, Matthias -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nan

Re: Apache Beam v2.1.0 - Spark Runner Issue

to iterators is supposed to fix this issue or missing something else . ? please confirm. Thanks in Advance. Suri -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Apache Beam v2.1.0 - Spark Runner Issue

about the validate runner test. However you can already take a look. Regards JB On 09/04/2017 04:00 PM, Mahender Devaruppala wrote: Sure, thanks very much JB, will look forward to your link. -Original Message- From: Jean-Baptiste Onofré [mailto:j...@nanthrax.net] Sent: Monday, Septem

Re: Apache Beam v2.1.0 - Spark Runner Issue

Spark 2.x? Thanks, Mahender -Original Message- From: Jean-Baptiste Onofré [mailto:j...@nanthrax.net] Sent: Friday, September 1, 2017 1:54 AM To: user@beam.apache.org Subject: Re: Apache Beam v2.1.0 - Spark Runner Issue Sure, I will send the PR during the weekend. I will let you know. Re

Re: [Proposal] Host office hours to help new Beam users/contributors

tters to always have someone who can help a larger group. If you guys like the idea I can help organize them. G -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Apache Beam v2.1.0 - Spark Runner Issue

- From: Jean-Baptiste Onofré [mailto:j...@nanthrax.net] Sent: Thursday, August 31, 2017 12:11 AM To: user@beam.apache.org Subject: Re: Apache Beam v2.1.0 - Spark Runner Issue Hi, I'm working on a Spark runner specific to Spark 2.x as the API changed. So, for now, I have two runners: one for

Re: Apache Beam v2.1.0 - Spark Runner Issue

etParameters(String element, PreparedStatement query) *throws*SQLException { String[] datas= element.split("\t"); *if*(datas.length>0) { *for*(*int*j=0 ; j -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Spark job hangs up at Evaluating ParMultiDo(ParseInput)

arget/word-count-beam-0.1.jar --runner=SparkRunner --inputFile=pom.xml --output=counts Regards, Sathish. J On 01-Aug-2017, at 4:02 PM, Jean-Baptiste Onofré <mailto:j...@nanthrax.net>> wrote: Hi Sathish, Do you see the tasks submitted on the history server ? Regards JB On 08/01/2017

Re: Spark job hangs up at Evaluating ParMultiDo(ParseInput)

.jar —inputFile=./pom.xml --output=csvout --runner=SparkRunner Have attached log file for reference. Can anyone please help me find out whats going on? Regards, Sathish. J -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Provide docker image

ainsi que toutes les pieces attachees. L'utilisation, la divulgation, la publication, la distribution, ou la reproduction non expressement autorisees de ce message et de ses pieces attachees sont interdites. --

Re: Provide docker image

lication, la distribution, ou la reproduction non expressement autorisees de ce message et de ses pieces attachees sont interdites. -------- -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: slack

Hi Antony, done. You should have received an invite. Welcome aboard ! Regards JB On 07/20/2017 10:58 AM, Antony Mayi wrote: can I get invited please? thx, a. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Slack.

oogle.com>> wrote: Welcome. Sent you an invite. On Wed, Jul 12, 2017 at 12:16 PM, Matthew Sole mailto:matthews...@gmail.com>> wrote: Hi, Could you add me to slack please? Thank you, Matt -- Jean-Baptiste Onofré jbo

Re: Join slack

You should have received an invite. Welcome ! Regards JB On 07/06/2017 08:29 AM, Dennis Mårtensson wrote: Hi, I would like to join the slack channel. Thank you Dennis -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Beam Slack channel

ces attachees. L'utilisation, la divulgation, la publication, la distribution, ou la reproduction non expressement autorisees de ce message et de ses pieces attachees sont interdites. ---- -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Recommended Flink Version

jobs to it. Best, Aljoscha On 29. Jun 2017, at 10:15, Jean-Baptiste Onofré wrote: Yes, it's the same with the spark runner using bin/spark-submit. From my standpoint, it's not a bug, it's a feature request. Regards JB On 06/29/2017 10:12 AM, Aljoscha Krettek wrote: I also

Re: Recommended Flink Version

Jun 2017, at 07:23, Jean-Baptiste Onofré wrote: Hi Will, assuming you are using Beam 2.0.0, the Flink runner uses Flink 1.2.1 by default. So, I would recommend this version or 1.2.x. Regards JB On 06/28/2017 10:39 PM, Will Walters wrote: Hello, I've been attempting to run Beam through Fli

Re: Recommended Flink Version

ouble with getting a job to submit, partly because of incompatibility between versions. Does anyone know what versions of Beam and Flink I should be using to give myself the best chance of finding compatibility? Thank you, Will. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthra

Re: Beam Slack channel

hannel as well? Thanks, Cao Manh Dat On Tue, Jun 27, 2017 at 1:47 PM Arun Mahadevan mailto:ar...@apache.org>> wrote: Hi JB, Can you please send me an invite to the slack channel? Thanks, Arun On 6/27/17, 12:32 AM, "Jean-Bapti

Re: Beam Slack channel

Done. Welcome ! Regards JB On 06/26/2017 07:15 PM, Vinay Patil wrote: Hi, I am interested to join the slack channel. Can you please send me an invite. Regards, Vinay Patil Regards, Vinay Patil On Mon, Jun 26, 2017 at 1:42 PM, Jean-Baptiste Onofré <mailto:j...@nanthrax.net>&

Re: Beam Slack channel

ase add me to the slack channel? Best regards Aleksandr Gortujev. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Regarding Beam Slack Channel

inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature, which may be contained in this attachment and shall not be liable for direct, indirect, consequential or special damages in connection with this e-m

Re: Slack access.

Hi, welcome aboard ! you should have received an invite. Regards JB On 06/21/2017 02:24 PM, Łukasz Gajowy wrote: Hello! I'd like to start contributing to Apache Beam therefore slack access would be helpful. Could anyone help? Best Regards, Łukasz Gajowy -- Jean-Baptiste Onofré

Re: Quickstart Issue

;Unable to find registrar for HDFS." If I do not, it is unable to find the specified file. -Will On Tuesday, June 13, 2017 12:23 PM, Jean-Baptiste Onofré wrote: Hi, which sample do you try ? For HDFS, how did you define the location of the hdfs-site.xml ? Regards JB On 06/13/2017

Re: Quickstart Issue

recognize the input file. I have tried local, absolute, and hdfs filepaths, and none of them have seemed to work. Does anyone have advice about what form the filepaths should be in? Thank you, Will Walters. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http

Re: Python SDK 2.0.0

ire ainsi que toutes les pieces attachees. L'utilisation, la divulgation, la publication, la distribution, ou la reproduction non expressement autorisees de ce message et de ses pieces attachees sont interdites. -------

Re: No TransformEvaluator registered error

, May 30, 2017 at 7:11 PM Jean-Baptiste Onofré <mailto:j...@nanthrax.net>> wrote: Hi Antony, which version of the Spark runner are you using ? Regards JB On 05/30/2017 06:04 PM, Antony Mayi wrote: > Hi, > > started getting following error whe

Re: No TransformEvaluator registered error

org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:85) at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295) at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281) -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Make runner implementation smoother?

Got your points ;) I think a first step would be kind of "runner author guide" documentation. Kenn created the corresponding Jira already. While writing this guide, I'm pretty sure we would find some improvements ;) Thanks buddy. Talk to you on private message about my wip on runners. Regards J

Re: Problem when trying to specify the runner in examples

wn 'runner' specified 'FlinkRunner', supported pipeline runners [DirectRunner] -> [Help 1]/ / / //I am wondering if anyone got the same issue and how you solved it. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: question about running examples in command

inal? -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Make runner implementation smoother?

<https://twitter.com/rmannibucau> | Blog <https://blog-rmannibucau.rhcloud.com> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory <https://javaeefactory-rm

Re: Question regarding on how to run Beam locally

anyone could help in how to run program locally in Ubuntu. Sincerely, a user -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Beam 2.0 Release Q and A

Awesome ! Great work Jesse ! Regards JB On May 17, 2017, 14:26, at 14:26, Jesse Anderson wrote: >After the first release of Beam, I did a Q and A > >with the users and developers of Beam. Now that we've done t

Re: program does not exit from waitUntilFinish()

uejoe -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: appending beam pipeline to spark job

ny Mayi wrote: very useful, thanks! btw. to avoid calling the Create.of(rdd.collect()) - is there by any chance way to get a pcollection directly from rdd? thx, antony. On Wednesday, 10 May 2017, 10:37, Jean-Baptiste Onofré wrote: Hi Antony, yes, it's possible to "inject"/

Re: appending beam pipeline to spark job

ssible to build the beam pipeline on existing spark context? thx, Antony. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [HEADS UP] Using "new" filesystem layer

specified, you should only need to write: Pipeline pipeline = Pipeline.create(options); pipeline.apply(TextIO.write().to("hdfs://localhost/path")); On Fri, May 5, 2017 at 6:23 AM, Jean-Baptiste Onofré mailto:j...@nanthrax.net>> wrote: Hi guys, thanks Luke, I upd

Re: [HEADS UP] Using "new" filesystem layer

be setting the Hadoop configuration on HadoopFileSystemOptions. Also, I just merged https://github.com/apache/beam/pull/2890 which will auto detect Hadoop configuration based upon your HADOOP_CONF_DIR and YARN_CONF_DIR environment variables. On Thu, May 4, 2017 at 8:58 AM, Jean-Baptiste

Re: Fwd: Slack Invite

Done Regards JB On 05/05/2017 10:02 AM, Edward Bosher wrote: i, Whenever you have time I'd love to get an invite to slack on this email address. edbosher at gmail com Thanks, Ed -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: [HEADS UP] Using "new" filesystem layer

will auto detect Hadoop configuration based upon your HADOOP_CONF_DIR and YARN_CONF_DIR environment variables. On Thu, May 4, 2017 at 8:58 AM, Jean-Baptiste Onofré mailto:j...@nanthrax.net>> wrote: Hi guys, One of key refactoring/new feature we bring in the first stable release is

[HEADS UP] Using "new" filesystem layer

edSink.java:193) at org.apache.beam.sdk.io.TextIO$Write.to(TextIO.java:292) at org.apache.beam.samples.data.ingestion.JdbcToHdfs.main(JdbcToHdfs.java:39) I gonna investigate tonight and I will let you know. Regards JB -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Reprocessing historic data with streaming jobs

n the streaming > version of the job would not apply in batch, and I would want to avoid stitching > together results from two jobs if I can. > > > These are the approaches I've thought of currently, and any input is much > appreciated. Have any of

Re: Reprocessing historic data with streaming jobs

ur in the streaming version of the job would not apply in batch, and I would want to avoid stitching together results from two jobs if I can. These are the approaches I've thought of currently, and any input is much appreciated. Have any of you faced similar situations, and how did you solve

Re: Apache Beam Slack channel

Done, Regards JB On 05/01/2017 07:34 AM, Maurice M Manning wrote: Hi, I would like to request an invite to the Slack team too. Regards, Maurice Maurice M. Manning Berkeley Research Computing <http://research-it.berkeley.edu/brc> UC Berkeley -- Jean-Baptiste Onofré jbono...@apac

Re: Apache Beam Slack channel

Aviem already sent the invite on your gmail address. You should have it in your mbox. Can you check please ? Thanks, Regards JB On 04/30/2017 05:31 PM, Lars BK wrote: Yes please. Thanks, Lars On Sun, Apr 30, 2017 at 5:22 PM Jean-Baptiste Onofré mailto:j...@nanthrax.net>> wrote:

Re: Apache Beam Slack channel

Hi Should I use your Gmail address ? Regards JB On Apr 30, 2017, 16:58, at 16:58, Lars BK wrote: >Invitation not received, did something go wrong? > >Lars > >On Sun, Apr 30, 2017 at 3:57 PM Aviem Zur wrote: > >> Invitation sent. >> >> On Sun, Apr 30, 2017 at 4:41 PM Lars BK >wrote: >> >>> Hi,

Re: Slack Channel Invite

Done Regards JB On 04/24/2017 09:43 AM, Jonas Grabber wrote: Hello, could I have an invite to the Apache Beam Slack Channel, please? Best, Jonas Grabber -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Questions about writing data to Azure

ould be a DoFn>. I'm not sure what would happen if the result of the GroupBy couldn't fit in a single machine's memory though (say we uploaded a bunch of data during that 1 hour). Would the Iterable's records be split across multiple machines? If so, I like this solutio

Re: beam + scala + streamline

ner and sort of mesh beam with runner native code to integrate these features? Regards, Georg Jean-Baptiste Onofré mailto:j...@nanthrax.net>> schrieb am Mi., 12. Apr. 2017 um 20:04 Uhr: Hi Georg, You can use Java API via Scala, or you can use the Scio Scala DSL (this DSL use

Re: beam + scala + streamline

support beam in https://github.com/hortonworks/streamline? regards, Georg -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: 答复: 答复: Public in-progress I/O Transform list

Sure, I will push on my github. I will let you know. Regards JB On 04/12/2017 08:30 AM, Tang Jijun(上海_技术部_数据平台_唐觊隽) wrote: Great work.Could you show me your code? Thanks. -邮件原件- 发件人: Jean-Baptiste Onofré [mailto:j...@nanthrax.net] 发送时间: 2017年4月12日 14:22 收件人: user@beam.apache.org 主题

Re: 答复: Public in-progress I/O Transform list

's more "ambitious" than the producer mode. Regards JB On 04/12/2017 08:16 AM, Tang Jijun(上海_技术部_数据平台_唐觊隽) wrote: Hi Stephen, We need restIO module. I am looking forward to your pr about it. For restIO ,I create a jira https://issues.apache.org/jira/browse/BEAM-1946. -邮件原件---

Re: Public in-progress I/O Transform list

that's not listed, feel free to send a PR adding it to the list. Thanks, Stephen -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Regarding Beam Slack Channel

Done Regards JB On 03/28/2017 08:22 AM, Gareth Western wrote: Could I also have an invite please? On 28. mars 2017 07:43, Jean-Baptiste Onofré wrote: Oh my bad. I know there's some limitation about the number of members/history when using the free version. Thanks ! Regards JB On

Re: Regarding Beam Slack Channel

s, if you've got an invite -- please accept!) On Mon, Mar 27, 2017 at 4:39 AM, Jean-Baptiste Onofré mailto:j...@nanthrax.net>> wrote: Hi, As said during the week end, we reached the max number of members on the Slack channel (90). We are checking different ways to increase

Re: Regarding Beam Slack Channel

dd me to the Beam slack channel? Thanks. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Slack Channel Request

Unfortunately we reached the max number of people on Slack (90). Let me see what we can do. Regards JB On 03/24/2017 09:49 PM, Prabeesh K. wrote: Hi, Can someone please add me to the Apache Beam slack channel? Regards, Prabeesh K. -- Jean-Baptiste Onofré jbono...@apache.org http

Re: HDFS IO - Sink implemented or not?

th is implemented.'. Can this statement either be removed or extended to give note about ongoing implementation efforts (that are visible in HDFSFileSink.java)? -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: guava collections and kryo under spark runner

On 03/24/2017 09:10 AM, Antony Mayi wrote: I am on 0.6.0 thx, a. On Friday, 24 March 2017, 8:58, Jean-Baptiste Onofré wrote: Hi Antony, which Beam version are you using ? We did some improvement about guava shading recently, wanted to check if it's related. Regards JB On 03/24/201

Re: guava collections and kryo under spark runner

ethod.java:498) at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282) at java.lang.Thread.run(Thread.java:745) -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Kafka Offset handling for Restart/failure scenarios.

d ? > > Thanks, > Jins George -- Mingmin -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: grpc IO?

I guess I can create JIRA and see if I can contribute it? > > we need grpc IO... > > > > On Fri, 10 Mar 2017 at 10:08 Jean-Baptiste Onofré mailto:j...@nanthrax.net> > <mailto:j...@nanthr

Re: grpc IO?

Absolutely ! Thanks ! Regards JB On 03/10/2017 11:18 AM, Borisa Zivkovic wrote: ok thanks JB, I guess I can create JIRA and see if I can contribute it? we need grpc IO... On Fri, 10 Mar 2017 at 10:08 Jean-Baptiste Onofré mailto:j...@nanthrax.net>> wrote: Hi Borisa, Not

Re: grpc IO?

Hi Borisa, Not on my side, but definitely a good idea. Regards JB On 03/10/2017 10:58 AM, Borisa Zivkovic wrote: Hi guys, any reason why there is no grpc IO available? Is it maybe planned? thanks -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http

Re: Monitoring and Management Tools for Beam an Friends

M, Stadin, Benjamin wrote: Hi, I’m trying to collect a list of open source monitoring and management tools for Beam and supported runners. What do you use in your daily routine, and what’s your experience? Regards Ben -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Apache Beam @ JavaZone 2017?

talk about Apache Beam there this year. Is there anyone on the > > list who would be interested in submitting a presentation?> > > More about the conference here: https://2017.javazone.no/speak> > > Feel free to contact me if you have any questions!> > >

Re: How to use the new Coder interface / TableRowJsonCoder in 0.5.0?

ache/beam/sdk/coders/CoderRegistry.java#L94 -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: collect to local

I assuming it would need some type of Sink). Thanks, Antony. On Monday, 20 February 2017, 10:40, Jean-Baptiste Onofré mailto:j...@nanthrax.net>> wrote: Hi Antony, The Spark runner deals with caching/persist for you (analyzing how many time the same PColle

Re: collect to local

emory/process (something like calling .collect() on Spark rdd)? Do I need to implement custom Sink? Thanks for any clues, Antony. -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: New blog post: "Stateful processing with Apache Beam"

Hey Ken Just take a quick look and it's a great post ! Thanks Regards JB On Feb 13, 2017, 18:44, at 18:44, Kenneth Knowles wrote: >Hi all, > >I've just published a blog post about Beam's new stateful processing >capabilities: > >https://beam.apache.org/blog/2017/02/13/stateful-processing.ht

Re: Pico WordCount

gt;> >> >> > On Wed, Dec 7, 2016, 11:05 AM Robert Bradshaw > >> >> > wrote: >> >> >> >> >> >> Nice. Of course for ultimate conciseness, you should have gone >with >> >> >> Python >> >> >> :

Re: Streaming with Direct Runner

Hi Direct runner is able to read from unbounded source. The next transforms of your pipeline have to deal with unbounded pcollection. Regards JB On Feb 7, 2017, 22:16, at 22:16, Wyatt Frelot wrote: >Good evening all, > >I am working on project whose goal is to show the strength of beam and >bu

Re: beam vs camel

t;http://www.jesse-anderson.com/2016/07/question-and-answers-with-the-apache-beam-team/>, but haven't found much of it. Moreover I would like to know when I should use one or the other. If the decision is a matter of data volume or there are other elements that should be taken into account. R

Re: Documentation - Programming Guide - Creating a PCollection

Fully agree. It's better for tracking. Regards JB⁣​ On Jan 24, 2017, 18:04, at 18:04, Lukasz Cwik wrote: >Yes, please report documentation issues via JIRA (added you as a >contributor so that you can create it), also feel free to open a PR >addressing the issue. > >On Tue, Jan 24, 2017 at 5:34 A

Re: Regarding Beam Slack Channel

to please make it an open slack channel if possible. And if not, please enlarge the set of approvers as quickly as possible. On Sun, Jan 22, 2017 at 11:10 PM, Ritesh Kasat mailto:riteshka...@gmail.com>> wrote: Thanks Jean —Ritesh On 22-Jan-20

Re: Regarding Beam Slack Channel

I will. By the way, not sure it's possible (I gonna check) but maybe it would be great to have a open slack channel (like IRC) without invite needed. Regards JB⁣​ On Jan 23, 2017, 07:32, at 07:32, Ritesh Kasat wrote: >Hello, > >Can someone add me to the Beam slack channel. > >Thanks >Ritesh

Re: Beam Spark/Flink runner with DC/OS

find on the web. So some pointers are greatly appreciated. Thank you, Chaoran Yu -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: Extending Beam to Write/Read to Apache Accumulo.

Welcome and fully agree with Davor. You can count on me to do the review ! Regards JB⁣​ On Jan 12, 2017, 06:12, at 06:12, Davor Bonaci wrote: >Hi Wyatt -- welcome! > >If you'd like to write to a PCollection to Apache Accumulo's key/value >store, writing an new IO connector would be the best pat

Re: Some questions and kudos to everyone contributed to this project

clear those errors so I can see newer ones and clear the red notification :) Again thanks for this great project Ufuk -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com

Re: 回复：how to use key-value storage like redis with PCollection?

a lot. Ya-Feng -- 发件人：Jean-Baptiste Onofré 发送时间：2016年11月22日(星期二) 03:29 收件人：user 主　题：Re: how to use key-value storage like redis with PCollection? Hi Amir, I'm working on MqttIO right now, I will pu

<    1   2