Re: [DISCUSS] Next steps for update of Avro dependency in Beam

Hi, Thanks Alexey for bringing this topic up. I'd be in favor of 3 Best Etienne Le 12/05/2022 à 23:21, Brian Hulette a écrit : Regarding Option (3) "but keep and shade Avro for “core” needs as v.1.8.2 (still have an issue with CVEs)" Do we actually need to keep avro in core for any reason?

Remove support for Elasticsearch 5 and 6 ?

Hi all, Elastic no more supports Elastic 5 and 6 (1). We are in the middle of removing long overdue Elasticsearch 2 support but maybe it is time to remove support for ES5 and ES6 from Beam as well. WDYT ? [1] https://endoflife.date/elasticsearch Best Etienne

Re: [ANNOUNCE] New committer: Moritz Mack

Congrats Moritz ! Well deserved ! Etienne Le 10/03/2022 à 19:44, Sachin Agarwal a écrit : Congratulations Moritz! On Thu, Mar 10, 2022 at 10:44 AM Alexey Romanenko wrote: Hi everyone, Please join me and the rest of the Beam PMC in welcoming a new committer: Moritz Mack Mo

[ANNOUNCE] New committer: Evan Galpin

Hi all, Please join me and the rest of the Beam PMC in welcoming a new committer: Evan Galpin Since joining the Beam community Evan has done lots of contributions to IOs mainly Elasticsearch, but also to SDK transforms. He also gave support on the ML and tested releases. Considering these

Re: Intro

Welcome onboard Moritz ! Best Etienne On 22/10/2021 15:52, Moritz Mack wrote: Hi all, I’m very much looking forward to start contributing to Beam and just want to briefly introduce myself. My name is Moritz (mosche) and I’m working together with Alexey and Etienne. Having worked mostly w

Spark Structured Streaming runner migrated to Spark 3

Hi all, Just to let you know that Spark Structured Streaming runner was migrated to Spark 3. Enjoy ! Etienne

Re: Spark Structured Streaming Runner Roadmap

Hi, Sorry for the late answer: the streaming mode in spark structured streaming runner is stuck because of spark structured streaming framework implementation of watermark at the apache spark project side. See https://echauchot.blogspot.com/2020/11/watermark-architecture-proposal-for.html be

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.11.0

+1 (binding) on releasing vendored bytebuddy for testing in https://github.com/apache/beam/pull/14824 Etienne On 19/05/2021 23:43, Kai Jiang wrote: +1 (non-binding) On Wed, May 19, 2021 at 12:23 PM Jan Lukavský > wrote: +1 (non-binding) verified correct shad

Re: [DISCUSS] Drop support for Flink 1.8 and 1.9

Hi, +1 on drop Etienne On 12/03/2021 20:39, Ismaël Mejía wrote: Do we now support 1.8 through 1.12? Yes and that's clearly too much given that the Flink community only support the two latest release. It also hits us because we run tests for all those versions on precommit. On Fri, Mar 12, 20

Re: Unit tests vs. Integration Tests

Big +1 on using testcontainers rather than embedded real backends. That is what we plan to use for ES refactoring. I'm a strong believer that mocks are useless to replace complex backends. Testing things like IOs against mocks are a almost certain failure because mocks cannot be representative

Re: Combine with multiple outputs case Sample and the rest

Hi all, Regarding leveraging the Pardo part of Combine (Combine <=> GBK + Pardo) to have multiple outputs, please note that most of the time Combine is translated by the runners with a native (destination-tech) Combine and not a GBK + Pardo. Regarding using the Stateful DoFn I agree with Ken

Re: ElasticsearchIO.Write() dynamic ES indices

+1 What is not supported yet is the wildcard indexes (index* pattern). It will be when the refactoring (use high level ES objects rather than low level REST String objects) is done. Best Etienne On 13/01/2021 16:45, Brian Hulette wrote: It looks like you should be able to accomplish this wi

Re: [blog about Beam]

Hi Pablo, Thanks for reading these. Etienne On 10/11/2020 20:09, Pablo Estrada wrote: Thanks Etienne! I read your post on why we can't have multiple aggregations in Spark Streaming. It was informative. Thanks for writing these! Best -P. On Tue, Nov 10, 2020 at 3:39 AM Etienne Cha

[blog about Beam]

Hi all, In case anyone is interested, I started a blog [1] this year about big data technologies. There are 8 articles so far and they are mainly related to Beam even if some are related to Spark (but with the knowledge acquired while working on the Beam Spark runner). I just published the f

[Beam Spark Structured Streaming runner]

Hi all, In case anyone wanted some details about the new Beam runner based on Spark Structured Streaming framework, here are 2 talks I gave at the ApacheCon this year and last year about this subject. https://www.youtube.com/watch?v=oEehQwOEFvg https://www.youtube.com/watch?v=_dCmV1ZW3M4 Be

Re: Shutting down Perfkit Explorer

Thanks Kamil ! Etienne On 25/09/2020 16:10, Kamil Wasilewski wrote: They have been migrated: http://metrics.beam.apache.org The website doesn't support HTTPS, so if you can't access it, you may need to add an exception to your browser extension. On Fri, Sep 25, 2020 at 3:49

Re: Shutting down Perfkit Explorer

Hi all, I'm coming a bit after the battle, but how should we access the dashboards now (load tests, nexmark etc...)? Are the dashboards lost or have they migrated to another environment ? Thanks Etienne On 24/09/2020 13:51, Robert Burke wrote: LGTM Good clear message. On Thu, Sep 24, 2020

Re: Chronically flaky tests

Hi all, +1 on ping the assigned person. For the flakes I know of (ESIO and CassandraIO), they are due to the load of the CI server. These IOs are tested using real embedded backends because those backends are complex and we need relevant tests. Counter measures have been taken (retrial insid

Re: [ANNOUNCE] New PMC Member: Alexey Romanenko

Congrats Alexey ! Well deserved ! Etienne On 17/06/2020 16:30, Gleb Kanterov wrote: Congratulations! Thanks for your hard work On Wed, Jun 17, 2020 at 1:11 PM Alexey Romanenko mailto:aromanenko@gmail.com>> wrote: Thank you Ismaël and everybody! Happy to be a part of Beam commun

Re: Is org.apache.beam.sdk.transforms.FlattenTest.testFlattenMultipleCoders supposed to be supported ?

ts a choice between performance and debuggability. On Wed, Dec 11, 2019 at 3:47 AM Etienne Chauchot wrote: Ok, Thanks Kenn. Le Flatten javadoc says that by default the coder of the output should be the coder of the first input. But in the test, it sets the output coder to something differen

Re: Add options to CassandraIO

Hi Nathan, Thanks for raising this, and thanks for the PR proposal. I would recommend (as it was done in other IOs such as ElasticsearchIO) the third solution: you could add a method called withConnectTimeout(Integer) to both the Read and Write builders of the IO (there is no common conf obje

Re: A new reworked Elasticsearch 7+ IO module

on and indicates urgency)  - reject completely Kenn On Fri, Mar 6, 2020 at 2:26 AM Etienne Chauchot <mailto:echauc...@apache.org>> wrote: Hi all, it's been 3 weeks since the survey on ES versions the users use. The survey received very few responses: only 9 responses f

Re: A new reworked Elasticsearch 7+ IO module

Etienne Chauchot. On 06/03/2020 11:26, Etienne Chauchot wrote: Hi all, it's been 3 weeks since the survey on ES versions the users use. The survey received very few responses: only 9 responses for now (multiple versions possible of course). The responses are the following: ES2: 0 cl

Re: A new reworked Elasticsearch 7+ IO module

f ES2 support but for now it is still not very representative. I'm cross-posting to @users to let you know that I'm closing the survey within 1 or 2 weeks. So please respond if you're using ESIO. Best Etienne On 13/02/2020 12:37, Etienne Chauchot wrote: Hi Cham, thanks for you

Re: Beam Emitted Metrics Reference

Hi, There is a doc about metrics here: https://beam.apache.org/documentation/programming-guide/#metrics You can also export the metrics to sinks (REST http endpoint and Graphite), see MetricsOptions class for configuration. Still, there is no doc for export on website, I'll add some Best

Re: GroupIntoBatches not Working properly for Direct Runner Java

Hi, +1 to what Kenn asked: your pipeline is in streaming mode and GIB preserves windowing, the elements are buffered until one of these conditions are true: batchsize reached or end of window. I your case I think it is the second one. Best Etienne On 28/02/2020 19:15, Kenneth Knowles wrote

Re: [ANNOUNCE] New committer: Alex Van Boxel

Congrats Alex ! Well deserved ! Etienne On 20/02/2020 12:23, Michał Walenia wrote: Congratulations! On Thu, Feb 20, 2020 at 2:31 AM Chamikara Jayalath mailto:chamik...@google.com>> wrote: Congrats Alex! On Wed, Feb 19, 2020 at 7:21 AM Ryan Skraba mailto:r...@skraba.com>> wrote:

Re: big data blog

Hi all, I just sent the link to the blog articles on @ApacheBeam twitter as Kenn suggested. Etienne On 10/02/2020 10:01, Etienne Chauchot wrote: Yes sure, Here is the link to the spreadsheet for review of the tweet: https://docs.google.com/spreadsheets/d

Re: A new reworked Elasticsearch 7+ IO module

Jayalath wrote: On Thu, Feb 6, 2020 at 8:13 AM Etienne Chauchot <mailto:echauc...@apache.org>> wrote: Hi, please see my comments inline On 06/02/2020 16:24, Alexey Romanenko wrote: Please, see my comments inline. On 6 Feb 2020, at 10:50, Etienne Chauchot mail

Re: big data blog

gle.com>> wrote: Cool! On Fri, Feb 7, 2020 at 7:24 AM Etienne Chauchot mailto:echauc...@apache.org>> wrote: Hi all, FYI, I just started a blog around big data technologies and for now it

big data blog

Hi all, FYI, I just started a blog around big data technologies and for now it is focused on Beam. https://echauchot.blogspot.com/ Feel free to comment, suggest or anything. Etienne

Re: A new reworked Elasticsearch 7+ IO module

Hi, please see my comments inline On 06/02/2020 16:24, Alexey Romanenko wrote: Please, see my comments inline. On 6 Feb 2020, at 10:50, Etienne Chauchot <mailto:echauc...@apache.org>> wrote: 1. regarding version support: ES v2 is no more maintained by Elastic sinc

Re: A new reworked Elasticsearch 7+ IO module

Jayalath, my aswers are inline. Have a good day ! Ludovic Le mer. 5 févr. 2020 à 20:15, Chamikara Jayalath <mailto:chamik...@google.com>> a écrit : On Wed, Feb 5, 2020 at 6:35 AM Etienne Chauchot mailto:echauc...@apache.org>> wrote: Still there is something I don&

Re: A new reworked Elasticsearch 7+ IO module

his is really great job! As I recall, we already have some support of Elasticsearch7 in current ElasticsearchIO (afaik, at least they are compatible), thanks to Zhong Chen and Etienne Chauchot, who were working on adding this [1][2] and it should be releas

Re: A new reworked Elasticsearch 7+ IO module

support of Elasticsearch7 in current ElasticsearchIO (afaik, at least they are compatible), thanks to Zhong Chen and Etienne Chauchot, who were working on adding this [1][2] and it should be released in Beam 2.19. Would you think you can leverage t

Re: A new reworked Elasticsearch 7+ IO module

can be included in ASF projects. Best, Etienne On 25/01/2020 14:23, Ludovic Boutros wrote: Hi all, First, thank you for your great answers. I thank Zhong Chen and Etienne Chauchot for their great job on this too ! Alexey and Chamikara, I understand your point of view. Actually, I have the

[Spark Structured Streaming runner] perfs and encoders

Hi all, good news ! I did some refactoring of the encoders to improve maintenability and replace as much as possible string generated code with compiled code and the perf results are awesome ! Best Etienne

Re: Is org.apache.beam.sdk.transforms.FlattenTest.testFlattenMultipleCoders supposed to be supported ?

oblem. Flatten just became quite an interesting transform, for me :-) Kenn On Tue, Dec 10, 2019 at 12:37 AM Etienne Chauchot mailto:echauc...@apache.org>> wrote: Hi all, I have an interrogation around testFlattenMultipleCoders test: This test uses 2 collections 1. long

Is org.apache.beam.sdk.transforms.FlattenTest.testFlattenMultipleCoders supposed to be supported ?

Hi all, I have an interrogation around testFlattenMultipleCoders test: This test uses 2 collections 1. long and null data encoded using NullableCoder(BigEndianLongCoder) 2. long data encoded using VarlongCoder It then flattens the 2 collections and set the coder of the resulting collection t

Re: consurrent PRs

Hi all, FYI, I closed the most recent one (with explanation and a sorry message): https://github.com/apache/beam/pull/10025 Etienne On 26/11/2019 17:06, Robert Bradshaw wrote: On Tue, Nov 26, 2019 at 6:15 AM Etienne Chauchot wrote: Hi guys, I wanted your opinion about something: I have 2

consurrent PRs

Hi guys, I wanted your opinion about something: I have 2 concurrent PRs that do the same: https://github.com/apache/beam/pull/10010 https://github.com/apache/beam/pull/10025 The first one is a bit better because it addresses a deprecation that the other does not address. Except that they are

Re: [spark structured streaming runner] available on master

Forgot to say thanks everyone for their contribution to this especially Alexey, Ryan and Ismael. Etienne On 20/11/2019 17:12, Etienne Chauchot wrote: Hi all, I'm glad to announce that the new Spark runner based on Spark structured streaming framework has been merged into master !

[spark structured streaming runner] available on master

Hi all, I'm glad to announce that the new Spark runner based on Spark structured streaming framework has been merged into master ! It is not based on RDD/DStream API. See https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html It is still experimental, its coverage

Re: [spark structured streaming runner] merge to master?

Ok for 1 jar with the 2 runners then. I'll add the banner to the logs and the Experimental in the code and in in the javadocs. Thanks for your opinions guys ! Etienne On 08/11/2019 18:50, Kenneth Knowles wrote: On Thu, Nov 7, 2019 at 5:32 PM Etienne Chauchot <mailto:echauc...@ap

Re: [spark structured streaming runner] merge to master?

lay a big role comparing to the whole size of shaded jar of users job. Even more, I think it will be quite confusing for users to choose which jar to use if we will have 3 different ones for similar purposes. Though, let’s see what others think. >>> >>

Re: [spark structured streaming runner] merge to master?

eased". +1 On 28 Oct 2019, at 10:27, Etienne Chauchot <mailto:echauc...@apache.org>> wrote: Hi guys, Any opinions on the point2 communication to users ? Etienne On 24/10/2019 15:44, Etienne Chauchot wrote: Hi guys, I'm glad to announce that the PR for the merge to m

Re: [spark structured streaming runner] merge to master?

x provides both RDD/Dstream and Structured Streaming. Etienne Kenn On Thu, Oct 10, 2019 at 11:50 AM Robert Bradshaw <mailto:rober...@google.com>> wrote: On Thu, Oct 10, 2019 at 12:39 AM Etienne Chauchot mailto:echauc...@apache.org>> wrote: > > Hi guys

Re: [spark structured streaming runner] merge to master?

Hi Robert comments inline: On 10/10/2019 20:49, Robert Bradshaw wrote: On Thu, Oct 10, 2019 at 12:39 AM Etienne Chauchot wrote: Hi guys, You probably know that there has been for several months an work developing a new Spark runner based on Spark Structured Streaming framework. This work is

Re: [spark structured streaming runner] merge to master?

, it makes sense. Question is: do we keep the "old" spark runner for a while or not (or just keep on previous version/tag on git) ? Regards JB On 10/10/2019 09:39, Etienne Chauchot wrote: Hi guys, You probably know that there has been for several months an work developing a ne

[spark structured streaming runner] merge to master?

Hi guys, You probably know that there has been for several months an work developing a new Spark runner based on Spark Structured Streaming framework. This work is located in a feature branch here: https://github.com/apache/beam/tree/spark-runner_structured-streaming To attract more contribu

Re: Feedback on how we use Apache Beam in my company

Very nice ! Thanks ccing dev list Etienne On 09/10/2019 16:55, Pierre Vanacker wrote: Hi Apache Beam community, We’ve been working with Apache Beam in production for a few years now in my company (Dailymotion). If you’re interested to know how we use Apache Beam in combination with Goog

Re: Cassandra flaky on Jenkins?

Hi all,I just created a PR (1) that tries to fix the flakiness of CassandraIOTest (underlying ticket https://jira.apache.org/jira/browse/BEAM-8025 that was assigned to me). We will see with the test repetitions if it is no more flaky. JB, I don't know if my PR will also fix the ticket https://i

Re: Pointers on Contributing to Structured Streaming Spark Runner

27;m very happy to join the sync-up meeting. Please > > forward the meeting info to me. I am based in > > California, US and hopefully the time will work :). > > Thanks, > > Xinyu > > On Wed, Sep 18, 2019 at 6:39 AM Etienne Chauchot > > wrote: > > >

Re: Pointers on Contributing to Structured Streaming Spark Runner

supporting multiple-aggregations > > in its streaming mode but design is ongoing. Do you have a link or > > something else to their design discussion/doc? > > > > > > -Rui > > On Wed, Sep 11, 2019 at 5:10 PM Etienne Chauchot > > wrote: > > > Hi

Re: Pointers on Contributing to Structured Streaming Spark Runner

to their design discussion/doc? > > > -Rui > On Wed, Sep 11, 2019 at 5:10 PM Etienne Chauchot wrote: > > Hi Rahul,Sure, and great ! Thanks for proposing !If you want details, here > > is the presentation I did 30 mins ago at > > the apachecon. You will find the vide

[Off for 3 weeks]

Hi guys, Just to let you know, I'll be off for 3 weeks starting tonight. See you when I get back Etienne

Re: [Current spark runner] Combine globally translation is risky and not very performant

of potential OOM will be just moved to some worker. Or > would you see some other option? > Jan > [1] https://issues.apache.org/jira/browse/BEAM-7574 > On 6/27/19 11:43 AM, Etienne Chauchot wrote: > Hi guys, > FYI, while I'm working on the combine translation for the new spark r

[Current spark runner] Combine globally translation is risky and not very performant

Hi guys, FYI, while I'm working on the combine translation for the new spark runner poc, I saw something that do not seem right in the current runner: https://issues.apache.org/jira/browse/BEAM-7647 Best, Etienne

Re: Congrats to Beam's first 6 Google Open Source Peer Bonus recipients!

Congrats ! Etienne Le lundi 06 mai 2019 à 22:28 -0700, Joana Filipa Bernardo Carrasqueira a écrit : > Thank you for your work in the community and Congratulations!! :) > > On Thu, May 2, 2019 at 9:44 PM Ankur Goenka wrote: > > Congratulations and thank you for making Beam awesome! > > From: Ch

Re: [ANNOUNCE] New committer announcement: Boyuan Zhang

Congrats ! Etienne Le vendredi 12 avril 2019 à 15:53 -0700, Thomas Weise a écrit : > Congrats! > > On Thu, Apr 11, 2019 at 6:03 PM Reuven Lax wrote: > > Congratulations Boyuan! > > On Thu, Apr 11, 2019 at 4:53 PM Ankur Goenka wrote: > > > Congrats Boyuan! > > > On Thu, Apr 11, 2019 at 4:52 PM Ma

Re: [ANNOUNCE] New committer announcement: Mark Liu

Congrats ! Le lundi 25 mars 2019 à 10:55 -0700, Chamikara Jayalath a écrit : > Congrats Mark! > > On Mon, Mar 25, 2019 at 10:50 AM Alexey Romanenko > wrote: > > Congratulations, Mark! > > > > > On 25 Mar 2019, at 18:36, Mark Liu wrote: > > > > > > Thank you all! It's a great pleasure to work

Structured streaming based spark runner.

Hi guys, As part of the ongoing work on spark runner POC based on structured streaming framework, I sketched up a design doc (1) to share context and design principles. Feel free to comment. [1] https://s.apache.org/spark-structured-streaming-runner Etienne

Re: CVE audit gradle plugin

mercredi 24 avril 2019 à 15:56 +0200, Etienne Chauchot a écrit : > Hi all,FYI I just submitted a PR (1) to add the CVE audit plugin to the build > as an optional task gradlew audit -- > info. > [1] https://github.com/apache/beam/pull/8388 > Etienne > Le mardi 23 avril 2019 à 17

Re: [VOTE] Release 2.12.0, release candidate #4

Hi, Thanks for all your work and patience Andrew ! PS: as a side note, there were 5 binding votes (I voted +1) Etienne Le jeudi 25 avril 2019 à 11:16 -0700, Andrew Pilloud a écrit : > I reran the Nexmark tests, each runner passed. I compared the numbers > on the direct runner to the dashboard an

Re: CVE audit gradle plugin

Hi all,FYI I just submitted a PR (1) to add the CVE audit plugin to the build as an optional task gradlew audit --info. [1] https://github.com/apache/beam/pull/8388 Etienne Le mardi 23 avril 2019 à 17:25 +0200, Etienne Chauchot a écrit : > Hi,should I merge my branch > https://gith

Re: [VOTE] Release 2.12.0, release candidate #4

; > > > degraded performance around 04/10 it is not part of the release > > > > > > > > we are > > > > > > > > > > > > > > > > voting, so please consider reverting your -1. > > > > > > >

Re: [VOTE] Release 2.12.0, release candidate #4

> > > > > > > > > > > > > > > However the issue you are reporting looks important, from a quick > > > > > > look > > > > > > > > > > > > I am guessing it could be related to BEAM-5775 that was merged on > > > > > > > > > > > > 12/04 however the performance regressions started

Re: CVE audit gradle plugin

y chance we can get this > > > > > > running > > > > > > > > > > > > in or main repo maybe in a weekly basis like we do for the > > > > > > dependency > > > > > >

Re: [VOTE] Release 2.12.0, release candidate #4

Hi guys ,I will vote -1 (binding) on this RC (although degradation is before RC4 cut date). I took a look at Nexmark graphs for the 3 major runners :- there seem to have functional regressions on Dataflow: https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856 . 13 queries

Re: [PROPOSAL] commit granularity in master

sily see before > > > > > > > doing the merge. > > > > > > > > > > > > > > > I would like to propose a small modification to the commit title style > > > > > > > > on that guide. We use two brackets to e

Re: [PROPOSAL] commit granularity in master

ging). > > In my opinion, the most important rule is that every commit should be atomic > in terms of added/fixed functionality and > rolling it back should not break master branch. > > [1] > https://beam.apache.org/contribute/committer-guide/#pull-request-review-objectives &

[PROPOSAL] commit granularity in master

Hi all, It has already been discussed partially but I would like that we agree on the commit granularity that we want in our history. Some features were squashed to only one commit which seems a bit too granular to me for a big feature. On the contrary I see PRs with very small commits such as "a

[spark runner dataset POC] workCount works !

Hi guys, We are glad to announce that the spark runner POC that was re-written from scratch using the structured-streaming framework and the dataset API can now run WordCount ! It is still embryonic. For now it only runs in batch mode and there is no fancy stuff like state, timer, SDF, metrics,

Re: [PROPOSAL] Preparing for Beam 2.12.0 release

Sounds great, thanks for volunteering to do the release. Etienne Le mercredi 13 mars 2019 à 12:08 -0700, Andrew Pilloud a écrit : > Hello Beam community! > Beam 2.12 release branch cut date is March 27th according to the release > calendar [1]. I would like to volunteer > myself to do this release

Re: JIRA hygiene

ime to first understand a problem and then the PR to realise > > nothing has to be done anymore. Or not > > knowing what's left out and for what reason. But of course, this is another > > issue which we definitely need to invest > > time into - kenn already asked

Re: New Contributor

Welcome ! Etienne Le mardi 05 mars 2019 à 16:28 -0800, Daniel Oliveira a écrit : > Welcome to Beam Boris! > > On Tue, Mar 5, 2019 at 2:03 PM Mikhail Gryzykhin wrote: > > Welcome to the community! > > --Mikhail > > Have feedback? > > > > > > On Tue, Mar 5, 2019 at 1:53 PM Ruoyun Huang wrote:

Re: New contributor to Beam

Welcome ! Etienne Le lundi 11 mars 2019 à 14:15 -0700, Kenneth Knowles a écrit : > Welcome! > > On Mon, Mar 11, 2019 at 12:22 PM Melissa Pashniak > wrote: > > Welcome! > > > > > > > > On Mon, Mar 11, 2019 at 12:16 PM Suneel Marthi > > wrote: > > > Welcome Aizhamal > > > > > > Sent from my

Re: hi from DevRel land

Welcome onboard Reza Etienne Le mardi 12 mars 2019 à 09:10 -0700, Rose Nguyen a écrit : > Welcome, Reza! Really excited to have you! > > On Tue, Mar 12, 2019 at 9:00 AM Reza Ardeshir Rokni wrote: > > Thanx folks! > > > > Oppsy on the link, here it is again: > > https://stackoverflow.com/questi

Re: Apache Beam Newsletter - February/March 2019

gt; > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > https://github.com/apache/beam/tree/master/website/src/community > > > > > >

Re: JIRA hygiene

a commit on Feb 25, 2019 11:48 PM PST. > > > > > > And that is a commit on the release > > > > > > branch. > > > > > > > > > > > > After cutting the release branch, I only merged cherry picks from > > > > > > master

Re: [VOTE] Release 2.11.0, release candidate #2

e PRs. > > Does this answer your question? > > Ahmet > > [1] > https://github.com/apache/beam/commit/a103edafba569b2fd185b79adffd91aaacb790f0 > On Mon, Mar 11, 2019 at 1:50 AM Etienne Chauchot wrote: > > @Ahmet sorry I did not have time to check 2.11 release but a f

Re: [ANNOUNCE] New committer announcement: Raghu Angadi

Congrats ! Well deserved Etienne Le lundi 11 mars 2019 à 13:22 +0100, Alexey Romanenko a écrit : > My congratulations, Raghu! > > > On 8 Mar 2019, at 10:39, Łukasz Gajowy wrote: > > > > Congratulations! :) > > pt., 8 mar 2019 o 10:16 Gleb Kanterov napisał(a): > > > Congratulations! > > > On Thu

Re: [VOTE] Release 2.11.0, release candidate #2

@Ahmet sorry I did not have time to check 2.11 release but a fellow Beam contributor drew my attention on something: the 2.11 release tag points on commit of 02/26 and this[1] PR was merged 02/20 and that [2] PR was merged on 02/18. So, both commits should be in the released code but they are not

Re: [BEAM-6759] CassandraIOTest failing in presubmit in multiple PRs

Hi guys, As I introduced embedded Cassandra backend in the IO tests, I'll fix this issue. It is very common (as discussed) that embedded backends cause flakiness. But it is the price to pay for more relevant tests :) Etienne Le lundi 04 mars 2019 à 11:28 +0100, Maximilian Michels a écrit : > Hey

Re: Apache Beam Newsletter - February/March 2019

Hi,I would add in what's been done: Work on cassandraIO (Etienne Chauchot, Mathieu Blanchard, Frank Shahar) : refactorings, bugfixes, new where clause, security fix Etienne Le lundi 04 mars 2019 à 18:36 +0100, Suneel Marthi a écrit : > Is this the final draft? - we had 2 beam talks at

CVE audit gradle plugin

Hi guys, I came by this [1] gradle plugin that is a client to the Sonatype OSS Index CVE database. I have set it up here in a branch [2], though the cache is not configured and the number of requests is limited. It can be run with "gradle --info audit" It could be nice to have something like t

Re: Signing off

Thank you for your contributions Scott ! Your new project seems very fun. Enjoy ! Etienne Le vendredi 15 février 2019 à 15:01 +0100, Ismaël Mejía a écrit : > Your work and willingness to make Beam better will be missed.Good luck for > the next phase! > On Fri, Feb 15, 2019 at 1:39 PM Łukasz Gajow

Re: [VOTE] Release 2.10.0, release candidate #3

Thanks Robert ! Etienne Le vendredi 08 février 2019 à 16:42 +0100, Robert Bradshaw a écrit : > +1 (binding) > > I have verified that the artifacts and their checksums/signatures look good, > and also checked the Python wheels > against simple pipelines. > On Fri, Feb 8, 2019

Re: [VOTE] Release 2.10.0, release candidate #3

Hi,I did the same visual checks of Nexmark that I did on RC2 for both functional regressions (output size) and performance regressions (execution time) on all the runners/modes for RC3 cut date (02/06) and I saw no regression except the one that I already mentioned (end of october perf degradatio

Re: Another another new contributor! :)

Hi, Help much appreciated !And welcome ! Etienne Le jeudi 07 février 2019 à 15:44 +0800, Reza Ardeshir Rokni a écrit : > Welcome! > On Tue, 5 Feb 2019 at 23:34, Kenneth Knowles wrote: > > Welcome Kyle! > > On Tue, Feb 5, 2019 at 4:34 AM Maximilian Michels wrote: > > > Welcome Kyle! Excited to see

Re: [VOTE] Release 2.10.0, release candidate #1

Hi, I just fixed both (one was not a bug but an error in test code) in this [1] PR[1] https://github.com/apache/beam/pull/7751 Etienne Le mardi 05 février 2019 à 17:37 +0100, Etienne Chauchot a écrit : > Hi guys, > I just found 2 bugs while replacing the mock in CassandraIO by a

Re: [DISCUSSION] UTests and embedded backends

100, Etienne Chauchot a écrit : > Guys, > I will try using mocks where I see it is needed. As there is a current PR > opened on Cassandra, I will take this > opportunity to add the embedded cassandra server > (https://github.com/jsevellec/cassandra-unit) to the UTests.Ticket >

Re: [VOTE] Release 2.10.0, release candidate #2

Hi,I checked Nexmark on both output size (functional regression detection) and run time (performance regression). The only thing I see is a performance regression on query7 (side input + fanout) in spark runner but this regression is there since the previous release cut.Indeed 2.9 was cut on 18/1

Re: [VOTE] Release 2.10.0, release candidate #1

Hi guys, I just found 2 bugs while replacing the mock in CassandraIO by a proper instance: https://issues.apache.org/jira/browse/BEAM-6592https://issues.apache.org/jira/browse/BEAM-6591 I don't think they are release blockers because they have been there since CassandraIO first version.One of the

Re: BEAM-6324 / #7340: "I've pretty much given up on the PR being merged. I use my own fork for my projects"

Jenkins jobs). Given > > > that there might be misunderstandings > > > authors of such commits should give a clear message saying "do not merge > > > yet" or "not ready for review" in title > > > or comments or even close such PR and reop

Re: [ANNOUNCE] New PMC member: Etienne Chauchot

> > > > > > <mailto:sc...@apache.org>> wrote: > > > > > > > > > > > > > > Congrats Etienne! > > > > > > > > > > > > > > On Fri, Jan 25, 201

Re: [DISCUSSION] UTests and embedded backends

e they are > often anexpensive (in production and > maintenance) way to get what amounts tolow true coverage. > On Mon, Jan 28, 2019 at 11:16 AM Etienne Chauchot > wrote: > > Guys, > I will try using mocks where I see it is needed. As there is a current PR > opened on Cassa

Re: [DISCUSSION] UTests and embedded backends

s easy to embed and the backend behavior is predictable, then itmakes > sense.In other cases, we can fallback to > mock. > RegardsJB > On 21/01/2019 10:07, Etienne Chauchot wrote: > Hi guys, > Lately I have been fixing various Elasticsearch flakiness issues in theUTests > by: in

Re: BEAM-6324 / #7340: "I've pretty much given up on the PR being merged. I use my own fork for my projects"

Sure it's a pity than this PR got unnoticed and I think it is a combination of factors (PR date around Christmas, the fact that the author forgot - AFAIK - to ping a reviewer in either the PR or the ML). I agree with Rui's proposal to enhance visibility of the "how to get a reviewed" process. IM

Re: [ANNOUNCE] New committer announcement: Gleb Kanterov

Congrats Gleb and welcome onboard ! Etienne Le vendredi 25 janvier 2019 à 10:39 +0100, Alexey Romanenko a écrit : > Congrats to Gleb and welcome on board! > > > On 25 Jan 2019, at 09:22, Tim Robertson wrote: > > > > Welcome Gleb and congratulations! > > > > On Fri, Jan 25, 2019 at 8:06 AM Kenne

  1   2   3   4   >