Re: [PROPOSAL] Test performance of basic Apache Beam operations

neat! left a comment or two On Mon, Sep 3, 2018 at 3:53 AM Łukasz Gajowy wrote: > Hi all! > > I'm bumping this (in case you missed it). Any feedback and questions are > welcome! > > Best regards, > Łukasz > > pon., 13 sie 2018 o 13:51 Jean-Baptiste Onofré > napisał(a): > >> Hi Lukasz, >> >> Tha

Re: [PROPOSAL] Prepare Beam 2.7.0 release

+1, thanks for volunteering, Charles! On Mon, Aug 20, 2018 at 12:09 PM Charles Chen wrote: > Thank you Andrew for pointing out my mistake. We should follow the > calendar and aim to cut on 8/29, not 9/7 as I incorrectly wrote earlier. > > On Mon, Aug 20, 2018 at 12:02 PM Andrew Pilloud > wrote

Re: [Discuss] Add EXTERNAL keyword to CREATE TABLE statement

Strictly speaking, they are not necessarily tables either. We could also introduce something like CREATE EXTERNAL DATA SOURCE (a-la T-SQL ), if it's somehow advantageous for us to le

Re: [Discussion] Clarify the support story for released Beam versions

I think this will great for the project! It's worked well for others (such as Ubuntu). I like that this remains compatible with our desire to release every six weeks, while keeping the support/patch load manageable. Release: +1 single process. This is just a statement of what we commit to service.

Re: [HELP] Blog post for 2.6.0 Release

Howdy! https://github.com/apache/beam-site/pull/536 should take care of this. Cheers, r On Mon, Aug 13, 2018 at 9:14 AM Austin Bennett wrote: > Alexey, > > I believe seeing the same. > > Best, > Austin > > On Mon, Aug 13, 2018 at 8:54 AM, Alexey Romanenko < > aromanenko@gmail.com> wrote: >

[DISCUSS] Communicating in dev@

Hi, I think it's important we all discuss in the community what we want to do to make sure we communicate effectively. I ask because I've seen preferences expressed in dev@, and want to make sure we're conscious about our practices. I think we all want discussions to be accessible to all members

Re: [VOTE] Community Examples Repository

Here is the Rose', David's, and Gris' proposal in text form, I hope the copy/paste helps: Apache Beam Examples Repository Authors: Rose Nguyen (rtngu...@google.com), David Cavazos (dcava...@google.com), Gris Cuevas (g...@apache.org) Status: Proposal Created: 2018-07-30 Updated: 2018-07-30 Summ

Re: [Vote] Dev wiki engine

-1 .md! :-) On Thu, Jul 19, 2018 at 6:38 PM Gaurav Thakur wrote: > +1 for confluence > > On Fri, Jul 20, 2018 at 11:42 AM Thomas Weise wrote: > >> +1 for Confluence >> >> >> On Thu, Jul 19, 2018 at 4:00 PM Kai Jiang wrote: >> >>> +1 Apache Confluence >>> >>> On Thu, Jul 19, 2018, 15:18 Lukasz

Re: CODEOWNERS for apache/beam repo

+1 using blame -- nifty :) On Mon, Jul 16, 2018 at 2:31 AM Huygaa Batsaikhan wrote: > +1. This is great. > > On Sat, Jul 14, 2018 at 7:44 AM Udi Meiri wrote: > >> Mention bot looks cool, as it tries to guess the reviewer using blame. >> I've written a quick and dirty script that uses only CODEO

Re: CODEOWNERS for apache/beam repo

+1! On Tue, Jul 10, 2018 at 8:51 AM Robert Burke wrote: > +1 > If non-committers are welcome in the file, I'm happy to assist Henning > with Go SDK reviews.(@lostluck) > > > On Tue, Jul 10, 2018, 8:47 AM Alexey Romanenko > wrote: > >> +1, >> Udi, thank you for taking care of this! >> I added my

Re: Parallelizing test runs

tself runs faster -- I believe it's on Alan's radar Cheers, r On Mon, Jul 2, 2018 at 4:23 PM Rafael Fernandez wrote: > Done! > > On Mon, Jul 2, 2018 at 4:10 PM Scott Wegner wrote: > >> Hey Rafael, looks like we need more 'INSTANCE_TEMPLATES' quota [1]. Ca

Re: Parallelizing test runs

ll/5861#issuecomment-401963630 > > On Mon, Jul 2, 2018 at 11:33 AM Rafael Fernandez > wrote: > >> OK, Scott just sent https://github.com/apache/beam/pull/5860 . Quotas >> should not be a problem, if they are, please file a JIRA under gcp-quota. >> >> Cheers,

Re: Parallelizing test runs

t fork is run on a different gradle worker, so the number of >>> parallel test runs is limited to the max number of workers configured which >>> is controlled here: >>> >>> https://github.com/apache/beam/blob/fbfe6ceaea9d99cb1c8964087aafaa2bc2297a03/.test-infra/

Re: Parallelizing test runs

s much easier to manage. > > Reuven > > On Sat, Jun 30, 2018 at 9:32 AM Rafael Fernandez > wrote: > >> +Reuven Lax told me yesterday that he was waiting for >> some test to be scheduled and run, and it took 6 hours or so. I would like >> to help reduce these

Parallelizing test runs

+Reuven Lax told me yesterday that he was waiting for some test to be scheduled and run, and it took 6 hours or so. I would like to help reduce these wait times by increasing parallelism. I need help understanding the continuous minimum of what we use. It seems the following is true: - There

Re: [DISCUSS] Automation for Java code formatting

. It also lets a committer fix up a good PR > and merge it even if the contributor is, say, asleep. > ​This is a great practice, review the technical part, use the tool to address the mechanical part. ​ > > Kenn > > On Wed, Jun 27, 2018 at 9:24 AM Rafael Fernandez > wrote: &

Re: Jenkins Groovy files naming convention

+1 CamelCase. Feels "normal" to me in Groovy. On Wed, Jun 27, 2018 at 9:18 AM Lukasz Cwik wrote: > I don't really have a strong preference. > > On Wed, Jun 27, 2018 at 9:13 AM Łukasz Gajowy > wrote: > >> Hi all, >> >> I think we should change the naming convention that we have in >> jenkins .gr

Re: [DISCUSS] Automation for Java code formatting

Luke: Anything that helps contributors and reviewers work better together - +1! :D On Wed, Jun 27, 2018 at 9:04 AM Lukasz Cwik wrote: > If spotless is run against a PR that is already well formatted its a > non-issue as the formatting changes are usually related to the change but I > have revi

Re: [DISCUSS] Automation for Java code formatting

+1! Remove guesswork :D On Tue, Jun 26, 2018 at 9:15 PM Kenneth Knowles wrote: > Hi all, > > I like readable code, but I don't like formatting it myself. And I > _really_ don't like discussing in code review. "Spotless" [1] can enforce - > and automatically apply - automatic formatting for Jav

Re: Auto closing stale PRs label

ee the options we've set up here; there may be more: > https://github.com/apache/beam/blob/master/.github/stale.yml > > Kenn > > On Tue, Jun 26, 2018 at 6:42 PM Rafael Fernandez > wrote: > >> The new label makes sense to me, but Ismael: I want to make sure your >>

Re: Auto closing stale PRs label

The new label makes sense to me, but Ismael: I want to make sure your concern is fully addressed. I see your point about making sure we are not shutting the door on a small fix that perhaps went unatended for benign reasons. Perhaps a step before closure is feasble? something like getting a nice me

Re: [Design Proposal] Improving Beam code review

I did a quick pass on the doc and left minor comments, thanks! I have some feedback and thoughts: - For metrics and tools, there ought to be mature OSS projects out there we can learn from. I believe Kubernetes has a very healthy practice, it'd be ideal to learn from them. +Griselda Cueva

Re: bad logger import?

Perhaps we can adjust checkstyle, or some >> other more semantic analysis, to forbid it. >> >> Kenn >> >> On Tue, Jun 26, 2018 at 6:48 AM Rafael Fernandez >> wrote: >> >>> +Lukasz Cwik , +Henning Rohde >>> >>> On

Re: bad logger import?

+Lukasz Cwik , +Henning Rohde On Tue, Jun 26, 2018 at 1:25 AM Romain Manni-Bucau wrote: > Hi guys, > > answering a question on slack i realized flink > ExecutableStageDoFnOperator.java uses JUL instead of SLF4J, not sure it is > intended so thought I would mention it here. > > Side note: arche

Re: Going on leave for a bit

Have a great time, Kenn! On Tue, Jun 26, 2018 at 5:49 AM Ismaël Mejía wrote: > Enjoy your family time. > > Best wishes, > Ismael > > > On Tue, Jun 26, 2018 at 12:13 PM Pei HE wrote: > >> (A late) Congrats for the newborn! >> -- >> Pei >> >> On Tue, Jun 26, 2018 at 1:42 PM, Kenneth Knowles wrot

Re: FYI on Slack Channels

juin 2018 à 01:05, Lukasz Cwik a écrit : > >> +u...@beam.apache.org >> >> On Mon, Jun 25, 2018 at 4:04 PM Rafael Fernandez >> wrote: >> >>> Hello! >>> >>> I took the liberty to create area-specific channels (such as #beam-java, >>>

Re: scala/scio

er and has a stronger community than when > > you try to erase those distinctions in a giant monoproject. > > > > Starting with an active project that already exists and has users > > and maintainers works perfectly for getting the granularity right. > >

Re: [RESULT][VOTE] Apache Beam, version 2.5.0, release candidate #2

tween when the release branches are cut, and not when the release >>>>>> is >>>>>> > published. >>>>>> > >>>>>> > If 2.5.0 was cut on June 5, then 2.6.0 could be cut July 17? >>>>>> > >>>>>>

Re: Fixing flaky tests and infrastructure stability

On Mon, Jun 25, 2018 at 3:49 PM Kenneth Knowles wrote: > There's also a "flake" label and a saved search here: > https://issues.apache.org/jira/issues/?filter=12343195 > > Kenn > > On Mon, Jun 25, 2018 at 12:48 PM Mikhail Gryzykhin > wrote: > >> Hello everyone, >> >> I have assembled short list

FYI on Slack Channels

Hello! I took the liberty to create area-specific channels (such as #beam-java, #beam-python, #beam-go, etc.) As our project and community grows, I am seeing more and more "organic" interest groups forming -- this may help us chat more online. If they don't, we can delete later. Any thoughts? (I

Re: [RESULT][VOTE] Apache Beam, version 2.5.0, release candidate #2

Great news! Thanks so much to our Release Manager and everybody who helped iron out the wrinkles! If you haven't seen it already, look for the thread "[PROPOSAL] Add a blog post for Beam release 2.5.0 ​" [1]​ in dev@ - Alexey Romanenko has put together a very nice summary of all the good stuff in

Re: [PROPOSAL] Add a blog post for Beam release 2.5.0

+1 and no additional suggestions. Great stuff! On Fri, Jun 22, 2018, 2:08 PM Kenneth Knowles wrote: > +1 and added some too. > > On Fri, Jun 22, 2018 at 1:18 PM Ahmet Altay wrote: > >> Thank you Alexey! >> >> It is a great idea. I added my suggestions to the doc. >> >> On Fri, Jun 22, 2018 at 1

Re: scala/scio

I like the idea of per-module releases for Beam. I know Henning and others have thought about that space as well. BTW, I'm a big fan of scio, and happy to help in any way possible if they are interested in turning "de facto" into "de jure" :D In such a world, Scio could be a very good first use c

Re: [PROPOSAL] Merge samza-runner to master

prototype working sometime next quarter :). >> >> >> >> Btw, thanks everyone for the comments! >> >> >> >> Thanks, >> >> Xinyu >> >> >> >> On Mon, Jun 18, 2018 at 4:59 PM, Robert Burke >> wrote: >> >>&

Re: [DISCUSS] Releasing Beam in the presence of emergencies

t;> On Thu, Jun 14, 2018 at 10:29 PM Jean-Baptiste Onofré >>>> wrote: >>>> >>>>> Hi Rafael, >>>>> >>>>> It's a good point but I don't see nothing more to do on our side: if a >>>>> emergency issue is det

Re: [PROPOSAL] Merge samza-runner to master

I've just read this and wanted to share my excitement :D On Mon, Jun 18, 2018 at 3:10 PM Kenneth Knowles wrote: > One thing that will be necessary is porting the build to Gradle. > > Kenn > > On Mon, Jun 18, 2018 at 11:57 AM Xinyu Liu wrote: > >> Hi, Folks, >> >> On behalf of the Samza team,

[DISCUSS] Releasing Beam in the presence of emergencies

Hi Beam devs, Emergencies can and will happen. As Apache Beam adoption continues to grow, the user community will naturally expect the Beam developer community to react to critical issues, such as security vulnerabilities in our dependencies. I want to make sure the dev community is in agreement t

Re: [VOTE] Go SDK

+1 ! On Tue, May 22, 2018 at 7:54 AM Lukasz Cwik wrote: > +1 (binding) > > On Tue, May 22, 2018 at 6:16 AM Robert Burke wrote: > >> +1 (non-binding) >> >> I'm looking forward to helping gophers solve their big data problems in >> their language of choice, and runner of choice! >> >> Next stop,

Re: [ANNOUCEMENT] New Foundation members!

Congratulations!!! On Fri, Mar 30, 2018 at 8:29 PM Aviem Zur wrote: > Congrats! > > On Sat, Mar 31, 2018 at 2:30 AM Ahmet Altay wrote: > >> Congratulations to all of you! >> >> >> On Fri, Mar 30, 2018, 4:29 PM Pablo Estrada wrote: >> >>> Congratulations y'all! Very cool. >>> Best >>> -P. >>> >

Re: Proposed improvements to our documentation

018 at 10:53 AM Lukasz Cwik wrote: >>> >>>> +1 >>>> >>>> On Wed, Feb 28, 2018 at 10:46 AM, Kenneth Knowles >>>> wrote: >>>> >>>>> Yes! I love the idea of having a good cross-language transform >>>>> referenc

Proposed improvements to our documentation

Hi folks, I think we've all seen a few areas of improvement here and there in our docs. For example, one can find a a Javadoc entry with outdated content here and there [1], or "sample" code snippets that have problems, such as not compiling [2]. I think a good thing to do is to invest in extendi

Re: Beam 2.4.0

+1 on having release trains scheduled. Romain: Do you have a list of PRs that could benefit from increased focus if they want to make it on the upcoming train? On Tue, Feb 20, 2018 at 3:30 PM Ahmet Altay wrote: > +1 for having regular release cycles. Finalizing a release takes time in > the or

Re: rename: BeamRecord -> Row

Very strong +1 On Fri, Feb 2, 2018 at 1:24 PM Reuven Lax wrote: > We're looking at renaming the BeamRecord class > , that was used for columnar > data. There was sufficient discussion on the naming, that I want to make > sure the dev list is aware of na

Re: [DISCUSS] Towards Beam 2.3.0

+1! I like the predictability a schedule would bring. And I think it helps feature users to budget their time a little better -- there's always the next scheduled train, so no need to stress out to ship in the current one. On Mon, Jan 8, 2018 at 10:37 AM Kenneth Knowles wrote: > +1 to not hol