Re: [RESULT][VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

Beam Firefly is fine with me (I guess people tend to forget mascot names anyway). But if anyone comes up with something particularly cute/clever we can consider it. On Mon, Apr 13, 2020 at 6:33 PM Aizhamal Nurmamat kyzy wrote: > @Alex, Beam Firefly? > > On Thu, Apr 9, 2020 at 10:57 PM Alex Van

Re: Implementing type hints on multi-output PTransforms

On Tue, Mar 31, 2020 at 10:16 AM Joshua B. Harrison wrote: > Ok - that makes sense. My specific workaround was to remove the > with_output_types for now, so advising the user on this in the error > message would be nice. I was just worried about silently passing. > > As for the formalization: >

Re: [RESULT][VOTE] Accept the Firefly design donation as Beam Mascot - Deadline Mon April 6

@Alex, Beam Firefly? On Thu, Apr 9, 2020 at 10:57 PM Alex Van Boxel wrote: > We forgot something > > ... > > ... > > it/she/he needs a *name*! > > > _/ > _/ Alex Van Boxel > > > On Fri, Apr 10, 2020 at 6:19 AM Kenneth Knowles wrote: > >> Looking forward to the guide. I enjoy doing (bad)

Re: Usage metrics for Beam

@Pablo, https://blog.sonatype.com/2010/12/now-available-central-download-statistics-for-oss-projects/ suggests for Apache projects, if you have "deployer" permissions then you can access -- I gather that you have sufficient permissions, and that Kenn was able to follow the steps within. Though,

Re: [DISCUSS] Let's establish a guideline for using Python type annotations in Beam codebase

I agree with Robert to only put Any where it really can be any type. I'm not sure how much typing we should add. At minimum: external APIs and wherever mypy complains. Ideally I would like to have annotations everywhere, because this reduces uncertainty when modifying existing code. You are

Re: Usage metrics for Beam

I'm also curious about this, but I don't know where to look for the maven numbers. Where exactly are they? Do I need to get access to the repository? Thanks -P. On Thu, Apr 9, 2020, 9:27 PM Robert Bradshaw wrote: > Yes, it's hard to know what can conclusively be drawn from the raw totals. > I

Re: [DISCUSS] Let's establish a guideline for using Python type annotations in Beam codebase

On Mon, Apr 13, 2020 at 11:48 AM Valentyn Tymofieiev wrote: > > On Mon, Apr 13, 2020 at 10:53 AM Robert Bradshaw > wrote: > >> On Mon, Apr 13, 2020 at 10:38 AM Valentyn Tymofieiev >> wrote: >> >>> To clarify, I don't suggest that every variable should have a defined >>> type that doesn't

Re: [Proposal] Requesting PMC approval to start planning for Beam Summits 2020

That sounds good to me. On Mon, Apr 13, 2020 at 11:33 AM Maximilian Michels wrote: > I suggest to move forward with a vote on the dev mailing list to > formally approve the proposal and copy ASF legal for the use of the Beam > trademark. > > Does that sound good? > > @Kenneth We are trying not

Re: GCP Integration Tests Authentication

The Jenkins machines run on the GCP VMs and will pick up the VMs credentials by default. This likely means that you have to grant access to the GCE service account. On Mon, Apr 13, 2020 at 10:55 AM Jacob Ferriero wrote: > Hi Dev List, > > I'm wondering how GCP integration tests are

Re: [DISCUSS] Let's establish a guideline for using Python type annotations in Beam codebase

On Mon, Apr 13, 2020 at 10:53 AM Robert Bradshaw wrote: > On Mon, Apr 13, 2020 at 10:38 AM Valentyn Tymofieiev > wrote: > >> To clarify, I don't suggest that every variable should have a defined >> type that doesn't change. However, I'd like to establish a culture where we >> consistently add

Re: [Proposal] Requesting PMC approval to start planning for Beam Summits 2020

I suggest to move forward with a vote on the dev mailing list to formally approve the proposal and copy ASF legal for the use of the Beam trademark. Does that sound good? @Kenneth We are trying not to conflict with any ASF events. Also, we'll have half-day sessions which gives us additional

GCP Integration Tests Authentication

Hi Dev List, I'm wondering how GCP integration tests are authenticated for in the apache-beam-testing project. In this Java Post Commit [1] for this PR [2] I have a test failing that succeeds when I run with my own credentials. I have a hunch that this is failing because all writes are failing

Re: [DISCUSS] Let's establish a guideline for using Python type annotations in Beam codebase

On Mon, Apr 13, 2020 at 10:38 AM Valentyn Tymofieiev wrote: > To clarify, I don't suggest that every variable should have a defined type > that doesn't change. However, I'd like to establish a culture where we > consistently add type annotations when we write new code. Where type is > defined

Re: [DISCUSS] Let's establish a guideline for using Python type annotations in Beam codebase

To clarify, I don't suggest that every variable should have a defined type that doesn't change. However, I'd like to establish a culture where we consistently add type annotations when we write new code. Where type is defined gradually, we can use flexible annotations: "# type: (Any) -> Any" or

Re: Jenkins jobs not running for my PR 10438

Done. On Mon, Apr 13, 2020 at 8:52 AM Shoaib Zafar wrote: > Hello Beam Committers. > > Would you please trigger the pre-commit checks on the PR: > https://github.com/apache/beam/pull/11210 along with the python > post-commit checks (Run Python PostCommit, Run Python 3.5 PostCommit)? > > Thanks!

Re: Portable timer loops

On Mon, Apr 13, 2020 at 8:53 AM Luke Cwik wrote: > In non portable implementations you would have to wait till the element/timer > was finished processing before you could process any newly created timers. > Timers that are created with the same key+window+timer family overwrite > existing

Re: Portable timer loops

This is probably related to issue I was having with Direct runner and timer ordering. The problem is that there might be multiple timers (for given key) inside bundle and that each timer might set another timer. To ensure timer ordering, timers must be fired one at a time and when fired timer

Re: BEAM-8751: Code Review Wanted for PR 11208

Thanks for the remainder. I took a look and merged it. On Mon, Apr 13, 2020 at 7:51 AM Tomo Suzuki wrote: > Gentle reminder. Can somebody approve and merge this dependency upgrade > "google-api-client 1.30.9" >

Re: Portable timer loops

I'm not sure I understand the difference - do any "classic" runners add new timers to the bundle? I know that at least the Dataflow runner would end up the new timer in a new bundle. One thing we do need to ensure is that modifications to timers are reflected in a bundle. So if a bundle contains

Re: Portable timer loops

In non portable implementations you would have to wait till the element/timer was finished processing before you could process any newly created timers. Timers that are created with the same key+window+timer family overwrite existing timers that have been set which can lead to a timer being

Re: Jenkins jobs not running for my PR 10438

Hello Beam Committers. Would you please trigger the pre-commit checks on the PR: https://github.com/apache/beam/pull/11210 along with the python post-commit checks (Run Python PostCommit, Run Python 3.5 PostCommit)? Thanks! *Shoaib Zafar* Software Engineering Lead Mobile: +92 333 274 6242

Portable timer loops

Hi, In the "classic" Java mode one can set timers which in turn can set timers which enables to create a timer loop, e.g.: @ProcessElement public void processElement( ProcessContext context, @TimerId("timer") Timer timer) { // Initial timer timer.withOutputTimestamp(new

Re: BEAM-8751: Code Review Wanted for PR 11208

Gentle reminder. Can somebody approve and merge this dependency upgrade "google-api-client 1.30.9" https://github.com/apache/beam/pull/11208? On Fri, Apr 3, 2020 at 11:38 AM Luke Cwik wrote: > Still too busy to help with this. Does anyone else have

Beam Dependency Check Report (2020-04-13)

High Priority Dependency Updates Of Beam Python SDK: Dependency Name Current Version Latest Version Release Date Of the Current Used Version Release Date Of The Latest Release JIRA Issue google-cloud-datastore 1.7.4 1.12.0

Re: Jenkins jobs not running for my PR 10438

done On Mon, Apr 13, 2020 at 12:42 PM Rehman Murad Ali wrote: > > Hi Beam Committers! > > Thanks( Ismael ) > > I appreciate if someone could trigger these tests on this PR > https://github.com/apache/beam/pull/11154 > > run dataflow validatesrunner > run flink validatesrunner > Run Java Flink

Re: Jenkins jobs not running for my PR 10438

Hi Beam Committers! Thanks( Ismael ) I appreciate if someone could trigger these tests on this PR https://github.com/apache/beam/pull/11154 run dataflow validatesrunner run flink validatesrunner Run Java Flink PortableValidatesRunner Streaming Thanks *Rehman Murad Ali* Software Engineer