Re: Can we solve WindowFn.getOutputTime another way?

On Wed, Feb 10, 2021 at 2:24 PM Alex Amato wrote: > > > On Wed, Feb 10, 2021 at 12:14 PM Kenneth Knowles wrote: > >> On a PR (https://github.com/apache/beam/pull/13927) we got into a >> discussion of a very old and strange feature of Beam that I think we should >> revisit. >> >> The WindowFn

Re: [Proposal]: Add concepts guide and glossary to docs

This is a great idea. I would encourage others to look at the list of glossary items and see if anything interesting is missing from the list. On Wed, Feb 10, 2021 at 3:15 PM David Huntsperger wrote: > Feedback welcome: > >- [Proposal] Beam concepts & glossary > >

Re: Builds Meeting this Thursday

I suspect that Beam would suffer from the same problems that Airflow did, specifically: "1) we won't be stuck in a queue behind other ASF projects waiting for our "slot"". This is similar to why Beam migrated to using a dedicated pool of Jenkins agents. I'm not following the performance of our GH

[Proposal]: Add concepts guide and glossary to docs

Feedback welcome: - [Proposal] Beam concepts & glossary Thanks!

Re: Can we solve WindowFn.getOutputTime another way?

On Wed, Feb 10, 2021 at 12:14 PM Kenneth Knowles wrote: > On a PR (https://github.com/apache/beam/pull/13927) we got into a > discussion of a very old and strange feature of Beam that I think we should > revisit. > > The WindowFn has the ability to shift timestamps forward in order to > unblock

Re: Builds Meeting this Thursday

Nice, thank you for sharing Aizhamal. Change looks relatively straightforward. What is the urgency of this for Beam? Is this already impacting Beam's gh actions? On Tue, Feb 9, 2021 at 7:15 PM Aizhamal Nurmamat kyzy wrote: > Hi all, > In case you may find this interesting / valuable: Airflow

Do we need synchronized processing time? / What to do about "continuation triggers"?

Hi all, TL;DR: 1. should we replace "after synchronized processing time" with "after count 1"? 2. should we remove "continuation trigger" and leave this to runners? "AfterSynchronizedProcessingTime" triggers were invented to solve a specific problem. They are inconsistent across SDKs

Re: [Vote] Publishing new website designs

On Wed, Feb 10, 2021 at 11:56 AM Robert Bradshaw wrote: > Let's revert these content changes. > > On Wed, Feb 10, 2021 at 11:54 AM Kenneth Knowles wrote: > >> On the "become a committer" page, the link to "our best practices" is a >> 404 to the old site. >> >> Under the header "Knows, upholds,

Can we solve WindowFn.getOutputTime another way?

On a PR (https://github.com/apache/beam/pull/13927) we got into a discussion of a very old and strange feature of Beam that I think we should revisit. The WindowFn has the ability to shift timestamps forward in order to unblock downstream watermarks. Why? Specifically in this situation: -

Re: [Vote] Publishing new website designs

Let's revert these content changes. On Wed, Feb 10, 2021 at 11:54 AM Kenneth Knowles wrote: > On the "become a committer" page, the link to "our best practices" is a > 404 to the old site. > > Under the header "Knows, upholds, and reinforces the responsibilities of > an Apache Software

Re: [Vote] Publishing new website designs

On the "become a committer" page, the link to "our best practices" is a 404 to the old site. Under the header "Knows, upholds, and reinforces the responsibilities of an Apache Software Foundation committer" I am -1 to removing the details. The bullets we chose are not included in the linked ASF

Re: [Vote] Publishing new website designs

+1 based on some basic browsing around. It would have been nice if the old and new site could have been generated from the same content, to keep the old one around for a bit (if needed, in that case we might have more confidence nothing was lost), but probably not worth making and staging a copy.

Re: Error when trying to read from S3

+dev@beam.apache.org Digging more into the code it looks more like a bug that was introduced in a recent PR (https://github.com/apache/beam/pull/13180). It seems that pipeline_options is absent when using external workers and it causes the S3IO to fail in an assertion in its constructor. I

Re: Migrate S3FileSystem

Hello, Thanks for the contribution Raphael. I have been a bit busy this week, but I will take a look as soon as I can probably end of the week/beginning of next. Sorry. Best, Ismaël On Wed, Feb 10, 2021 at 10:39 AM Raphael Sanamyan < raphael.sanam...@akvelon.com> wrote: > Hello Ismaël, > > >

Re: Migrate S3FileSystem

Hello Ismaël, I have finished the task "Migrate S3FileSystem to AWS SDK for Java 2" and made the PR https://github.com/apache/beam/pull/13914. Could you please review this PR or suggest somebody who could do it? Thank you, Raphael. От: Raphael Sanamyan