Re: Create a Dataset in GCP Testing Project?

Following up here, it seems I have lost my access to the (GCP-based) Testing project -- had not addressed/finished this ticket for some time, as had been working on other things for a bit after being distracted. Can someone else please re-add me? [ apologies if I can't figure out access and one

Contributor permission for Beam Jira tickets

Hi Apache Beam Dev Team, I'm Victor from Google and I am working with Ning on the Python Interactive Runner. My ASF Jira ID is victorhc. Could I please have permissions to create and assign tickets for my work? Thank you, Victor

Re: [RFC][design/idea] CDAP plugins support in Apache Beam

Very cool. Thanks for sharing! On Tue, Jan 18, 2022 at 11:42 AM Ilya Kozyrev wrote: > TL:DR: We want to develop support for Apache CDAP batch and streaming > plugins to enrich Apache Beam connectors to external applications. Please > review the design[1] to help us bring CDAP plugins

Re: Default output timestamp of processing-time timers

Yea, it makes sense. This is an issue for the global window where there isn't automatic cleanup of state. I've had a few user cases where they would like a good way of doing state cleanup in the global window too - something where whenever state gets buffer there is always a finite timer that will

[RFC][design/idea] CDAP plugins support in Apache Beam

TL:DR: We want to develop support for Apache CDAP batch and streaming plugins to enrich Apache Beam connectors to external applications. Please review the design[1] to help us bring CDAP plugins integrations into Apache Beam. Hi all, I along with a few community members thought of an idea to

Re: Default output timestamp of processing-time timers

This is an interesting case, and a legitimate counterexample to consider. I'd call it a workaround :-). The semantic thing they would want/need is "output timestamp" associated with buffered data (also implemented with watermark hold). I do know systems that designed their state with this built

Flaky test issue report (45)

This is your daily summary of Beam's current flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20labels%20%3D%20flake) These are P1 issues because they have a major negative impact on the community and make it hard to

P1 issues report (66)

This is your daily summary of Beam's current P1 issues, not including flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20priority%20%3D%20P1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20!%3D%20flake). See

Re: Default output timestamp of processing-time timers

On Tue, Dec 14, 2021 at 2:38 PM Steve Niemitz wrote: > > I think this wouldn't be very robust to different situations where > processing time and event time may not be that close to each other. > > if you do something like `min(endOfWindow, max(eventInputTimestamp, > computedFiringTimestamp))`

Re: Beam Java starter project template

I want to clarify one thing: I am not certain the requirement of ASL2 applies to example code snippets. I am also not sure if it makes a material difference to users. I _am_ sure we would need to deal with some process to use something other than ASL2, so I'd rather not. Kenn On Tue, Jan 18,

Re: add developer

Hi Andrei, I've added you to the "Contributors" role on Jira, so you can be assigned tickets. Is this what you mean? Kenn On Tue, Jan 18, 2022 at 6:15 AM Andrei Kustov wrote: > Hi community, sorry if I confuse somebody in my previous mail. > Could someone please add me to the Apache Jira as a

Re: Beam Java starter project template

Agree with Luke here. "Just git clone and go" is a big part of it. But also the answer to "I simply don't know what one would put in a Python repo than, other than a bare setup.py that lists a dependency on apache_beam" is answered by David's initial email and his repo, namely: - GitHub Actions

Re: add developer

Hi community, sorry if I confuse somebody in my previous mail. Could someone please add me to the Apache Jira as a developer? This is my Jira ID: andreykus От: Andrei Kustov Отправлено: 17 января 2022 г. 10:15:32 Кому: dev@beam.apache.org Тема: add developer

Re: [DISCUSS] propdeps removal and what to do going forward

On Fri, Jan 14, 2022 at 9:34 AM Daniel Collins wrote: > > In particular the Hadoop/Spark and Kafka dependencies must be > **provided** as they were. I am not sure of others but those three matter. > > I think there's a bit of a difference here between what should be the > state in the short term

Re: [DISCUSS] Migrate Jira to GitHub Issues?

I also think that we are at the point where a document describing them side-by-side is needed. I would very much like to help. I strongly support moving to GitHub Issues. I'm less concerned about pros/cons (I think the one big pro of "everyone knows it and already has an account" outweighs almost

add developer

Good day. I want to participate in the development of Apache Beam. Add me as a developer Best regards, Kustov Andrey (andrei.kus...@akvelon.com) Akvelon Inc.