Re: Percentile metrics in Beam

2021-10-01 Thread Ajo Thomas
Thanks, encoding sketch data using a new URN makes sense. - Ajo On Fri, Oct 1, 2021 at 11:22 AM Luke Cwik wrote: > Yes you could encode the sketch information but would need to use new URNs > because the encoding for the existing ones are already fixed. The point of > adding new URNs is to

Re: [VOTE] Release 2.33.0, release candidate 1

2021-10-01 Thread Judah Rand
Might happy with development. Dataflow will pose problems, however. On Fri, 1 Oct 2021, 20:51 Chamikara Jayalath, wrote: > > > On Fri, Oct 1, 2021 at 11:49 AM Judah Rand wrote: > >> That's very disappointing as I'd worked very hard over the last few days >> to get the dependency updated in

DebeziumIO dependency update

2021-10-01 Thread Judah Rand
Hi all, I identified the other day that Debezium does not work with many cloud providers Postgres instances. This includes Azure and Google CloudSQL. I determined that this is due to the version of Postgres JDBC that Debezium uses (42.2.14). I've been working with the Debezium devs to update

Flaky test issue report (33)

2021-10-01 Thread Beam Jira Bot
This is your daily summary of Beam's current flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20labels%20%3D%20flake) These are P1 issues because they have a major negative impact on the community and make it hard to

Re: [VOTE] Release 2.33.0, release candidate 1

2021-10-01 Thread Judah Rand
That's very disappointing as I'd worked very hard over the last few days to get the dependency updated in Debezium and 6 weeks is quite a significant and problematic delay for me. On Fri, 1 Oct 2021, 19:44 Udi Meiri, wrote: > RC2 progress: I've merged 3 cherrypicks, and then reverted 1 (it

Re: [VOTE] Release 2.33.0, release candidate 1

2021-10-01 Thread Judah Rand
I need to use it via the Python `ReadFromDebezium` interface which makes that more tricky as it just goes and fetches the prebuilt jar. On Fri, 1 Oct 2021, 19:26 Luke Cwik, wrote: > Based upon the PR, it doesn't look like there were any code changes. Would > overriding the debezium version in

P1 issues report (44)

2021-10-01 Thread Beam Jira Bot
This is your daily summary of Beam's current P1 issues, not including flaky tests (https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20statusCategory%20!%3D%20Done%20AND%20priority%20%3D%20P1%20AND%20(labels%20is%20EMPTY%20OR%20labels%20!%3D%20flake). See

Re: Percentile metrics in Beam

2021-10-01 Thread Ajo Thomas
Thanks for the pointers, Luke and sorry for replying late on this thread. Distribution metric's - *void update(long sum, long count, long min, long max)* certainly seems like dead code and should be okay to remove. I can reach out to the original author to confirm. As for the approach for

Re: [VOTE] Release 2.33.0, release candidate 1

2021-10-01 Thread Judah Rand
I was wondering if there might be any possibility of cherry-picking the changes from https://github.com/apache/beam/pull/15636 into the 2.33.0 release for next week? Currently DebeziumIO is completely unusable for both Azure and Google CloudSQL Postgres instances due to a bug in the Postgres

Re: Contributor permission for Beam Jira tickets

2021-10-01 Thread Alexey Romanenko
Hi David! Done. Welcome to Beam! — Alexey > On 29 Sep 2021, at 00:25, David Prieto Rivera > wrote: > > Hi, > > My name is David Prieto. I work at Europcar Mobility Group as a Data > Engineer. We are currently using Beam for both batch and streaming pipelines > and I would like to be

DebeziumIO dependency update

2021-10-01 Thread Judah Rand
Hi all, I identified the other day that Debezium does not work with many cloud providers Postgres instances. This includes Azure and Google CloudSQL. I determined that this is due to the version of Postgres JDBC that Debezium uses (42.2.14). I've been working with the Debezium devs to update