Apache beam python pipeline for read from pubsub

2023-07-18 Thread Pothula Yamini
I wrote an apache beam pipeline in python to read messages from pubsub subscription. The data rate at which the messages published from that pubsub topic which is in us-east4 region is 10,000 tuples/sec. The pipeline looks likes this: | 'read from pubsub' >> beam.io.textio.ReadFromText() | '

Re: [Feature Proposal] Add ARM Support to Beam SDK Container Images

2023-07-18 Thread Valentyn Tymofieiev via dev
Hi Celeste, Thanks for the proposal and researching the options. Using multi-arch images seems like a good way to reduce the complexity associated with correctly selecting the architecture on the runner. It sounds like there may be implications for release process, which future release managers

[Feature Proposal] Add ARM Support to Beam SDK Container Images

2023-07-18 Thread Celeste Zeng
Hi everyone, My name is Celeste. I work for the GCP Dataflow team and I am trying to add ARM support to Beam SDK container images. The ultimate goal is to make the released Beam SDK container images become multi-arch images, which support both x86 and ARM. I compiled the following doc to include

Re: [VOTE] Release 2.49.0, release candidate #2

2023-07-18 Thread Chamikara Jayalath via dev
Thanks Yi. PMC finalization should be done. - Cham On Tue, Jul 18, 2023 at 6:51 AM Yi Hu via dev wrote: > Sorry for bothering, > > It appears the email delivery issue still exists. In case the last emails > are not received, check the complete thread here: >

Re: [VOTE] Release 2.49.0, release candidate #2

2023-07-18 Thread Yi Hu via dev
Sorry for bothering, It appears the email delivery issue still exists. In case the last emails are not received, check the complete thread here: https://lists.apache.org/thread/r7r5q5mq7rqjrfbf8nj90smrdkss0sbf Still waiting for PMC finalization, mainly deploy the source release from staging

Beam Metrics Report (2023-07-18)

2023-07-18 Thread Apache Jenkins Server
ERROR: File 'src/.test-infra/jenkins/metrics_report/beam-metrics_report.html' does not exist

Beam High Priority Issue Report (38)

2023-07-18 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/27486 [Bug]: Read from