RE: Join a meeting to help coordinate implementing a Dask Runner for Beam

2022-07-08 Thread Alex Merose via dev
Dear Beam & Dask communities, Together with Pablo and Charles, I've hacked together an initial prototype of a Dask runner for Beam. I'm happy to announce that I have minimum viable working version in a fork here: https://github.com/alxmrs/beam/pull/1 There's definitely more work to do here –

ClassNotFoundException when using Java external transforms in a Java job

2022-07-08 Thread Sahith Nallapareddy via dev
Hello, I sent an email sometime ago about java external transforms in a java job. Seems like things are for the most part working, but we have run into this error that I have attached. At the base cause it says [debug] Caused by: java.lang.ClassNotFoundException:

DRAFT - Apache Beam Board Report - July 2022

2022-07-08 Thread Kenneth Knowles
Hi all, The next Beam board report is due July 13. Please help me to write it here: https://s.apache.org/beam-draft-report-2022-07 Ideas: - highlights from CHANGES.md - interesting technical discussions - integrations with other projects - community events - major user facing

Re: [Dataflow][Java] Guidance on Transform Mapping Streaming Update

2022-07-08 Thread Evan Galpin
Ya fair enough, makes sense. I’ll reach out to GCP. Thanks Luke! - Evan On Fri, Jul 8, 2022 at 11:24 Luke Cwik wrote: > I was suggesting GCP support mainly because I don't think you want to > share the 2.36 and 2.40 version of your job file publicly as someone > familiar with the layout and

Re: [Dataflow][Java] Guidance on Transform Mapping Streaming Update

2022-07-08 Thread Luke Cwik via dev
I was suggesting GCP support mainly because I don't think you want to share the 2.36 and 2.40 version of your job file publicly as someone familiar with the layout and format may spot a meaningful difference. Also, if it turns out that there is no meaningful difference between the two then the

Re: [Dataflow][Java] Guidance on Transform Mapping Streaming Update

2022-07-08 Thread Evan Galpin
Thanks for your response Luke :-) Updating in 2.36.0 works as expected, but as you alluded to I'm attempting to update to the latest SDK; in this case there are no code changes in the user code, only the SDK version. Is GCP support the only tool when it comes to deciphering the steps added by

Beam High Priority Issue Report

2022-07-08 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/22192 [Bug]: