Re: WriteToBigquery on DataflowRunner with --experiment=beam_fn_api: B cannot be cast to org.apache.beam.sdk.values.KV

Hi Chamikara, I've tried both with and without "--experiment=use_beam_bq_sink" and it doesn't seem to affect the outcome. Regarding "--experiment=beam_fn_api", we're using the portability framework [1] so that we can ship our python runtime environment as an image via "--worker_harness_container_

Re: WriteToBigquery on DataflowRunner with --experiment=beam_fn_api: B cannot be cast to org.apache.beam.sdk.values.KV

On Mon, Jan 13, 2020 at 12:45 PM Bo Shi wrote: > Python 3.7.5 > Beam 2.17 > > I've used both WriteToBigquery and BigQueryBatchFileLoads successfully > using DirectRunner. I've boiled the issue down to a small > reproducible case (attached). The following works great: > > $ pipenv run python rep

Re: Question about triggering

Here is the Count.combineFn: https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Count.java#L133 Note that the type T is completely abstract and the function never touches the input element. Are you able to share the code of the CombineFn that yo

Re: NYC ? (or more generally East Coast)

I can do talks in either DC or NYC meetups. I can coordinate with CapitalOne to see if they would be willing to host the DC meetup. On Mon, Jan 13, 2020 at 4:02 PM Austin Bennett wrote: > Hi Devs and Users, > > We are looking for speakers for future Meetups and Events. Who is > building cool t

NYC ? (or more generally East Coast)

Hi Devs and Users, We are looking for speakers for future Meetups and Events. Who is building cool things with Beam? We are looking at hosting a Meetup at Spotify in February, and ideally keep some meetups going throughout the year. For this to occur, we need to hear about what people are worki

WriteToBigquery on DataflowRunner with --experiment=beam_fn_api: B cannot be cast to org.apache.beam.sdk.values.KV

Python 3.7.5 Beam 2.17 I've used both WriteToBigquery and BigQueryBatchFileLoads successfully using DirectRunner. I've boiled the issue down to a small reproducible case (attached). The following works great: $ pipenv run python repro-direct.py \ --project=CHANGEME \ --runner=DirectRunner \

Re: Beam Summit North America 2019 - recordings

Thanks Matthias! On Sun, Jan 12, 2020 at 7:51 AM Matthias Baetens wrote: > Hi everyone, > > It's our pleasure to share the recordings from the Beam Summit North > America 2019. > Please find them in the YouTube playlist >