On Mon, Jul 15, 2019 at 5:42 AM Chamikara Jayalath wrote:
>
> On Sat, Jul 13, 2019 at 7:41 PM Chad Dombrova wrote:
>>
>> Hi Chamikara,
why not make this part of the pipeline options? does it really need to
vary from transform to transform?
>>>
>>> It's possible for the same pipel
On Sat, Jul 13, 2019 at 7:41 PM Chad Dombrova wrote:
> Hi Chamikara,
>
>
>>>- why not make this part of the pipeline options? does it really
>>> need to vary from transform to transform?
>>>
>>> It's possible for the same pipeline to connect to multiple expansion
>> services, to use tr
Hi Chamikara,
>>- why not make this part of the pipeline options? does it really
>> need to vary from transform to transform?
>>
>> It's possible for the same pipeline to connect to multiple expansion
> services, to use transforms from more than one SDK language and/or version.
>
There
On Sat, Jul 13, 2019 at 10:35 AM Chad Dombrova wrote:
>
> Cross-language support for PubSub is not yet implemented but it can be
>> done similarly to ReadFromKafka. There are still some limitations regarding
>> the coders, i.e. only coders can be used which are available in both the
>> Java and t
> Cross-language support for PubSub is not yet implemented but it can be
> done similarly to ReadFromKafka. There are still some limitations regarding
> the coders, i.e. only coders can be used which are available in both the
> Java and the Python SDK (standard coders).
>
Yeah, I was just looking
Cross-language support for PubSub is not yet implemented but it can be done
similarly to ReadFromKafka. There are still some limitations regarding the
coders, i.e. only coders can be used which are available in both the Java and
the Python SDK (standard coders).
As of now the user experience is
Thanks for the response, Max. I saw that KafkaIO is now supported in
python via an external transform (
https://jira.apache.org/jira/browse/BEAM-7029), but I thought I read
somewhere that it was only supported in batch mode (though I don't see that
mentioned in the ticket or the PR, so not sure wh
Hi Chad,
This stub will only be replaced by the Dataflow service. It's an artifact of
the pre-portability era.
That said, we now have the option to replace ReadFromPubSub with an external
transform which would utilize Java's PubSubIO via the new cross-language
feature.
Thanks,
Max
On 12.07.1
Hi all,
This error came as a bit of a surprise.
Here’s a snippet of the traceback (full traceback below).
File "apache_beam/runners/common.py", line 751, in
apache_beam.runners.common.DoFnRunner.process
return self.do_fn_invoker.invoke_process(windowed_value)
File "apache_beam/runners/com
