Note also that once you get the Bytes in Python, you can use whatever
coder (or Map) to decode them that you want.

On Thu, Jul 16, 2020 at 9:21 AM Boyuan Zhang <boyu...@google.com> wrote:
>
> Hi Piotr,
>
> X-Lang uses TypedWithoutMetadata, which outputs the KV directly instead of 
> KafkaRecord: see here. Given the limit that x-lang can only work with 
> well-known coders, if you want to process the KV in python output from 
> KafkaIO, the coders of key and value should be well-known in beam. By 
> default, the key and value are bytes: see here.
>
> On Thu, Jul 16, 2020 at 8:48 AM Piotr Szuberski <piotr.szuber...@polidea.com> 
> wrote:
>>
>> I'm writing a python wrappers for KinesisIO and I encountered a problem that 
>> Read transform creates a PCollection with KinesisRecord class which's coder 
>> by default is assigned as 'beam:coders:javasdk:0.1'. I managed to register 
>> this coder using CoderTranslatorRegistrar which adds the coder to the 
>> KNOWN_CODER_URNS and therefore is sent with my custom urn.
>>
>> Kafka's cross language Write transform uses KV<>, is  encoded by default in 
>> beam.
>>
>> But I can't see how KafkaRecordCoder is translated in cross-language usage 
>> to python? I can't see any place in code where it gets registered.
>>
>> I just don't get how KafkaIO.Read works in cross-language. Could someone 
>> clarify me how does it work?
>>
>> Thanks in advance!

Reply via email to