Re: protobuf data as input to spark streaming
Hello Stelios, friendly reminder if you could share any sample code/repo Are you using a schema registry? Thanks Kiran On Fri, Apr 8, 2022 at 4:37 PM Kiran Biswal wrote: > Hello Stelios > > Just a gentle follow up if you can share any sample code/repo > > Regards > Kiran > > On Wed, Apr 6, 2022 at 3:19 PM Kiran Biswal wrote: > >> Hello Stelios >> >> Preferred language would have been Scala or pyspark but if Java is proven >> I am open to using it >> >> Any sample reference or example code link? >> >> How are you handling the peotobuf to spark dataframe conversion >> (serialization federalization)? >> >> Thanks >> Kiran >> >> On Wed, Apr 6, 2022, 2:38 PM Stelios Philippou >> wrote: >> >>> Yes we are currently using it as such. >>> Code is in java. Will that work? >>> >>> On Wed, 6 Apr 2022 at 00:51, Kiran Biswal wrote: >>> Hello Experts Has anyone used protobuf (proto3) encoded data (from kafka) as input source and been able to do spark structured streaming? I would appreciate if you can share any sample code/example Regards Kiran >
Re: protobuf data as input to spark streaming
Hello Stelios Just a gentle follow up if you can share any sample code/repo Regards Kiran On Wed, Apr 6, 2022 at 3:19 PM Kiran Biswal wrote: > Hello Stelios > > Preferred language would have been Scala or pyspark but if Java is proven > I am open to using it > > Any sample reference or example code link? > > How are you handling the peotobuf to spark dataframe conversion > (serialization federalization)? > > Thanks > Kiran > > On Wed, Apr 6, 2022, 2:38 PM Stelios Philippou wrote: > >> Yes we are currently using it as such. >> Code is in java. Will that work? >> >> On Wed, 6 Apr 2022 at 00:51, Kiran Biswal wrote: >> >>> Hello Experts >>> >>> Has anyone used protobuf (proto3) encoded data (from kafka) as input >>> source and been able to do spark structured streaming? >>> >>> I would appreciate if you can share any sample code/example >>> >>> Regards >>> Kiran >>>
Re: protobuf data as input to spark streaming
Hello Stelios Preferred language would have been Scala or pyspark but if Java is proven I am open to using it Any sample reference or example code link? How are you handling the peotobuf to spark dataframe conversion (serialization federalization)? Thanks Kiran On Wed, Apr 6, 2022, 2:38 PM Stelios Philippou wrote: > Yes we are currently using it as such. > Code is in java. Will that work? > > On Wed, 6 Apr 2022 at 00:51, Kiran Biswal wrote: > >> Hello Experts >> >> Has anyone used protobuf (proto3) encoded data (from kafka) as input >> source and been able to do spark structured streaming? >> >> I would appreciate if you can share any sample code/example >> >> Regards >> Kiran >> >>>
Re: protobuf data as input to spark streaming
Yes we are currently using it as such. Code is in java. Will that work? On Wed, 6 Apr 2022 at 00:51, Kiran Biswal wrote: > Hello Experts > > Has anyone used protobuf (proto3) encoded data (from kafka) as input > source and been able to do spark structured streaming? > > I would appreciate if you can share any sample code/example > > Regards > Kiran > >>
