Thanks. +1 for using RowCoder. We should try to use standard coders [1] in the x-lang SDK boundaries. If we use other coders (for example, ProtoCoder) it may or may not work depending on how various runners implement support for x-lang.
This might require slightly updating existing transforms or adding additional conversion transforms to cross-language builders. [1] https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/beam_runner_api.proto#L669 Thanks, Cham On Mon, Jun 15, 2020 at 10:00 AM Piotr Szuberski < [email protected]> wrote: > Right now I'm working on JdbcIO and I'm using Row and Schema protobuffs. > I'm figuring out how to use them properly. Thanks for the article - for > sure it will be helpful! > > On 2020/06/12 20:32:16, Brian Hulette <[email protected]> wrote: > > Thanks! I see there are jiras for SpannerIO and JdbcIO as part of that. > Are > > you planning on using row coder for them? > > If so I want to make sure you're aware of > > https://s.apache.org/beam-schema-io (sent to the dev list last week > > [1]). +Scott > > Lukas <[email protected]> will be working on building out the ideas > there > > this summer. His work could be useful for making these IOs cross-language > > (and you would get a mapping to SQL out of it without much more effort). > > > > Brian > > > > [1] > > > https://lists.apache.org/thread.html/rc1695025d41c5dc38cdf7bc32bea0e7421379b1c543c2d82f69aa179%40%3Cdev.beam.apache.org%3E > > > > On Tue, Jun 2, 2020 at 9:30 AM Piotr Szuberski < > [email protected]> > > wrote: > > > > > Sure, I'll do that > > > > > > On 2020/05/28 17:54:49, Chamikara Jayalath <[email protected]> > wrote: > > > > Great. Thanks for working on this. Can you please add these tasks and > > > JIRAs > > > > to the cross-language transforms roadmap under "Connector/transform > > > > support". > > > > https://beam.apache.org/roadmap/connectors-multi-sdk/ > > > > > > > > Happy to help if you run into any issues during this task. > > > > > > > > <https://beam.apache.org/roadmap/connectors-multi-sdk/>Thanks, > > > > Cham > > > > > > > > On Thu, May 28, 2020 at 9:59 AM Piotr Szuberski < > > > [email protected]> > > > > wrote: > > > > > > > > > I added to Jira task of creating cross-language wrappers for Java > IOs. > > > It > > > > > will soon be in progress. > > > > > > > > > > > > > > >
