On Tue, Jun 16, 2020 at 2:12 PM Brian Hulette <bhule...@google.com> wrote:
> > > On Tue, Jun 16, 2020 at 1:42 PM Piotr Szuberski < > piotr.szuber...@polidea.com> wrote: > >> I published a PR with Jdbc xlang write transform with python wrapper >> (#12023, #12022) - could you suggest someone to assign for CR? > > I could help review these > I'm happy to help as well. Thanks. > >> Btw. For now the standard row coder in python is very limited - it can >> only use simple types (no bytes, datetime, boolean) but I suppose it'll be >> developed in the near future? >> > > BEAM-7996 is tracking support for the remaining primitive types in > Python's row coder. None of these should be that difficult to add I just > haven't had the time myself. Boolean should be especially quick since Chad > made it a standard coder a while ago. > DATETIME is harder. We want it to be a logical type rather than a > primitive type for portable schemas. I was trying to do that with the > MillisInstant PR [1], but that raised a question about logical types and > SQL which we still haven't resolved [2]. > > [1] https://github.com/apache/beam/pull/11456 > [2] > https://lists.apache.org/thread.html/r2e05355b74fb5b8149af78ade1e3539ec08371a9a4b2b9e45737e6be%40%3Cdev.beam.apache.org%3E > > On 2020/06/15 17:08:29, Chamikara Jayalath <chamik...@google.com> wrote: >> > Thanks. +1 for using RowCoder. We should try to use standard coders [1] >> in >> > the x-lang SDK boundaries. >> > If we use other coders (for example, ProtoCoder) it may or may not work >> > depending on how various runners implement support for x-lang. >> > >> > This might require slightly updating existing transforms or adding >> > additional conversion transforms to cross-language builders. >> > >> > [1] >> > >> https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/beam_runner_api.proto#L669 >> > >> > Thanks, >> > Cham >> > >> > On Mon, Jun 15, 2020 at 10:00 AM Piotr Szuberski < >> > piotr.szuber...@polidea.com> wrote: >> > >> > > Right now I'm working on JdbcIO and I'm using Row and Schema >> protobuffs. >> > > I'm figuring out how to use them properly. Thanks for the article - >> for >> > > sure it will be helpful! >> > > >> > > On 2020/06/12 20:32:16, Brian Hulette <bhule...@google.com> wrote: >> > > > Thanks! I see there are jiras for SpannerIO and JdbcIO as part of >> that. >> > > Are >> > > > you planning on using row coder for them? >> > > > If so I want to make sure you're aware of >> > > > https://s.apache.org/beam-schema-io (sent to the dev list last week >> > > > [1]). +Scott >> > > > Lukas <slu...@google.com> will be working on building out the ideas >> > > there >> > > > this summer. His work could be useful for making these IOs >> cross-language >> > > > (and you would get a mapping to SQL out of it without much more >> effort). >> > > > >> > > > Brian >> > > > >> > > > [1] >> > > > >> > > >> https://lists.apache.org/thread.html/rc1695025d41c5dc38cdf7bc32bea0e7421379b1c543c2d82f69aa179%40%3Cdev.beam.apache.org%3E >> > > > >> > > > On Tue, Jun 2, 2020 at 9:30 AM Piotr Szuberski < >> > > piotr.szuber...@polidea.com> >> > > > wrote: >> > > > >> > > > > Sure, I'll do that >> > > > > >> > > > > On 2020/05/28 17:54:49, Chamikara Jayalath <chamik...@google.com> >> > > wrote: >> > > > > > Great. Thanks for working on this. Can you please add these >> tasks and >> > > > > JIRAs >> > > > > > to the cross-language transforms roadmap under >> "Connector/transform >> > > > > > support". >> > > > > > https://beam.apache.org/roadmap/connectors-multi-sdk/ >> > > > > > >> > > > > > Happy to help if you run into any issues during this task. >> > > > > > >> > > > > > <https://beam.apache.org/roadmap/connectors-multi-sdk/>Thanks, >> > > > > > Cham >> > > > > > >> > > > > > On Thu, May 28, 2020 at 9:59 AM Piotr Szuberski < >> > > > > piotr.szuber...@polidea.com> >> > > > > > wrote: >> > > > > > >> > > > > > > I added to Jira task of creating cross-language wrappers for >> Java >> > > IOs. >> > > > > It >> > > > > > > will soon be in progress. >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
