Re: Are unnested coders actually used in Portable SDKS?

2020-06-19 Thread Robert Burke
That makes sense which is how I ended up with item 2. In order to interpret the byte sequence 4,3,a,b,c one would need LP(LP(bytes)) rather than LP(bytes). (And the same for string etc). I've also just noticed that the tests for the Row coder are called out explicitly to be in the unnested

Re: Are unnested coders actually used in Portable SDKS?

2020-06-19 Thread Luke Cwik
Actually, I gave bad information since I wanted to double check what I said. The outer context still exists but only in the case of length prefixing. So testing the 'unnested'/'outer' encoding is still necessary for coders where it differs but it is only used in the LP(X) scenario. So updating the

Re: Are unnested coders actually used in Portable SDKS?

2020-06-19 Thread Luke Cwik
On Fri, Jun 19, 2020 at 11:53 AM Robert Burke wrote: > Marvelous! Thank you Luke! > I do like having less work to do, while finding out the Go SDK is doing > things the right way already (if not yet completely). > > Given that the portable coders are always in the "nested" context, it's >

Re: Are unnested coders actually used in Portable SDKS?

2020-06-19 Thread Robert Burke
Marvelous! Thank you Luke! I do like having less work to do, while finding out the Go SDK is doing things the right way already (if not yet completely). Given that the portable coders are always in the "nested" context, it's reasonable that we don't mention them in the proto. I'm less concerned

Re: Are unnested coders actually used in Portable SDKS?

2020-06-19 Thread Luke Cwik
Thanks for offering to help with the documentation (BEAM-3203). The high level documentation about this is here[1] and the beam_runner_api.proto also contains a bunch of information related to coders. Length prefixing is not the same as the nesting coder property in Java/Python and we have been

Are unnested coders actually used in Portable SDKS?

2020-06-19 Thread Robert Burke
Hello dev@! I have questions. Context: I'm adding Beam Schemas to the Go SDK, and on the way I'm validating the Go SDK coders against standard_coders.yaml, per BEAM-7009[0]. *When is it reasonable for a runner to send an SDK an unnested byte or string coder (AKA, no length prefixing)?* *What

Re: cwiki edit access

2020-06-19 Thread Luke Cwik
I have granted the permissions. Please reach out if there are any issues. On Fri, Jun 19, 2020 at 6:55 AM Damian Gadomski wrote: > Hi, > > I would like to have write (edit) access on the cwiki. I'm planning to > update the Jenkins URLs as we've migrated recently. > > The username is: dgadomski

cwiki edit access

2020-06-19 Thread Damian Gadomski
Hi, I would like to have write (edit) access on the cwiki. I'm planning to update the Jenkins URLs as we've migrated recently. The username is: dgadomski Thanks, Damian

Re: JIRA contributor permission

2020-06-19 Thread Alexey Romanenko
Done. Welcome! > On 18 Jun 2020, at 23:37, Yuwei Fu wrote: > > Sorry I didn't mention my username. > > My JIRA username is: fuyuwei > > On Thu, Jun 18, 2020 at 2:21 PM Yuwei Fu > wrote: > Hello, > > I would like to request JIRA contributor permission.

Re: [ANNOUNCE] New PMC Member: Alexey Romanenko

2020-06-19 Thread Etienne Chauchot
Congrats Alexey ! Well deserved ! Etienne On 17/06/2020 16:30, Gleb Kanterov wrote: Congratulations! Thanks for your hard work On Wed, Jun 17, 2020 at 1:11 PM Alexey Romanenko mailto:aromanenko@gmail.com>> wrote: Thank you Ismaƫl and everybody! Happy to be a part of Beam