Re: Schema-Aware PCollections revisited

One more thing. If anyone here has experience with various OSS metadata stores (e.g. Kafka Schema Registry is one example), would you like to collaborate on implementation? I want to make sure that source schemas can be stored in a variety of OSS metadata stores, and be easily pulled into a Beam

Re: Schema-Aware PCollections revisited

Hi all, If there are no concerns, I would like to start working on a prototype. It's just a prototype, so I don't think it will have the final API (e.g. for the prototype I'm going to avoid change the API of PCollection, and use a "special" Coder instead). Also even once we go beyond prototype,

Re: rename: BeamRecord -> Row

This thread exists so others can weigh in on the name if they want :) If I don't hear any conflicting opinions, I'll merge the PR with Row (though of course this is all still an experimental API, so we can easily change our minds about the name later) On Sat, Feb 3, 2018 at 9:44 AM, Romain

Re: rename: BeamRecord -> Row

This is as true as the renaming is not needed so I guess the PR owner will decide ;). Thanks for the clarification. Romain Manni-Bucau @rmannibucau | Blog | Old Blog | Github

Re: rename: BeamRecord -> Row

Oh I agree 100%, however I'm just saying that we shouldn't ask the SQL effort to halt just because the schema effort overlaps. There's at least one other pending PR on this class (to do with automatic POJO generation). Also the name of the Record/Row class is somewhat independent of everything

Re: rename: BeamRecord -> Row

I know Reuven, but when you check what it does, it is exactly the same and the current work will be to replace by the schema work so better to avoid a round trip of work which will be throw away in any case. Also note that current structure is flat and very limiting for modern SQL so the alignment

Re: rename: BeamRecord -> Row

This is a core part of SQL which is ongoing. On Feb 2, 2018 11:45 PM, "Romain Manni-Bucau" wrote: > Hi > > Shouldnt the discussion on schema which has a direct impact on this > generic container be closed before any action on this? > > > Le 3 févr. 2018 01:09, "Ankur