PTypeFamily has an as(...) method right now that knows how to convert primitive and collection types from one PTypeFamily to another. I was thinking of extending it to support derived types and the more common kind of generic types-- so like Avros.generics and Writables.writables, which should be able to be converted from one type family to another with a little bit of magic.
On Mon, Mar 25, 2013 at 7:15 PM, Micah Whitacre <[email protected]> wrote: > Is this what you are thinking when you mentioned the PType.as(...) method? > http://pastebin.com/cjwss9Hs > (note i realize not really code complete but just as an example) > > Its similar to what you suggested for the UUID support enhancement. > > or you were really thinking the PType interface would get an "as(...)" > method? that took in the converted PTypeFamily (e.g. > PType.as(PTypeFamily destinationFamily)). > > I could see how making the code on pastebin more generic to at least > support Avros.toWritables(Class<?>) would be useful. > > All of these seem like good enhancements to me. > > On Tue, Mar 19, 2013 at 7:43 PM, Josh Wills <[email protected]> wrote: >> Hey Micah, >> >> Agreed that this isn't an ideal state of affairs. I think the trick for >> PTables should be to have the key type determine the value type, and have >> derived types that would either convert the PType automatically (via the >> PType.as(...) or some such thing), or convert the Avro type to a >> BytesWritable (for Writable keys) or the Writable type to use the >> Avros.writables functionality. I don't think it would be super-complicated >> to try it out and see if there are any edge cases here that I'm missing. >> >> J >> >> >> On Tue, Mar 19, 2013 at 4:31 PM, Micah Whitacre <[email protected]> >> wrote: >>> >>> Out of curiosity why can we not mix PTypes between Writables and >>> Avros? The situation I'm encountering are cases where someone wants >>> to create a PTable<UUID, SomeAvro>. UUID is hidden behind a >>> UUIDWritable (+ MapFn implementations) and exposed as a PType using >>> Writables.derive(...). SomeAvro's PType is exposed using >>> Avros.records(SomeAvro.class). >>> >>> If consumers want to create the PTable<UUID, SomeAvro> they can either >>> do Avros.tableOf(UUID.PTYPE, SomeAvros.PTYPE) or >>> Writables.tableOf(UUID.PType, SomeAvros.PTYPE) but that will throw an >>> IllegalArgumentException because in both cases something doesn't >>> extend the correct type. >>> >>> Part of the confusion is that the method signatures for both are just >>> PType and don't document or restrict the allowed types until runtime. >>> Is there a way that we might be missing to make both flavors of PTypes >>> play nicely and be able to create tables, pairs, etc easily? >>> >>> I was kind of hoping most data providers could hide whether they are >>> Avro, Writable, or whatever by just exposing PTypes for consumer to >>> use but now it seems consumer might need to implement twice as many >>> PTypes and wrappers for a mixed pipeline to work well. >>> >>> Thanks, >>> Micah >> >> >> >> >> -- >> Director of Data Science >> Cloudera >> Twitter: @josh_wills -- Director of Data Science Cloudera Twitter: @josh_wills
