Deeply nester structures in data stores are difficult. Hive allows List Struct Map to be infinitely nested (to a point). Most Query languages don't have ways to say: navigate to x go to map y inside x find all the values inside y that are = 10
I think they have a name for this: :) http://en.wikipedia.org/wiki/Object-relational_impedance_mismatch So it is pretty cool that cassandra has support for complex types, but how to access them/ slice and dice them is another question. I run into ths problem with hive, we can log VERY complex objects but if someone has to make a UDF to extract meaningful data they are not super useful. In the past have adopted to store json and then user jsonpath queries to extract information. (client side or server side) https://github.com/zznate/intravert-ug/wiki/Server-Side-JsonPath-example Personally I would rather see udfs/language features like lateral view that work on the collections that already exist before this feature. On Wed, Feb 26, 2014 at 2:09 AM, Theo Hultberg <t...@iconara.net> wrote: > thanks mikhail. > > it bothers me a bit that there is the possibility of arbitrarily deeply > nested types (it's great for usability, but a headache for me as a driver > implementer). it feels like just throwing a couple of regexes at this is > the wrong solution, the string has too much structure and hierarchy for > that and I think that not doing it with a proper parser will cause trouble > in the future. and then trying to use the parsed type structure to properly > encode something that the user wants to store will be very complex. kind of > like having to implement a whole orm. it's going to be a challenge. > > I see that you took the parser route in the python driver so I can at least > get some inspiration from there. what do you do when the user passes an > object in a custom type field and that object doesn't look like the type at > all? > > T# > > > On Tue, Feb 25, 2014 at 8:16 PM, Mikhail Stepura < > mikhail.step...@outlook.com> wrote: > > > I just realized that your driver returns fields' names in the type itself > > (which unfortunately is not the case with python driver) so you don't > need > > step #3. > > > > -M > > > > > > > > On 2/25/14, 10:42, Mikhail Stepura wrote: > > > >> The driver shall parse that. > >> I'm not sure if there is a doc for that, but here is what I did for > CQLSH > >> > >> 1. UserType is a CompositeType, where 1s subtype is a KS name, and 2nd > >> is hex-encoded name of the type > >> 2. Remainder of subtypes are types of Usertype's columns. So you can > >> easily decode *values* for fields > >> 3. Information about field *names* is stored in system.schema_usertypes > >> table > >> 4. The driver has to combine pieces 1-3 and create a new class/type for > >> a user. It was easy in Python, I guess it should be easy in Ruby as well > >> > >> -M > >> > >> > >> On 2/22/14, 12:29, Theo Hultberg wrote: > >> > >>> Hi, > >>> > >>> Is there any documentation on how CQL clients should handle the new > user > >>> defined types coming in 2.1? There's nothing in the protocol > >>> specification > >>> on how to handle custom types as far as I can understand. > >>> > >>> For example, I tried creating the "address" type from the description > of > >>> CASSANDRA-5590, and this is how its metadata looks (the metadata for a > >>> query contains a column with a custom type and this is the description > of > >>> it): > >>> > >>> org.apache.cassandra.db.marshal.UserType(user_defined_ > >>> types,61646472657373,737472656574:org.apache. > >>> cassandra.db.marshal.UTF8Type,63697479:org.apache.cassandra. > >>> db.marshal.UTF8Type,7a69705f636f6465:org.apache.cassandra.db.marshal. > >>> Int32Type,70686f6e6573:org.apache.cassandra.db.marshal. > >>> SetType(org.apache.cassandra.db.marshal.UTF8Type)) > >>> > >>> > >>> Is the client supposed to parse that description, and in that case > how? I > >>> could probably figure it out but it would be great if someone could > point > >>> me to the right docs. > >>> > >>> yours, > >>> Theo (author of cql-rb, the Ruby driver) > >>> > >>> > >> > >> > >> > > > > >
