A more high level overview: http://www.datastax.com/dev/blog/cql3_collections
"... internally each collection is implemented using one column per element, and with some composite comparator trick to make that work." On 22 Feb 2014, at 12:04, Heiko Braun <ike.br...@googlemail.com> wrote: > Writing the decoder might not be as trivial as looks at a first glance. It > seems the CQL3 driver uses composite columns to store collections: > > http://www.opensourceconnections.com/2013/07/24/understanding-how-cql3-maps-to-cassandras-internal-data-structure-sets-lists-and-maps > > /Heiko > > On 21 Feb 2014, at 21:43, Liam Stewart <liam.stew...@gmail.com> wrote: > >> On Fri, Feb 21, 2014 at 12:12 PM, Heiko Braun <ike.br...@googlemail.com> >> wrote: >> >> >> yes, within calliope the unmarshaller is defined as : >> >> unmarshaller: (CQLRowKeyMap, CQLRowMap) => T >> >> whereas the underlying types for the values are ByteBuffer's. >> >> It depends how you represent the sets within Cassandra. How do you store the >> collection values? >> >> Pretty standard cql3 - sets are on basic types (int, bool) - set<int> - >> while for maps the keys are always strings and values are basic types - >> map<text, int>. Poking around the datastax driver, it should be quite easy >> to unpack them. The format looks like the number of items and then, for each >> item, length + encoded value (and length + encoded value for maps). >> >> >> /Heiko >> >> On 21 Feb 2014, at 19:05, Liam Stewart <liam.stew...@gmail.com> wrote: >> >>> I'm wondering if anybody has had experience with cql3 containers and spark >>> - it seems like with calliope, we would have to provide an unmarshaller >>> that knows how to unpack containers to scala sets/maps? >> >> >> >> >> -- >> Liam Stewart :: liam.stew...@gmail.com >
