Hi Lewis, IMHO we are not gaining anything right now. I think we already had this discussion back in 2012 when creating the DynamoDb module as it didn't gain anything from storing in bytes (they were not even supported at the time). Binary formats tend to use up less memory and provide faster access to data compared to text-based formats. However, this requires having a specific software to work with a binary format. This can be more expensive and (as we suffer in Gora) it can be harder to share data. I think this was the reason why Gora was developed using Avro serialization i.e. taking the serialization out of the data store, and doing more low level accesses. But now, most data stores support features similar to databases and this original design might not be true in all use cases not in all data stores. AFAIK Cassandra has left Avro like serialization long time ago (pre 1.2). Cassandra has much more optimized techniques for querying and storing data. I think we should remain presenting the data model abstraction (which is useful), but provide different serialization formats depending on the data stores.
Renato M. 2014-09-12 19:46 GMT+02:00 Apostolis Giannakidis <[email protected]>: > Hello Lewis, > > I do not know Cassandra and I am not sure about how persistence is achieved > in gora-cassandra, but to my understanding since Gora is using Avro to > serialise the data then Bytes is the way to go. But I might be completely > wrong here. > > Best, > A. > > > > > > > On Fri, Sep 12, 2014 at 6:36 PM, Lewis John Mcgibbney < > [email protected]> wrote: > > > Hi Folks, > > Currently we persist everything into blobs within Cassandra. > > > > > https://github.com/apache/gora/blob/master/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraStore.java#L513 > > This is not just for Records... it is for all types > > I am wondering what the gains are for us to store everything within > Bytes. > > Do we need to do this? > > I very rarely Query using Gora... I just use the native Query mechanism > for > > wherever my data resides. > > This does however mean, in the case of Cassandra for example, that I use > > cqlsh which consequently means I get Bytes when I try to get anything > from > > anything. > > We know this already... but what I wonder is if we can take advantage of > > Cassandra Type support. > > Thanks > > Lewis > > > > > > > > > > -- > > *Lewis* > > >
