On Feb 23, 2014, at 8:30 PM, Sachith Withana <[email protected]> wrote:
> First of all thanks a lot for the detailed explanation Suresh. > I agree with the fact that with the current implementations and the Data > Models getting complex will require a more efficient way to handle the data > at the back end. > > But can you please elaborate on why you chose ZombieDB over Astyanax? > Hector and DataStax are also used widely in production. ZombieDB is a wrapper and takes care of the thrift model wrapping into cassandra. The key decision is to lock into zombiedb or not, the drivers themselves can be easily replaced with one over other (with some tradeoff’s and small effort). > > One more thing, We were planning to have real time processing of data > stored in the registry using Storm ( or using a better alternative). If > that's the case I suggest we can use Storm Cassandra project [1] as well. > > [1] > https://github.com/hmsonline/storm-cassandra<https://github.com/hmsonline/storm-cassandra> Storm is targeted for stream processing and current Airavata use cases do not fit well, at least not as of now. Suresh > > On Sun, Feb 23, 2014 at 5:20 PM, Suresh Marru <[email protected]> wrote: > >> Hi All, >> >> Airavata is actively migrating to use Thrift API for the RESTless design >> and to facilitate various language bindings from client gateways. The >> programming language support in thrift has been so far very encouraging. >> The current architecture is looking like Figure 1 at [1]. >> >> Language specific clients will be released as thrift SDK's (similar to >> evernote sdk's [1]). These clients will be integrated into gateway portals >> which connect to the API Server. The API operations brokers he simple calls >> into one or more backend CPI calls (Airavata internal component >> interfaces). An example set of mappings are illustrated in Figure 2 at >> [1]. The current draft of thrift API for version 0.12 is at [3], please pay >> attention to experiment model at [4]. >> >> For the persistent store, we had few iterations of Airavata Registry >> shifting from a legacy XRegistry to JackRabbit to now a OpenJPA based >> registry. To allow the API and the associated data models to evolve, it >> will be useful to explore object databases so we can store the serialized >> version of thrift objects directly. But it will be nice to have all (or >> most) of the fields queriable. This calls for a more column-family design >> of any NoSQL approaches. >> >> Any recommendations for a registry architecture? >> >> Quickly hacking through I find the following approach a viable one: >> ZombieDB[5] over astyanax[6] which talks to Cassandra. Airavata can benefit >> immediately from the replication and reliability of cassandra and >> scalability in near future. Some of the model objects like experiment >> creation will need to have strong consistency and most of the monitoring >> can live with eventual consistency. >> >> Critical comments please? >> >> Thanks for your time, >> Suresh >> >> [1] - >> https://cwiki.apache.org/confluence/display/AIRAVATA/2014/02/23/Brainstorming+Diagrams >> [2] - https://dev.evernote.com/doc/ >> [3] - >> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=tree;f=airavata-api/thrift-interface-descriptions;hb=HEAD >> [4] - >> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob_plain;f=airavata-api/thrift-interface-descriptions/experimentModel.thrift;hb=HEAD >> [5] - https://github.com/MisterTea/ZombieDB >> [6] - https://github.com/Netflix/astyanax >> >> > > > -- > Thanks, > Sachith Withana
