Hi Eran, Is this a On-Air event? Previously I had trouble changing the previously scheduled event to On-Air.
If you are creating a new hangout, can you first create it on G+ Airavata Community (all PMC Members are moderators on this community). This will be easier for archival reference - https://plus.google.com/communities/100700433662281905708 Suresh On Mar 2, 2014, at 7:21 PM, Eran Chinthaka Withana <[email protected]> wrote: > Here is the link to hangout: > https://plus.google.com/hangouts/_/event/c1sgvk7dha37rkr0adktb195lgc?authuser=0&hl=en > > Thanks, > Eran Chinthaka Withana > > > On Sun, Mar 2, 2014 at 12:46 PM, Suresh Marru <[email protected]> wrote: > >> Hi All, >> >> Since Eran has been the one who first proposed the hangout and has >> specific suggestion on this thread I prefer to postpone to 8pm (EST). But >> if others planned for 4pm, lets goahead with the plan. >> >> Any one who planned to attend now cannot make it at 8pm (EST)? If do not >> hear any objections lets shoot for 8pm. Otherwise, lets go as planned. >> >> Cheers, >> Suresh >> >> On Mar 2, 2014, at 3:31 PM, Eran Chinthaka Withana < >> [email protected]> wrote: >> >>> Hi Suresh, >>> >>> Sorry for the late reply. I don't think I can make it at 1pm PST today. >> Can >>> we please re-schedule this to 5pm PST (8pm EST) or later? >>> >>> Thanks, >>> Eran Chinthaka Withana >>> >>> >>> On Sun, Mar 2, 2014 at 6:38 AM, Suresh Marru <[email protected]> wrote: >>> >>>> Hi All, >>>> >>>> Great to see we have a good quorum. So how about 4pm EST (1pm PST) today >>>> with a hangout on air. It works best if we start a a hangout then >> (previous >>>> attempts to pre-schedules on-air events did not work well. So please >> check >>>> this mailing list around 4pm EST for the hangout on air link. >>>> >>>> Meanwhile, please join the Airavata Google Plus community, that might be >>>> easier to share the link - >>>> https://plus.google.com/communities/100700433662281905708 >>>> >>>> Thanks all for willing to take time on a sunday, >>>> Suresh >>>> >>>> On Feb 28, 2014, at 9:15 PM, Supun Kamburugamuva <[email protected]> >>>> wrote: >>>> >>>>> +1 for Sunday afternoon. I can make it after 4 pm EST. >>>>> >>>>> Thanks, >>>>> Supun.. >>>>> >>>>> >>>>> On Fri, Feb 28, 2014 at 5:04 PM, Shameera Rathnayaka < >>>> [email protected] >>>>>> wrote: >>>>> >>>>>> +1 >>>>>> >>>>>> Thanks, >>>>>> Shameera. >>>>>> >>>>>> >>>>>> On Sat, Mar 1, 2014 at 3:11 AM, Eran Chinthaka Withana < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> +1 for Sunday afternoon >>>>>>> >>>>>>> Thanks, >>>>>>> Eran Chinthaka Withana >>>>>>> >>>>>>> >>>>>>> On Fri, Feb 28, 2014 at 5:17 AM, Suresh Marru <[email protected]> >>>> wrote: >>>>>>> >>>>>>>> Hi Eran, >>>>>>>> >>>>>>>> This is a great idea. I myself owe few replies on this thread and >>>>>> unable >>>>>>>> to take time to comprehend my thoughts (and realized I should take >>>> time >>>>>>> to >>>>>>>> properly articulate the challenges otherwise we will be discussing >>>>>>>> orthogonal issues). >>>>>>>> >>>>>>>> A hangout will help us brainstorm more comprehensively. We can have >> it >>>>>> on >>>>>>>> air so we can refer back for archival purposes. How is Sunday >>>> afternoon >>>>>>> for >>>>>>>> everyone willing to join and contribute? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Suresh >>>>>>>> >>>>>>>> On Feb 28, 2014, at 1:45 AM, Eran Chinthaka Withana < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> Is there any chance of hosting a google hangout to talk about >> this. I >>>>>>>> think >>>>>>>>> with long emails and multiple directions things are getting little >>>>>> bit >>>>>>>>> confusing in thread (I'm partly responsible for this :) ). I can >>>>>> join a >>>>>>>>> video chat during a weekend but lets make sure its convenient for >>>>>> both >>>>>>>> east >>>>>>>>> and west coasts :) >>>>>>>>> >>>>>>>>> WDYT? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Eran Chinthaka Withana >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Feb 24, 2014 at 9:32 AM, Suresh Marru <[email protected]> >>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I could respond to each thread in detail, but I see the general >>>>>> sense >>>>>>> is >>>>>>>>>> inquiring on the use case, so let me try and explain this and see >> if >>>>>>> it >>>>>>>>>> comes across. I am fully onboard with perceptions of relational vs >>>>>>> nosql >>>>>>>>>> and also agree current Airavata needs are not a direct map for >> NoSQL >>>>>>>>>> migration. I will summarize the driving motivation: >>>>>>>>>> >>>>>>>>>> Background: The key problem Airavata needs to solve is getting the >>>>>> API >>>>>>>> and >>>>>>>>>> associated data model right. The problem is current relational >>>>>>> database >>>>>>>>>> (with OpenJPA overlay) is severely limiting the API evolution. >>>>>> Science >>>>>>>>>> Gateways by nature are very science domain and use-case specific. >>>>>> But >>>>>>>>>> Airavata is tackling this challenging problem of providing a >> generic >>>>>>> API >>>>>>>>>> which will meet and enable these use case centric integration. The >>>>>>> issue >>>>>>>>>> here is, we are designing an API to handle a wide range of known >>>>>> (and >>>>>>>> some >>>>>>>>>> foreseen) use cases. But at the same time trying to keep it simple >>>>>> and >>>>>>>> yet >>>>>>>>>> flexible. The only way we can get through a reasonable, normalized >>>>>>>> version >>>>>>>>>> of API is by hands-on programming against the API. Within the >>>>>> Airavata >>>>>>>> PMC >>>>>>>>>> itself, we can solicit a half-a-dozen different ways on how to >>>>>>> visualize >>>>>>>>>> the data model. And we need few hackethon's with real-end users of >>>>>>>> Airavata >>>>>>>>>> until we find a common ground. All of this needs rapid >> prototyping. >>>>>>>>>> Currently a slight change in the data model is taking close to two >>>>>>>> weeks of >>>>>>>>>> re-arcitecting the Open-JPA based registry. There are many known >>>>>>>> problems >>>>>>>>>> with current draft of data model which have to be put-down in the >>>>>>>> interest >>>>>>>>>> of making over all system progress. >>>>>>>>>> >>>>>>>>>> So the driving motivation is not certainly any of the classic >> NoSQL >>>>>>>> needs. >>>>>>>>>> But a simple one, can we have registry which is schema-agnostic >> and >>>>>>> yet >>>>>>>> is >>>>>>>>>> queriable for most of the fields in the model? Can we try 10 >>>>>> different >>>>>>>>>> variants of data model (hence API) within the next 3 months with >>>>>>> focused >>>>>>>>>> hackethon's and arrive at a stable 1.0 version of API? >>>>>>>>>> >>>>>>>>>> Part one is the discussion is successful that it raised every >> one's >>>>>>> eye >>>>>>>>>> brows. Now that we have every one's attention, what will be a good >>>>>>> data >>>>>>>>>> store for Airavata which will meet these needs? >>>>>>>>>> >>>>>>>>>> P.S: Additional background: The API has been in development for >>>>>> close >>>>>>>> to 3 >>>>>>>>>> years and is falling short of pleasing a majority. Many academic >>>>>>>>>> standardization efforts fail terribly trying to pretend to >>>>>> understand >>>>>>>> all >>>>>>>>>> use cases and proposing a standard way (which ends up >> unnecessarily >>>>>>>> complex >>>>>>>>>> and not usable). Science by nature is evolutionary, and >> restricting >>>>>>> the >>>>>>>>>> capabilities by a known set of use cases prevents the use of >>>>>>> middleware >>>>>>>> for >>>>>>>>>> real-scientific research (and gets limited to proof of concept >>>>>>>>>> demonstrations, papers, educational use). The only way meeting the >>>>>>>>>> challenges of these evolving needs is to have the framework which >>>>>> can >>>>>>>>>> evolve with minimal disruption. >>>>>>>>>> >>>>>>>>>> Great thoughts so far, please keep 'em coming until we can find a >>>>>>>> solution >>>>>>>>>> not by the technical fancies but to address the real need. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Suresh >>>>>>>>>> >>>>>>>>>> On Feb 24, 2014, at 11:53 AM, Lahiru Gunathilake < >> [email protected] >>>>>>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> On Mon, Feb 24, 2014 at 11:20 AM, Milinda Pathirage < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> I also think that moving to Cassandra or any other NoSQL will >> add >>>>>>>>>>>> unneccessary complexity to your solution. Also designing proper >>>>>>> (easy >>>>>>>> to >>>>>>>>>>>> manage changes, easy to query) NoSQL data models are hard >> (AFAIK, >>>>>>>>>> require >>>>>>>>>>>> lots of experience and understanding about data structures and >>>>>>>> queries). >>>>>>>>>>>> Also migrating from one NoSQL technology to other can require >>>>>>> complete >>>>>>>>>>>> re-write. And current relational databases can handle heavy >> loads >>>>>>>> except >>>>>>>>>>>> Google, Twitter, Amazon and Facebook like loads. I don't think >>>>>>>> Airavata >>>>>>>>>>>> will see Google and Amazon like loads. >>>>>>>>>>>> >>>>>>>>>>> +1 >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> If the constant changes to the data model is the problem , I >> think >>>>>>>> best >>>>>>>>>>>> option is to abstract registry implementation to something like >>>>>>>>>> collections >>>>>>>>>>>> and resources used in WSO2 Registry [1] or something suitable >> for >>>>>>>>>> Airavata >>>>>>>>>>>> context. That will make it easy to handle changes in data model. >>>>>>>>>>>> >>>>>>>>>>>> Also don't let the technologies drive design decision. Its >> always >>>>>>>>>> better to >>>>>>>>>>>> let use cases drive the design decision. >>>>>>>>>>>> >>>>>>>>>>> +1 >>>>>>>>>>> >>>>>>>>>>> Regards >>>>>>>>>>> Lahiru >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> Milinda >>>>>>>>>>>> >>>>>>>>>>>> [1] http://wso2.com/products/governance-registry/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Feb 24, 2014 at 10:57 AM, Supun Kamburugamuva < >>>>>>>>>> [email protected] >>>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not trying to discourage you on your exploration to NoSQL >>>>>>>>>> databases. >>>>>>>>>>>> I >>>>>>>>>>>>> have the following concern. >>>>>>>>>>>>> >>>>>>>>>>>>> Your database schema is moderately complex - even for a RDBMS >> it >>>>>>>> seems >>>>>>>>>>>>> complex and the data size is relatively small. I'm not sure >> about >>>>>>> the >>>>>>>>>>>>> current tools available but I think you will need to write more >>>>>>> code >>>>>>>> to >>>>>>>>>>>>> support all your requirements in a NoSQL database. So writing >>>>>> more >>>>>>>> code >>>>>>>>>>>> and >>>>>>>>>>>>> allow redundancy to support *relatively small* and *structured >>>>>>>>>>>>> data*doesn't seem right to me. May be I'm wrong and there are >>>>>>> better >>>>>>>>>>>>> tools in >>>>>>>>>>>>> NoSQL than RDBMS, which I doubt. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Supun.. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Sun, Feb 23, 2014 at 5:20 PM, Suresh Marru < >> [email protected] >>>>>>> >>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Airavata is actively migrating to use Thrift API for the >>>>>> RESTless >>>>>>>>>>>> design >>>>>>>>>>>>>> and to facilitate various language bindings from client >>>>>> gateways. >>>>>>>> The >>>>>>>>>>>>>> programming language support in thrift has been so far very >>>>>>>>>>>> encouraging. >>>>>>>>>>>>>> The current architecture is looking like Figure 1 at [1]. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Language specific clients will be released as thrift SDK's >>>>>>> (similar >>>>>>>> to >>>>>>>>>>>>>> evernote sdk's [1]). These clients will be integrated into >>>>>> gateway >>>>>>>>>>>>> portals >>>>>>>>>>>>>> which connect to the API Server. The API operations brokers he >>>>>>>> simple >>>>>>>>>>>>> calls >>>>>>>>>>>>>> into one or more backend CPI calls (Airavata internal >> component >>>>>>>>>>>>>> interfaces). An example set of mappings are illustrated in >>>>>>> Figure 2 >>>>>>>>>> at >>>>>>>>>>>>>> [1]. The current draft of thrift API for version 0.12 is at >> [3], >>>>>>>>>> please >>>>>>>>>>>>> pay >>>>>>>>>>>>>> attention to experiment model at [4]. >>>>>>>>>>>>>> >>>>>>>>>>>>>> For the persistent store, we had few iterations of Airavata >>>>>>> Registry >>>>>>>>>>>>>> shifting from a legacy XRegistry to JackRabbit to now a >> OpenJPA >>>>>>>> based >>>>>>>>>>>>>> registry. To allow the API and the associated data models to >>>>>>> evolve, >>>>>>>>>> it >>>>>>>>>>>>>> will be useful to explore object databases so we can store the >>>>>>>>>>>> serialized >>>>>>>>>>>>>> version of thrift objects directly. But it will be nice to >> have >>>>>>> all >>>>>>>>>> (or >>>>>>>>>>>>>> most) of the fields queriable. This calls for a more >>>>>> column-family >>>>>>>>>>>> design >>>>>>>>>>>>>> of any NoSQL approaches. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Any recommendations for a registry architecture? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Quickly hacking through I find the following approach a viable >>>>>>> one: >>>>>>>>>>>>>> ZombieDB[5] over astyanax[6] which talks to Cassandra. >> Airavata >>>>>>> can >>>>>>>>>>>>> benefit >>>>>>>>>>>>>> immediately from the replication and reliability of cassandra >>>>>> and >>>>>>>>>>>>>> scalability in near future. Some of the model objects like >>>>>>>> experiment >>>>>>>>>>>>>> creation will need to have strong consistency and most of the >>>>>>>>>>>> monitoring >>>>>>>>>>>>>> can live with eventual consistency. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Critical comments please? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your time, >>>>>>>>>>>>>> Suresh >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] - >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >> https://cwiki.apache.org/confluence/display/AIRAVATA/2014/02/23/Brainstorming+Diagrams >>>>>>>>>>>>>> [2] - https://dev.evernote.com/doc/ >>>>>>>>>>>>>> [3] - >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=tree;f=airavata-api/thrift-interface-descriptions;hb=HEAD >>>>>>>>>>>>>> [4] - >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>> >> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob_plain;f=airavata-api/thrift-interface-descriptions/experimentModel.thrift;hb=HEAD >>>>>>>>>>>>>> [5] - https://github.com/MisterTea/ZombieDB >>>>>>>>>>>>>> [6] - https://github.com/Netflix/astyanax >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Supun Kamburugamuva >>>>>>>>>>>>> Member, Apache Software Foundation; http://www.apache.org >>>>>>>>>>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 >>>>>>>>>>>>> Blog: http://supunk.blogspot.com >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Milinda Pathirage >>>>>>>>>>>> PhD Student Indiana University, Bloomington; >>>>>>>>>>>> E-mail: [email protected] >>>>>>>>>>>> Web: http://mpathirage.com >>>>>>>>>>>> Blog: http://blog.mpathirage.com >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> System Analyst Programmer >>>>>>>>>>> PTI Lab >>>>>>>>>>> Indiana University >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best Regards, >>>>>> Shameera Rathnayaka. >>>>>> >>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com >>>>>> Blog : http://shameerarathnayaka.blogspot.com/ >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Supun Kamburugamuva >>>>> Member, Apache Software Foundation; http://www.apache.org >>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 >>>>> Blog: http://supunk.blogspot.com >>>> >>>> >> >>
