Lets use this - https://plus.google.com/hangouts/_/hoaevent/AP36tYdve-71oizx25DGUbTZjSX4PtLxmDsddtqnfuDYlE9SXDSB9Q?authuser=0&hl=en
I will compile a set of instructions for website so any one of us can preschedule it for future. Sures On Mar 2, 2014, at 7:36 PM, Eran Chinthaka Withana <[email protected]> wrote: > Oops, in that case, Suresh, can you please create one? > > Thanks, > Eran Chinthaka Withana > > > On Sun, Mar 2, 2014 at 4:28 PM, Suresh Marru <[email protected]> wrote: > >> Hi Eran, >> >> Is this a On-Air event? Previously I had trouble changing the previously >> scheduled event to On-Air. >> >> If you are creating a new hangout, can you first create it on G+ Airavata >> Community (all PMC Members are moderators on this community). This will be >> easier for archival reference - >> https://plus.google.com/communities/100700433662281905708 >> >> Suresh >> >> On Mar 2, 2014, at 7:21 PM, Eran Chinthaka Withana < >> [email protected]> wrote: >> >>> Here is the link to hangout: >>> >> https://plus.google.com/hangouts/_/event/c1sgvk7dha37rkr0adktb195lgc?authuser=0&hl=en >>> >>> Thanks, >>> Eran Chinthaka Withana >>> >>> >>> On Sun, Mar 2, 2014 at 12:46 PM, Suresh Marru <[email protected]> wrote: >>> >>>> Hi All, >>>> >>>> Since Eran has been the one who first proposed the hangout and has >>>> specific suggestion on this thread I prefer to postpone to 8pm (EST). >> But >>>> if others planned for 4pm, lets goahead with the plan. >>>> >>>> Any one who planned to attend now cannot make it at 8pm (EST)? If do not >>>> hear any objections lets shoot for 8pm. Otherwise, lets go as planned. >>>> >>>> Cheers, >>>> Suresh >>>> >>>> On Mar 2, 2014, at 3:31 PM, Eran Chinthaka Withana < >>>> [email protected]> wrote: >>>> >>>>> Hi Suresh, >>>>> >>>>> Sorry for the late reply. I don't think I can make it at 1pm PST today. >>>> Can >>>>> we please re-schedule this to 5pm PST (8pm EST) or later? >>>>> >>>>> Thanks, >>>>> Eran Chinthaka Withana >>>>> >>>>> >>>>> On Sun, Mar 2, 2014 at 6:38 AM, Suresh Marru <[email protected]> >> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> Great to see we have a good quorum. So how about 4pm EST (1pm PST) >> today >>>>>> with a hangout on air. It works best if we start a a hangout then >>>> (previous >>>>>> attempts to pre-schedules on-air events did not work well. So please >>>> check >>>>>> this mailing list around 4pm EST for the hangout on air link. >>>>>> >>>>>> Meanwhile, please join the Airavata Google Plus community, that might >> be >>>>>> easier to share the link - >>>>>> https://plus.google.com/communities/100700433662281905708 >>>>>> >>>>>> Thanks all for willing to take time on a sunday, >>>>>> Suresh >>>>>> >>>>>> On Feb 28, 2014, at 9:15 PM, Supun Kamburugamuva <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> +1 for Sunday afternoon. I can make it after 4 pm EST. >>>>>>> >>>>>>> Thanks, >>>>>>> Supun.. >>>>>>> >>>>>>> >>>>>>> On Fri, Feb 28, 2014 at 5:04 PM, Shameera Rathnayaka < >>>>>> [email protected] >>>>>>>> wrote: >>>>>>> >>>>>>>> +1 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Shameera. >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Mar 1, 2014 at 3:11 AM, Eran Chinthaka Withana < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> +1 for Sunday afternoon >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Eran Chinthaka Withana >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Feb 28, 2014 at 5:17 AM, Suresh Marru <[email protected]> >>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Eran, >>>>>>>>>> >>>>>>>>>> This is a great idea. I myself owe few replies on this thread and >>>>>>>> unable >>>>>>>>>> to take time to comprehend my thoughts (and realized I should take >>>>>> time >>>>>>>>> to >>>>>>>>>> properly articulate the challenges otherwise we will be discussing >>>>>>>>>> orthogonal issues). >>>>>>>>>> >>>>>>>>>> A hangout will help us brainstorm more comprehensively. We can >> have >>>> it >>>>>>>> on >>>>>>>>>> air so we can refer back for archival purposes. How is Sunday >>>>>> afternoon >>>>>>>>> for >>>>>>>>>> everyone willing to join and contribute? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Suresh >>>>>>>>>> >>>>>>>>>> On Feb 28, 2014, at 1:45 AM, Eran Chinthaka Withana < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Is there any chance of hosting a google hangout to talk about >>>> this. I >>>>>>>>>> think >>>>>>>>>>> with long emails and multiple directions things are getting >> little >>>>>>>> bit >>>>>>>>>>> confusing in thread (I'm partly responsible for this :) ). I can >>>>>>>> join a >>>>>>>>>>> video chat during a weekend but lets make sure its convenient for >>>>>>>> both >>>>>>>>>> east >>>>>>>>>>> and west coasts :) >>>>>>>>>>> >>>>>>>>>>> WDYT? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Eran Chinthaka Withana >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Mon, Feb 24, 2014 at 9:32 AM, Suresh Marru <[email protected] >>> >>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> I could respond to each thread in detail, but I see the general >>>>>>>> sense >>>>>>>>> is >>>>>>>>>>>> inquiring on the use case, so let me try and explain this and >> see >>>> if >>>>>>>>> it >>>>>>>>>>>> comes across. I am fully onboard with perceptions of relational >> vs >>>>>>>>> nosql >>>>>>>>>>>> and also agree current Airavata needs are not a direct map for >>>> NoSQL >>>>>>>>>>>> migration. I will summarize the driving motivation: >>>>>>>>>>>> >>>>>>>>>>>> Background: The key problem Airavata needs to solve is getting >> the >>>>>>>> API >>>>>>>>>> and >>>>>>>>>>>> associated data model right. The problem is current relational >>>>>>>>> database >>>>>>>>>>>> (with OpenJPA overlay) is severely limiting the API evolution. >>>>>>>> Science >>>>>>>>>>>> Gateways by nature are very science domain and use-case >> specific. >>>>>>>> But >>>>>>>>>>>> Airavata is tackling this challenging problem of providing a >>>> generic >>>>>>>>> API >>>>>>>>>>>> which will meet and enable these use case centric integration. >> The >>>>>>>>> issue >>>>>>>>>>>> here is, we are designing an API to handle a wide range of known >>>>>>>> (and >>>>>>>>>> some >>>>>>>>>>>> foreseen) use cases. But at the same time trying to keep it >> simple >>>>>>>> and >>>>>>>>>> yet >>>>>>>>>>>> flexible. The only way we can get through a reasonable, >> normalized >>>>>>>>>> version >>>>>>>>>>>> of API is by hands-on programming against the API. Within the >>>>>>>> Airavata >>>>>>>>>> PMC >>>>>>>>>>>> itself, we can solicit a half-a-dozen different ways on how to >>>>>>>>> visualize >>>>>>>>>>>> the data model. And we need few hackethon's with real-end users >> of >>>>>>>>>> Airavata >>>>>>>>>>>> until we find a common ground. All of this needs rapid >>>> prototyping. >>>>>>>>>>>> Currently a slight change in the data model is taking close to >> two >>>>>>>>>> weeks of >>>>>>>>>>>> re-arcitecting the Open-JPA based registry. There are many known >>>>>>>>>> problems >>>>>>>>>>>> with current draft of data model which have to be put-down in >> the >>>>>>>>>> interest >>>>>>>>>>>> of making over all system progress. >>>>>>>>>>>> >>>>>>>>>>>> So the driving motivation is not certainly any of the classic >>>> NoSQL >>>>>>>>>> needs. >>>>>>>>>>>> But a simple one, can we have registry which is schema-agnostic >>>> and >>>>>>>>> yet >>>>>>>>>> is >>>>>>>>>>>> queriable for most of the fields in the model? Can we try 10 >>>>>>>> different >>>>>>>>>>>> variants of data model (hence API) within the next 3 months with >>>>>>>>> focused >>>>>>>>>>>> hackethon's and arrive at a stable 1.0 version of API? >>>>>>>>>>>> >>>>>>>>>>>> Part one is the discussion is successful that it raised every >>>> one's >>>>>>>>> eye >>>>>>>>>>>> brows. Now that we have every one's attention, what will be a >> good >>>>>>>>> data >>>>>>>>>>>> store for Airavata which will meet these needs? >>>>>>>>>>>> >>>>>>>>>>>> P.S: Additional background: The API has been in development for >>>>>>>> close >>>>>>>>>> to 3 >>>>>>>>>>>> years and is falling short of pleasing a majority. Many academic >>>>>>>>>>>> standardization efforts fail terribly trying to pretend to >>>>>>>> understand >>>>>>>>>> all >>>>>>>>>>>> use cases and proposing a standard way (which ends up >>>> unnecessarily >>>>>>>>>> complex >>>>>>>>>>>> and not usable). Science by nature is evolutionary, and >>>> restricting >>>>>>>>> the >>>>>>>>>>>> capabilities by a known set of use cases prevents the use of >>>>>>>>> middleware >>>>>>>>>> for >>>>>>>>>>>> real-scientific research (and gets limited to proof of concept >>>>>>>>>>>> demonstrations, papers, educational use). The only way meeting >> the >>>>>>>>>>>> challenges of these evolving needs is to have the framework >> which >>>>>>>> can >>>>>>>>>>>> evolve with minimal disruption. >>>>>>>>>>>> >>>>>>>>>>>> Great thoughts so far, please keep 'em coming until we can find >> a >>>>>>>>>> solution >>>>>>>>>>>> not by the technical fancies but to address the real need. >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> Suresh >>>>>>>>>>>> >>>>>>>>>>>> On Feb 24, 2014, at 11:53 AM, Lahiru Gunathilake < >>>> [email protected] >>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On Mon, Feb 24, 2014 at 11:20 AM, Milinda Pathirage < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I also think that moving to Cassandra or any other NoSQL will >>>> add >>>>>>>>>>>>>> unneccessary complexity to your solution. Also designing >> proper >>>>>>>>> (easy >>>>>>>>>> to >>>>>>>>>>>>>> manage changes, easy to query) NoSQL data models are hard >>>> (AFAIK, >>>>>>>>>>>> require >>>>>>>>>>>>>> lots of experience and understanding about data structures and >>>>>>>>>> queries). >>>>>>>>>>>>>> Also migrating from one NoSQL technology to other can require >>>>>>>>> complete >>>>>>>>>>>>>> re-write. And current relational databases can handle heavy >>>> loads >>>>>>>>>> except >>>>>>>>>>>>>> Google, Twitter, Amazon and Facebook like loads. I don't think >>>>>>>>>> Airavata >>>>>>>>>>>>>> will see Google and Amazon like loads. >>>>>>>>>>>>>> >>>>>>>>>>>>> +1 >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> If the constant changes to the data model is the problem , I >>>> think >>>>>>>>>> best >>>>>>>>>>>>>> option is to abstract registry implementation to something >> like >>>>>>>>>>>> collections >>>>>>>>>>>>>> and resources used in WSO2 Registry [1] or something suitable >>>> for >>>>>>>>>>>> Airavata >>>>>>>>>>>>>> context. That will make it easy to handle changes in data >> model. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Also don't let the technologies drive design decision. Its >>>> always >>>>>>>>>>>> better to >>>>>>>>>>>>>> let use cases drive the design decision. >>>>>>>>>>>>>> >>>>>>>>>>>>> +1 >>>>>>>>>>>>> >>>>>>>>>>>>> Regards >>>>>>>>>>>>> Lahiru >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> Milinda >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] http://wso2.com/products/governance-registry/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Feb 24, 2014 at 10:57 AM, Supun Kamburugamuva < >>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm not trying to discourage you on your exploration to NoSQL >>>>>>>>>>>> databases. >>>>>>>>>>>>>> I >>>>>>>>>>>>>>> have the following concern. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Your database schema is moderately complex - even for a RDBMS >>>> it >>>>>>>>>> seems >>>>>>>>>>>>>>> complex and the data size is relatively small. I'm not sure >>>> about >>>>>>>>> the >>>>>>>>>>>>>>> current tools available but I think you will need to write >> more >>>>>>>>> code >>>>>>>>>> to >>>>>>>>>>>>>>> support all your requirements in a NoSQL database. So writing >>>>>>>> more >>>>>>>>>> code >>>>>>>>>>>>>> and >>>>>>>>>>>>>>> allow redundancy to support *relatively small* and >> *structured >>>>>>>>>>>>>>> data*doesn't seem right to me. May be I'm wrong and there are >>>>>>>>> better >>>>>>>>>>>>>>> tools in >>>>>>>>>>>>>>> NoSQL than RDBMS, which I doubt. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Supun.. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sun, Feb 23, 2014 at 5:20 PM, Suresh Marru < >>>> [email protected] >>>>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Airavata is actively migrating to use Thrift API for the >>>>>>>> RESTless >>>>>>>>>>>>>> design >>>>>>>>>>>>>>>> and to facilitate various language bindings from client >>>>>>>> gateways. >>>>>>>>>> The >>>>>>>>>>>>>>>> programming language support in thrift has been so far very >>>>>>>>>>>>>> encouraging. >>>>>>>>>>>>>>>> The current architecture is looking like Figure 1 at [1]. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Language specific clients will be released as thrift SDK's >>>>>>>>> (similar >>>>>>>>>> to >>>>>>>>>>>>>>>> evernote sdk's [1]). These clients will be integrated into >>>>>>>> gateway >>>>>>>>>>>>>>> portals >>>>>>>>>>>>>>>> which connect to the API Server. The API operations brokers >> he >>>>>>>>>> simple >>>>>>>>>>>>>>> calls >>>>>>>>>>>>>>>> into one or more backend CPI calls (Airavata internal >>>> component >>>>>>>>>>>>>>>> interfaces). An example set of mappings are illustrated in >>>>>>>>> Figure 2 >>>>>>>>>>>> at >>>>>>>>>>>>>>>> [1]. The current draft of thrift API for version 0.12 is at >>>> [3], >>>>>>>>>>>> please >>>>>>>>>>>>>>> pay >>>>>>>>>>>>>>>> attention to experiment model at [4]. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> For the persistent store, we had few iterations of Airavata >>>>>>>>> Registry >>>>>>>>>>>>>>>> shifting from a legacy XRegistry to JackRabbit to now a >>>> OpenJPA >>>>>>>>>> based >>>>>>>>>>>>>>>> registry. To allow the API and the associated data models to >>>>>>>>> evolve, >>>>>>>>>>>> it >>>>>>>>>>>>>>>> will be useful to explore object databases so we can store >> the >>>>>>>>>>>>>> serialized >>>>>>>>>>>>>>>> version of thrift objects directly. But it will be nice to >>>> have >>>>>>>>> all >>>>>>>>>>>> (or >>>>>>>>>>>>>>>> most) of the fields queriable. This calls for a more >>>>>>>> column-family >>>>>>>>>>>>>> design >>>>>>>>>>>>>>>> of any NoSQL approaches. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Any recommendations for a registry architecture? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Quickly hacking through I find the following approach a >> viable >>>>>>>>> one: >>>>>>>>>>>>>>>> ZombieDB[5] over astyanax[6] which talks to Cassandra. >>>> Airavata >>>>>>>>> can >>>>>>>>>>>>>>> benefit >>>>>>>>>>>>>>>> immediately from the replication and reliability of >> cassandra >>>>>>>> and >>>>>>>>>>>>>>>> scalability in near future. Some of the model objects like >>>>>>>>>> experiment >>>>>>>>>>>>>>>> creation will need to have strong consistency and most of >> the >>>>>>>>>>>>>> monitoring >>>>>>>>>>>>>>>> can live with eventual consistency. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Critical comments please? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks for your time, >>>>>>>>>>>>>>>> Suresh >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [1] - >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>> >> https://cwiki.apache.org/confluence/display/AIRAVATA/2014/02/23/Brainstorming+Diagrams >>>>>>>>>>>>>>>> [2] - https://dev.evernote.com/doc/ >>>>>>>>>>>>>>>> [3] - >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>> >> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=tree;f=airavata-api/thrift-interface-descriptions;hb=HEAD >>>>>>>>>>>>>>>> [4] - >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>> >> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob_plain;f=airavata-api/thrift-interface-descriptions/experimentModel.thrift;hb=HEAD >>>>>>>>>>>>>>>> [5] - https://github.com/MisterTea/ZombieDB >>>>>>>>>>>>>>>> [6] - https://github.com/Netflix/astyanax >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Supun Kamburugamuva >>>>>>>>>>>>>>> Member, Apache Software Foundation; http://www.apache.org >>>>>>>>>>>>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 >>>>>>>>>>>>>>> Blog: http://supunk.blogspot.com >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Milinda Pathirage >>>>>>>>>>>>>> PhD Student Indiana University, Bloomington; >>>>>>>>>>>>>> E-mail: [email protected] >>>>>>>>>>>>>> Web: http://mpathirage.com >>>>>>>>>>>>>> Blog: http://blog.mpathirage.com >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> System Analyst Programmer >>>>>>>>>>>>> PTI Lab >>>>>>>>>>>>> Indiana University >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Best Regards, >>>>>>>> Shameera Rathnayaka. >>>>>>>> >>>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com >>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/ >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Supun Kamburugamuva >>>>>>> Member, Apache Software Foundation; http://www.apache.org >>>>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 >>>>>>> Blog: http://supunk.blogspot.com >>>>>> >>>>>> >>>> >>>> >> >>
