Oops, in that case, Suresh, can you please create one? Thanks, Eran Chinthaka Withana
On Sun, Mar 2, 2014 at 4:28 PM, Suresh Marru <[email protected]> wrote: > Hi Eran, > > Is this a On-Air event? Previously I had trouble changing the previously > scheduled event to On-Air. > > If you are creating a new hangout, can you first create it on G+ Airavata > Community (all PMC Members are moderators on this community). This will be > easier for archival reference - > https://plus.google.com/communities/100700433662281905708 > > Suresh > > On Mar 2, 2014, at 7:21 PM, Eran Chinthaka Withana < > [email protected]> wrote: > > > Here is the link to hangout: > > > https://plus.google.com/hangouts/_/event/c1sgvk7dha37rkr0adktb195lgc?authuser=0&hl=en > > > > Thanks, > > Eran Chinthaka Withana > > > > > > On Sun, Mar 2, 2014 at 12:46 PM, Suresh Marru <[email protected]> wrote: > > > >> Hi All, > >> > >> Since Eran has been the one who first proposed the hangout and has > >> specific suggestion on this thread I prefer to postpone to 8pm (EST). > But > >> if others planned for 4pm, lets goahead with the plan. > >> > >> Any one who planned to attend now cannot make it at 8pm (EST)? If do not > >> hear any objections lets shoot for 8pm. Otherwise, lets go as planned. > >> > >> Cheers, > >> Suresh > >> > >> On Mar 2, 2014, at 3:31 PM, Eran Chinthaka Withana < > >> [email protected]> wrote: > >> > >>> Hi Suresh, > >>> > >>> Sorry for the late reply. I don't think I can make it at 1pm PST today. > >> Can > >>> we please re-schedule this to 5pm PST (8pm EST) or later? > >>> > >>> Thanks, > >>> Eran Chinthaka Withana > >>> > >>> > >>> On Sun, Mar 2, 2014 at 6:38 AM, Suresh Marru <[email protected]> > wrote: > >>> > >>>> Hi All, > >>>> > >>>> Great to see we have a good quorum. So how about 4pm EST (1pm PST) > today > >>>> with a hangout on air. It works best if we start a a hangout then > >> (previous > >>>> attempts to pre-schedules on-air events did not work well. So please > >> check > >>>> this mailing list around 4pm EST for the hangout on air link. > >>>> > >>>> Meanwhile, please join the Airavata Google Plus community, that might > be > >>>> easier to share the link - > >>>> https://plus.google.com/communities/100700433662281905708 > >>>> > >>>> Thanks all for willing to take time on a sunday, > >>>> Suresh > >>>> > >>>> On Feb 28, 2014, at 9:15 PM, Supun Kamburugamuva <[email protected]> > >>>> wrote: > >>>> > >>>>> +1 for Sunday afternoon. I can make it after 4 pm EST. > >>>>> > >>>>> Thanks, > >>>>> Supun.. > >>>>> > >>>>> > >>>>> On Fri, Feb 28, 2014 at 5:04 PM, Shameera Rathnayaka < > >>>> [email protected] > >>>>>> wrote: > >>>>> > >>>>>> +1 > >>>>>> > >>>>>> Thanks, > >>>>>> Shameera. > >>>>>> > >>>>>> > >>>>>> On Sat, Mar 1, 2014 at 3:11 AM, Eran Chinthaka Withana < > >>>>>> [email protected]> wrote: > >>>>>> > >>>>>>> +1 for Sunday afternoon > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Eran Chinthaka Withana > >>>>>>> > >>>>>>> > >>>>>>> On Fri, Feb 28, 2014 at 5:17 AM, Suresh Marru <[email protected]> > >>>> wrote: > >>>>>>> > >>>>>>>> Hi Eran, > >>>>>>>> > >>>>>>>> This is a great idea. I myself owe few replies on this thread and > >>>>>> unable > >>>>>>>> to take time to comprehend my thoughts (and realized I should take > >>>> time > >>>>>>> to > >>>>>>>> properly articulate the challenges otherwise we will be discussing > >>>>>>>> orthogonal issues). > >>>>>>>> > >>>>>>>> A hangout will help us brainstorm more comprehensively. We can > have > >> it > >>>>>> on > >>>>>>>> air so we can refer back for archival purposes. How is Sunday > >>>> afternoon > >>>>>>> for > >>>>>>>> everyone willing to join and contribute? > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Suresh > >>>>>>>> > >>>>>>>> On Feb 28, 2014, at 1:45 AM, Eran Chinthaka Withana < > >>>>>>>> [email protected]> wrote: > >>>>>>>> > >>>>>>>>> Hi, > >>>>>>>>> > >>>>>>>>> Is there any chance of hosting a google hangout to talk about > >> this. I > >>>>>>>> think > >>>>>>>>> with long emails and multiple directions things are getting > little > >>>>>> bit > >>>>>>>>> confusing in thread (I'm partly responsible for this :) ). I can > >>>>>> join a > >>>>>>>>> video chat during a weekend but lets make sure its convenient for > >>>>>> both > >>>>>>>> east > >>>>>>>>> and west coasts :) > >>>>>>>>> > >>>>>>>>> WDYT? > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Eran Chinthaka Withana > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Mon, Feb 24, 2014 at 9:32 AM, Suresh Marru <[email protected] > > > >>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> I could respond to each thread in detail, but I see the general > >>>>>> sense > >>>>>>> is > >>>>>>>>>> inquiring on the use case, so let me try and explain this and > see > >> if > >>>>>>> it > >>>>>>>>>> comes across. I am fully onboard with perceptions of relational > vs > >>>>>>> nosql > >>>>>>>>>> and also agree current Airavata needs are not a direct map for > >> NoSQL > >>>>>>>>>> migration. I will summarize the driving motivation: > >>>>>>>>>> > >>>>>>>>>> Background: The key problem Airavata needs to solve is getting > the > >>>>>> API > >>>>>>>> and > >>>>>>>>>> associated data model right. The problem is current relational > >>>>>>> database > >>>>>>>>>> (with OpenJPA overlay) is severely limiting the API evolution. > >>>>>> Science > >>>>>>>>>> Gateways by nature are very science domain and use-case > specific. > >>>>>> But > >>>>>>>>>> Airavata is tackling this challenging problem of providing a > >> generic > >>>>>>> API > >>>>>>>>>> which will meet and enable these use case centric integration. > The > >>>>>>> issue > >>>>>>>>>> here is, we are designing an API to handle a wide range of known > >>>>>> (and > >>>>>>>> some > >>>>>>>>>> foreseen) use cases. But at the same time trying to keep it > simple > >>>>>> and > >>>>>>>> yet > >>>>>>>>>> flexible. The only way we can get through a reasonable, > normalized > >>>>>>>> version > >>>>>>>>>> of API is by hands-on programming against the API. Within the > >>>>>> Airavata > >>>>>>>> PMC > >>>>>>>>>> itself, we can solicit a half-a-dozen different ways on how to > >>>>>>> visualize > >>>>>>>>>> the data model. And we need few hackethon's with real-end users > of > >>>>>>>> Airavata > >>>>>>>>>> until we find a common ground. All of this needs rapid > >> prototyping. > >>>>>>>>>> Currently a slight change in the data model is taking close to > two > >>>>>>>> weeks of > >>>>>>>>>> re-arcitecting the Open-JPA based registry. There are many known > >>>>>>>> problems > >>>>>>>>>> with current draft of data model which have to be put-down in > the > >>>>>>>> interest > >>>>>>>>>> of making over all system progress. > >>>>>>>>>> > >>>>>>>>>> So the driving motivation is not certainly any of the classic > >> NoSQL > >>>>>>>> needs. > >>>>>>>>>> But a simple one, can we have registry which is schema-agnostic > >> and > >>>>>>> yet > >>>>>>>> is > >>>>>>>>>> queriable for most of the fields in the model? Can we try 10 > >>>>>> different > >>>>>>>>>> variants of data model (hence API) within the next 3 months with > >>>>>>> focused > >>>>>>>>>> hackethon's and arrive at a stable 1.0 version of API? > >>>>>>>>>> > >>>>>>>>>> Part one is the discussion is successful that it raised every > >> one's > >>>>>>> eye > >>>>>>>>>> brows. Now that we have every one's attention, what will be a > good > >>>>>>> data > >>>>>>>>>> store for Airavata which will meet these needs? > >>>>>>>>>> > >>>>>>>>>> P.S: Additional background: The API has been in development for > >>>>>> close > >>>>>>>> to 3 > >>>>>>>>>> years and is falling short of pleasing a majority. Many academic > >>>>>>>>>> standardization efforts fail terribly trying to pretend to > >>>>>> understand > >>>>>>>> all > >>>>>>>>>> use cases and proposing a standard way (which ends up > >> unnecessarily > >>>>>>>> complex > >>>>>>>>>> and not usable). Science by nature is evolutionary, and > >> restricting > >>>>>>> the > >>>>>>>>>> capabilities by a known set of use cases prevents the use of > >>>>>>> middleware > >>>>>>>> for > >>>>>>>>>> real-scientific research (and gets limited to proof of concept > >>>>>>>>>> demonstrations, papers, educational use). The only way meeting > the > >>>>>>>>>> challenges of these evolving needs is to have the framework > which > >>>>>> can > >>>>>>>>>> evolve with minimal disruption. > >>>>>>>>>> > >>>>>>>>>> Great thoughts so far, please keep 'em coming until we can find > a > >>>>>>>> solution > >>>>>>>>>> not by the technical fancies but to address the real need. > >>>>>>>>>> > >>>>>>>>>> Cheers, > >>>>>>>>>> Suresh > >>>>>>>>>> > >>>>>>>>>> On Feb 24, 2014, at 11:53 AM, Lahiru Gunathilake < > >> [email protected] > >>>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> On Mon, Feb 24, 2014 at 11:20 AM, Milinda Pathirage < > >>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> I also think that moving to Cassandra or any other NoSQL will > >> add > >>>>>>>>>>>> unneccessary complexity to your solution. Also designing > proper > >>>>>>> (easy > >>>>>>>> to > >>>>>>>>>>>> manage changes, easy to query) NoSQL data models are hard > >> (AFAIK, > >>>>>>>>>> require > >>>>>>>>>>>> lots of experience and understanding about data structures and > >>>>>>>> queries). > >>>>>>>>>>>> Also migrating from one NoSQL technology to other can require > >>>>>>> complete > >>>>>>>>>>>> re-write. And current relational databases can handle heavy > >> loads > >>>>>>>> except > >>>>>>>>>>>> Google, Twitter, Amazon and Facebook like loads. I don't think > >>>>>>>> Airavata > >>>>>>>>>>>> will see Google and Amazon like loads. > >>>>>>>>>>>> > >>>>>>>>>>> +1 > >>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> If the constant changes to the data model is the problem , I > >> think > >>>>>>>> best > >>>>>>>>>>>> option is to abstract registry implementation to something > like > >>>>>>>>>> collections > >>>>>>>>>>>> and resources used in WSO2 Registry [1] or something suitable > >> for > >>>>>>>>>> Airavata > >>>>>>>>>>>> context. That will make it easy to handle changes in data > model. > >>>>>>>>>>>> > >>>>>>>>>>>> Also don't let the technologies drive design decision. Its > >> always > >>>>>>>>>> better to > >>>>>>>>>>>> let use cases drive the design decision. > >>>>>>>>>>>> > >>>>>>>>>>> +1 > >>>>>>>>>>> > >>>>>>>>>>> Regards > >>>>>>>>>>> Lahiru > >>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> Thanks > >>>>>>>>>>>> Milinda > >>>>>>>>>>>> > >>>>>>>>>>>> [1] http://wso2.com/products/governance-registry/ > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> On Mon, Feb 24, 2014 at 10:57 AM, Supun Kamburugamuva < > >>>>>>>>>> [email protected] > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Hi all, > >>>>>>>>>>>>> > >>>>>>>>>>>>> I'm not trying to discourage you on your exploration to NoSQL > >>>>>>>>>> databases. > >>>>>>>>>>>> I > >>>>>>>>>>>>> have the following concern. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Your database schema is moderately complex - even for a RDBMS > >> it > >>>>>>>> seems > >>>>>>>>>>>>> complex and the data size is relatively small. I'm not sure > >> about > >>>>>>> the > >>>>>>>>>>>>> current tools available but I think you will need to write > more > >>>>>>> code > >>>>>>>> to > >>>>>>>>>>>>> support all your requirements in a NoSQL database. So writing > >>>>>> more > >>>>>>>> code > >>>>>>>>>>>> and > >>>>>>>>>>>>> allow redundancy to support *relatively small* and > *structured > >>>>>>>>>>>>> data*doesn't seem right to me. May be I'm wrong and there are > >>>>>>> better > >>>>>>>>>>>>> tools in > >>>>>>>>>>>>> NoSQL than RDBMS, which I doubt. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>> Supun.. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Sun, Feb 23, 2014 at 5:20 PM, Suresh Marru < > >> [email protected] > >>>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> Hi All, > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Airavata is actively migrating to use Thrift API for the > >>>>>> RESTless > >>>>>>>>>>>> design > >>>>>>>>>>>>>> and to facilitate various language bindings from client > >>>>>> gateways. > >>>>>>>> The > >>>>>>>>>>>>>> programming language support in thrift has been so far very > >>>>>>>>>>>> encouraging. > >>>>>>>>>>>>>> The current architecture is looking like Figure 1 at [1]. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Language specific clients will be released as thrift SDK's > >>>>>>> (similar > >>>>>>>> to > >>>>>>>>>>>>>> evernote sdk's [1]). These clients will be integrated into > >>>>>> gateway > >>>>>>>>>>>>> portals > >>>>>>>>>>>>>> which connect to the API Server. The API operations brokers > he > >>>>>>>> simple > >>>>>>>>>>>>> calls > >>>>>>>>>>>>>> into one or more backend CPI calls (Airavata internal > >> component > >>>>>>>>>>>>>> interfaces). An example set of mappings are illustrated in > >>>>>>> Figure 2 > >>>>>>>>>> at > >>>>>>>>>>>>>> [1]. The current draft of thrift API for version 0.12 is at > >> [3], > >>>>>>>>>> please > >>>>>>>>>>>>> pay > >>>>>>>>>>>>>> attention to experiment model at [4]. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> For the persistent store, we had few iterations of Airavata > >>>>>>> Registry > >>>>>>>>>>>>>> shifting from a legacy XRegistry to JackRabbit to now a > >> OpenJPA > >>>>>>>> based > >>>>>>>>>>>>>> registry. To allow the API and the associated data models to > >>>>>>> evolve, > >>>>>>>>>> it > >>>>>>>>>>>>>> will be useful to explore object databases so we can store > the > >>>>>>>>>>>> serialized > >>>>>>>>>>>>>> version of thrift objects directly. But it will be nice to > >> have > >>>>>>> all > >>>>>>>>>> (or > >>>>>>>>>>>>>> most) of the fields queriable. This calls for a more > >>>>>> column-family > >>>>>>>>>>>> design > >>>>>>>>>>>>>> of any NoSQL approaches. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Any recommendations for a registry architecture? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Quickly hacking through I find the following approach a > viable > >>>>>>> one: > >>>>>>>>>>>>>> ZombieDB[5] over astyanax[6] which talks to Cassandra. > >> Airavata > >>>>>>> can > >>>>>>>>>>>>> benefit > >>>>>>>>>>>>>> immediately from the replication and reliability of > cassandra > >>>>>> and > >>>>>>>>>>>>>> scalability in near future. Some of the model objects like > >>>>>>>> experiment > >>>>>>>>>>>>>> creation will need to have strong consistency and most of > the > >>>>>>>>>>>> monitoring > >>>>>>>>>>>>>> can live with eventual consistency. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Critical comments please? > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> Thanks for your time, > >>>>>>>>>>>>>> Suresh > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> [1] - > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>> > >> > https://cwiki.apache.org/confluence/display/AIRAVATA/2014/02/23/Brainstorming+Diagrams > >>>>>>>>>>>>>> [2] - https://dev.evernote.com/doc/ > >>>>>>>>>>>>>> [3] - > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>> > >> > https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=tree;f=airavata-api/thrift-interface-descriptions;hb=HEAD > >>>>>>>>>>>>>> [4] - > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>> > >> > https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob_plain;f=airavata-api/thrift-interface-descriptions/experimentModel.thrift;hb=HEAD > >>>>>>>>>>>>>> [5] - https://github.com/MisterTea/ZombieDB > >>>>>>>>>>>>>> [6] - https://github.com/Netflix/astyanax > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> -- > >>>>>>>>>>>>> Supun Kamburugamuva > >>>>>>>>>>>>> Member, Apache Software Foundation; http://www.apache.org > >>>>>>>>>>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 > >>>>>>>>>>>>> Blog: http://supunk.blogspot.com > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> -- > >>>>>>>>>>>> Milinda Pathirage > >>>>>>>>>>>> PhD Student Indiana University, Bloomington; > >>>>>>>>>>>> E-mail: [email protected] > >>>>>>>>>>>> Web: http://mpathirage.com > >>>>>>>>>>>> Blog: http://blog.mpathirage.com > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> -- > >>>>>>>>>>> System Analyst Programmer > >>>>>>>>>>> PTI Lab > >>>>>>>>>>> Indiana University > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Best Regards, > >>>>>> Shameera Rathnayaka. > >>>>>> > >>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com > >>>>>> Blog : http://shameerarathnayaka.blogspot.com/ > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Supun Kamburugamuva > >>>>> Member, Apache Software Foundation; http://www.apache.org > >>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 > >>>>> Blog: http://supunk.blogspot.com > >>>> > >>>> > >> > >> > >
