Please find the proposal for "AMQP Messaging protocol support for Airavata WS-Messenger" at [1]
Thanks, Danushka [1] - https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/danushka/1# On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <[email protected]> wrote: > Ping!! No proposals yet beyond Shameera's and Danushka's place holder. > > Suresh > > On May 1, 2013, at 5:13 PM, Suresh Marru <[email protected]> wrote: > > > Hi Vijayendra, > > > > As you can see from Shameera's proposal, he proposed a JSON conversion > in front of WS Messenger. Also Danuska has been proposing for the AMQP and > idea and deliberating its advantages. So given all these, I would suggest > for you to keep focused on the UI aspects of the monitoring and write into > your proposal a plan for determining a good strategy for the plumbing to > WS-Eventing based existing system. You can have the concrete deliverables > of new UI to change colors based on executions (as it currently does in > XBaya), double click and show error messages and so forth. And keep it > exploratory for the actually messaging format. > > > > I do not have any opinion on the libraries you mentioned, but yaa a ajax > based pub system might be the right way to go. Pending the content format > (JSON or WS-Eventing or JMS or AMQP or something else) > > > > Suresh > > > > On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit < > [email protected]> wrote: > > > >> Hi Suresh > >> > >> I am writing proposal for monitoring tool . The monitoring tool is > based on > >> pub-sub model (ws-messenger). > >> While writing proposal , I have to back it by technical stuff that tells > >> how can we achieve our purpose. > >> As this monitoring tool is supposed to be a web based , and we are > thinking > >> in the lines of > >> developing it in javascript. > >> I was looking into javascript libraries that can we used with > ws-messenger > >> in the monitoring module. > >> Please correct me if I am wrong. > >> I came across some of the libraries > >> > >> - jQuery custom > >> events< > http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events> > >> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/> > >> - PubSubJS <https://github.com/mroderick/PubSubJS> > >> - js-signals <http://millermedeiros.github.com/js-signals/> > >> > >> please tell me am I thinking in right direction? > >> > >> Regards > >> Vijayendra > >> > >> > >> > >> > >> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <[email protected]> wrote: > >> > >>> Hi Shameera, > >>> > >>> This is great, I appreciate you sharing it, I realize this is still > >>> working document, but I want other students to start seeing it and > model > >>> their proposals in a similar way. > >>> > >>> Airavata Mentors, > >>> > >>> Please provide feedback directly on the melange site and uncheck the > >>> "private" box when you comment. > >>> > >>> Suresh > >>> > >>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka < > [email protected]> > >>> wrote: > >>> > >>>> Hi Suresh and All, > >>>> > >>>> Of course I am very much happy to share my proposal with everybody, > >>>> actually i was going to update this thread with the melange link in > few > >>>> hours once i have done writing all the sections in the proposal. I > >>> haven't > >>>> yet added the milestone plan into it and now working on it. > >>>> > >>>> The sub area i am going to work from the Master project is ' > >>> Implementing > >>>> a JSON interface to Airavata Client side and Registry component'. > Here is > >>>> the link > >>>> > >>> > http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002# > >>>> . > >>>> > >>>> > >>>> Please note that i haven't completed everything in this and still > doing > >>>> modifications .Therefore proposal content may be changed bit, need to > add > >>>> more technical details of the approach which explains it well. > >>>> > >>>> I would like to know the feedback from all of you regarding the > proposal > >>>> and will be modifying it if there is anything to be done. Also please > >>>> contact me if you need any help and i am very much willing to share my > >>>> thoughts with all. > >>>> > >>>> Thanks! > >>>> Shameera > >>>> > >>>> > >>>> > >>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <[email protected]> > wrote: > >>>> > >>>>> Hi Shameera, > >>>>> > >>>>> Excellent proposal, great job. Would you mind to make your proposal > >>>>> public and post the link here? Your proposal should help others to > look > >>> at > >>>>> it and learn from. > >>>>> > >>>>> Again I emphasize to all students, please don't feel you will be > >>> competing > >>>>> with each others. If all of you write good proposals, there is a good > >>>>> chance all of you will be selected. But without a good proposal, we > >>> cannot > >>>>> help. > >>>>> > >>>>> Suresh > >>>>> > >>>>> > >>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka < > >>> [email protected]> > >>>>> wrote: > >>>>> > >>>>>> Hi, > >>>>>> > >>>>>> Yes it is not easy to solve all problems, But defining our own > standard > >>>>> or > >>>>>> adhere to any standard > >>>>>> provided by third party library will solve the problem to some > extend. > >>>>>> > >>>>>> Here i see two possible approaches, > >>>>>> > >>>>>> 1. Use existing third party library(we can find which is best) > adhere > >>> to > >>>>> it > >>>>>> standard and see how we change the > >>>>>> backend to be inline with it. > >>>>>> > >>>>>> 2. Use our own convention with help of XMLSchema (The way i > suggest). > >>>>>> > >>>>>> As Suresh mentioned we can do a POC with both approaches to compare > >>>>>> performance > >>>>>> and changes need to be done in server side. Then select the best > one. > >>>>>> > >>>>>> Another question was, can we works with graph data in JSON format. > >>>>>> There are few JS graph framworks[1] which provide that > functionality. > >>>>>> we can use one of them to show airavata monitoring data as graphs > >>>>>> > >>>>>> Thanks, > >>>>>> Shameera. > >>>>>> > >>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/> > , > >>>>>> Processing.js <http://processingjs.org> , Sencha > >>>>>> Charts<http://www.sencha.com/products/extjs/> > >>>>>> > >>>>>> > >>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <[email protected]> > >>> wrote: > >>>>>> > >>>>>>> Hi Vijeyandra, > >>>>>>> > >>>>>>> Airavata Messaging is based on a pub-sub model and the events > >>> themselves > >>>>>>> are xml (WS-Eventing [1]). > >>>>>>> > >>>>>>> The Messenger paper [2] should give you more information. > >>>>>>> > >>>>>>> Hi All (Especially those at WS02): > >>>>>>> > >>>>>>> Here is an old effort from a Morotuwa undergrad project, you may > want > >>> to > >>>>>>> read through these papers and chat with the authors to get > >>> experiences: > >>>>>>> > >>>>>>> http://dl.acm.org/citation.cfm?id=1890807 > >>>>>>> > http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf > >>>>>>> > http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html > >>>>>>> http://mooshabaya.wikidot.com/ > >>>>>>> > >>>>>>> > >>>>> > >>> > http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/ > >>>>>>> > >>>>>>> Suresh > >>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/ > >>>>>>> [2] - > >>>>>>> > >>>>> > >>> > http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf > >>>>>>> > >>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit < > >>>>>>> [email protected]> wrote: > >>>>>>> > >>>>>>>> Hi Suresh > >>>>>>>> > >>>>>>>> I wanted to know more about the monitoring tool . > >>>>>>>> Currently from where does the monitoring tool gets data . Is it > from > >>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that might > >>>>>>> continuously > >>>>>>>> send messages to monitoring tool as to tell how much is the > progress > >>>>>>>> and what are the variables getting changed) ? > >>>>>>>> > >>>>>>>> Again the how is the data being exchanged. I guess it must be xml > ? > >>>>>>>> It must be one way data exchange . I mean the data is TO the > >>>>>>>> monitoring module. > >>>>>>>> Then monitoring Tool is sending back this > >>>>>>>> data to Xbaya for displaying to the user ? Please correct me if I > am > >>>>>>> wrong > >>>>>>>> > >>>>>>>> I have downloaded the source code from the trunk . can you please > >>> point > >>>>>>>> me which part of code should I be code at for this module. > >>>>>>>> > >>>>>>>> Regards > >>>>>>>> Vijayendra > >>>>>>>> > >>>>>>>> > >>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit < > >>>>>>> [email protected]> wrote: > >>>>>>>> Hi > >>>>>>>> > >>>>>>>> What i am suggesting is, we send the JSON message directly to > >>> Airavata > >>>>>>>> Backend(or Registry) > >>>>>>>> When the message gets build after the transport phase, convert > JSON > >>>>>>> message > >>>>>>>> to SOAP(XML). > >>>>>>>> From that point message will treated as SOAP message. > >>>>>>>> > >>>>>>>> If we look at the JSON <--> XML conversion there are set of third > >>> party > >>>>>>>> libraries we > >>>>>>>> can use for. But before selecting a one we need to think about > >>> problems > >>>>>>>> having > >>>>>>>> > >>>>>>>> with JSON <--> XML and how these libraries handle those issues. > >>> Because > >>>>>>> we > >>>>>>>> need a robust > >>>>>>>> way to do this conversions. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> Shameera what you are suggesting is sending the JSON message > directly > >>>>> to > >>>>>>> Registry. > >>>>>>>> when the message gets built after the transport phase , convert > it to > >>>>>>> SOAP . > >>>>>>>> > >>>>>>>> If you are suggesting Registry will have JSON data instead of > WSDL , > >>>>>>> Then this might > >>>>>>>> complicate the things for us . > >>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the > workflows > >>> and > >>>>>>> for other details . > >>>>>>>> Which means we might again have to do some changes with workflow > >>>>>>> interpretor . > >>>>>>>> Yesterday from what I heard in discussion is that , they do not > want > >>> to > >>>>>>> mess with workflow > >>>>>>>> interpreter atleast for GSOC projects. > >>>>>>>> > >>>>>>>> What I want to suggest is , why carry the JSON data till > Regisrty . > >>>>>>> Build a interface > >>>>>>>> before (Apache server API) where we can do the necessary > conversion > >>>>>>> (JSON to SOAP). > >>>>>>>> In this way we can avoid messing up with Airavata server as a > whole. > >>>>>>>> Client ( using a we browser) is interacting with JSON (web > service) > >>> but > >>>>>>> the Apache server > >>>>>>>> is interacting with SOAP. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> Secondly yesterday Suresh was speaking about validating the > >>> connections > >>>>>>> of the workflow. > >>>>>>>> for example , the workflow is expecting a file as input > >>>>>>>> but user is giving a sting or int . > >>>>>>>> > >>>>>>>> Here what I suggest is , while creating wsdl in the registry for a > >>>>>>> particular > >>>>>>>> workflow , we can add extra information in the form of > >>>>>>>> annotation as the kind of input/ output the workflow is > accepting. > >>>>>>>> Then we will be able to check these against users entry during > >>>>> execution. > >>>>>>>> Please correct me if I am wrong. > >>>>>>>> > >>>>>>>> Regards > >>>>>>>> Vijayendra > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee < > [email protected] > >>>> > >>>>>>> wrote: > >>>>>>>> Well exactly, as long as you can define standard way of > >>> communication. > >>>>>>> That > >>>>>>>> is, you can define in advance what should be a string, array and > what > >>>>>>>> should be a integer etc. We have no problem. > >>>>>>>> > >>>>>>>> So, when you look at problems, with JSON <-> XML or the other way > >>>>> round, > >>>>>>>> they talk of the very general case (where you no nothing about the > >>> data > >>>>>>> you > >>>>>>>> are converting other than it is valid XML/JSON). There are a > myriad > >>> of > >>>>>>>> problems in that case, which you pointed out. > >>>>>>>> > >>>>>>>> But when there is standard, there is only one way of doing things, > >>> and > >>>>>>> not > >>>>>>>> several. I think that is the way forward. So what I am proposing > is > >>>>> maybe > >>>>>>>> we all discuss and define this standard within the first week of > GSoC > >>>>>>>> starting and then actually move into coding. So as long as we work > >>> with > >>>>>>> the > >>>>>>>> presumption that this will be done, we really dont have to worry a > >>> lot > >>>>>>>> about this. > >>>>>>>> > >>>>>>>> Cheers, > >>>>>>>> Subho. > >>>>>>>> > >>>>>>>> > >>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka < > >>>>>>>> [email protected]> wrote: > >>>>>>>> > >>>>>>>>> Hi, > >>>>>>>>> > >>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee < > >>> [email protected]> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Some of these problems are very specific to what the XML is > >>>>>>>>> representing, > >>>>>>>>>> it might not be an actual problem in Airavata, > >>>>>>>>>> maybe some one more experienced with the codebase can point this > >>> out. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> All issues pointed out in the paper is not directly valid to our > >>>>>>>>> conversion, I didn't list the issues actually need to address in > >>> this > >>>>>>> case > >>>>>>>>> because thought it is worth to read that introduction part which > >>>>>>> explain > >>>>>>>>> the all the issues we have with this conversion and give us a > solid > >>>>>>>>> background of that. > >>>>>>>>> > >>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets -- > I > >>>>>>>>> really > >>>>>>>>>> dont see these as problems, as long as you can agree that all > >>>>>>> parts of > >>>>>>>>>> airavata will treat the JSON in a standard (probably we have to > >>>>>>> define > >>>>>>>>>> this) way. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> The issue with JSON array only comes when we try to convert XML > to > >>>>>>> JSON not > >>>>>>>>> the other way. If we map with JSON, inputparameters and > >>>>>>> outputparameters in > >>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore > we > >>>>>>> need to > >>>>>>>>> solve this issue. > >>>>>>>>> > >>>>>>>>> JSON XML JSON > >>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> --> > >>> {"inputs":["test"]} > >>>>>>> // > >>>>>>>>> correct one > >>>>>>>>> --> {"inputs":"test"} // incorrect one > >>>>>>>>> > >>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required? > >>>>>>>>> > >>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see > >>>>>>> this > >>>>>>>>>> being > >>>>>>>>>> used is probably in the WSDL, but if we can agree on another way > >>>>>>>>>> of communicating registered applications' I/O parameters to the > >>>>>>> front > >>>>>>>>>> end > >>>>>>>>>> (JSON based), then maybe we can work around this (minor) > problem. > >>>>>>> Are > >>>>>>>>>> custom processing instructions to the Xbaya XML parse even used? > >>>>>>>>>> 3. Attributes -- Again, this can be fixed easily > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> Yes,attributes convertion will not be a big issues we can solve > it. > >>> As > >>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a > big > >>>>>>> issue > >>>>>>>>> with Airavata. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> <array name="abc"> > >>>>>>>>>> <element>1</element> > >>>>>>>>>> <element>2</element> > >>>>>>>>>> <element>3</element> > >>>>>>>>>> <element>4</element> > >>>>>>>>>> </array> > >>>>>>>>>> > >>>>>>>>>> Can become > >>>>>>>>>> > >>>>>>>>>> { > >>>>>>>>>> > >>>>>>>>>> abc : ['1', '2', '3', '4'] > >>>>>>>>>> > >>>>>>>>>> } > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> With this example it show us we need to change the XML message > >>> format > >>>>>>> of > >>>>>>>>> server side, which require to change the all schemas, If we are > >>> going > >>>>>>> to > >>>>>>>>> change the schemas then we need to change the way it process it > in > >>>>>>> Ariavara > >>>>>>>>> core. We have dropped our initial major requirement, which is > keep > >>> the > >>>>>>>>> Airavata Server side as it is. > >>>>>>>>> > >>>>>>>>> with this conversion we only deal with json strings, yes we can > send > >>>>>>> JSON > >>>>>>>>> request with other formats supported by JSON like boolen, null, > >>> Number > >>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML > only > >>>>>>> deal > >>>>>>>>> only with Strings. I think it is good if we can consume a this > >>>>> features > >>>>>>>>> with JSON. > >>>>>>>>> > >>>>>>>>> let say i need to send a integer or float to the server using > JSON > >>>>> then > >>>>>>>>> proper way is to send {"<name>":123.45} this will works fine but > >>>>>>> problem is > >>>>>>>>> how we get the same output ? > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Shameera. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Cheers, > >>>>>>>>>> Subho. > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Best Regards, > >>>>>>>>> Shameera Rathnayaka. > >>>>>>>>> > >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/ > >>>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Best Regards, > >>>>>> Shameera Rathnayaka. > >>>>>> > >>>>>> Blog : http://shameerarathnayaka.blogspot.com/ > >>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> Best Regards, > >>>> Shameera Rathnayaka. > >>>> > >>>> email: shameera AT apache.org , shameerainfo AT gmail.com > >>>> Blog : http://shameerarathnayaka.blogspot.com/ > >>> > >>> > > > >
