Borries, I haven't used this vnc thing before. Do I just need the free client mentioned on this page? http://www.realvnc.com/products/vnc/
Mark -----Original Message----- From: Borries Demeler [mailto:[email protected]] Sent: Friday, March 14, 2014 11:48 AM To: [email protected] Subject: Re: Experiment Summary retrieval Marlon, would it be possible to put me on the agenda for about 30 mins. during our next telecon? Everyone should have a vnc viewer client on their computer to follow along. Thanks, -borries On Fri, Mar 14, 2014 at 06:42:46PM +0000, Miller, Mark wrote: > I think that would be useful, personally. > > -----Original Message----- > From: Borries Demeler [mailto:[email protected]] > Sent: Friday, March 14, 2014 11:37 AM > To: [email protected] > Subject: Re: Experiment Summary retrieval > > > Mark, Lahiru and others: > > If you guys aren't averse to yet another online conference, I would be happy > to give an on-screen demo of our system. It is a bit difficult to explain per > email how this rather complicated system is organized, but it may be > worthwhile to get some insights of one possible solution to how this can be > handled. I wouldn't say that the gateway is completely separated from the > task of processing output data, but the task is abstracted to a very basic > and uncomplicated level, and then the more complex issues are handled in a > custom fashion by our software. This way we can on the one hand have robust > communications and on the other fine-grained and customized data handling for > the specific needs of the user. I would probably need about 20-30 mins of > your time. Perhaps during the next SciGap meeting? > It would be best if everyone had a vnc viewer on their desktop to follow > along. > > -Borries > > > On Fri, Mar 14, 2014 at 04:49:15PM +0000, Miller, Mark wrote: > > Very interesting Borries. So if I read that right, delivery of results for > > your Gateway takes place by a completely separate system. > > > > I think the RESTful access we provide will be similar in some ways. We > > will be passing results directly to the user's application, which will be > > in effect like the UltraScan package, I think. In that case, what the user > > extracts from the data will be up to their application. Of course we will > > track the relevant usage parameters just as we do currently on our side. > > > > The access to intermediate results for ongoing jobs is also critically > > important for us as well, and very much outside of the main Gateway > > software. But we don't currently support access to status of a list of > > running jobs. Although the idea of being able to provide users with a > > window into the status of all their tasks is certainly a nice feature, and > > we would likely consume it and present it to users if it was easy to access. > > > > Mark > > > > > > -----Original Message----- > > From: Borries Demeler [mailto:[email protected]] > > Sent: Friday, March 14, 2014 9:12 AM > > To: [email protected] > > Subject: Re: Experiment Summary retrieval > > > > I am not sure if this relates at all to the getAllExperiments() question, > > but I thought it may help to see how we deal with retrieving expt. data > > from the gateway in UltraScan. > > > > In the case of UltraScan we have the HPC output very much separated from > > the information of the results that are useful to the user. All data are > > getting tar.gzipped into a single archive which is transported back out and > > automatically parsed into a relational database that is part of the > > UltraScan software, and not a function of the gateway. The user will then > > access the database with secondary software and create a type of > > meta-analysis and visualization of the results that ultimately are then > > useful to the user. Having said this, while data are being calculated on > > the HPC resource, the user can monitor the process of the calculation by > > reviewing a queue viewer that shows all pending and active jobs and their > > actual state. This state include a lot of information keeping the user > > apprised of the state. This information is continually being sent via UDP > > from the HPC resource and monitored by a daemon on our backend LIMS server, > > again, this is semi-independent from the gateway software. This may be very > > specialized for our use case, but it is one example of how the problem can > > be solved. > > > > -borries > > > > On Fri, Mar 14, 2014 at 11:37:02AM -0400, Saminda Wijeratne wrote: > > > Thanks for identifying the problem Sachith. To be precise > > > potentially there are many subset variations of interested fields > > > in the Experiment Data which different gateways will be interested > > > in the same usecase or different usecase. We cannot scale by > > > providing different functions for each of these scenarios. > > > > > > > > > On Fri, Mar 14, 2014 at 11:25 AM, Sachith Withana > > > <[email protected]>wrote: > > > > > > > Hi all, > > > > > > > > Almost all gateways have a requirement of retrieving the > > > > experiment summaries of all the experiments . The fields that > > > > are required differ based on the gateway. > > > > > > > > For Example: > > > > CIPRES requires : Experiment name and status Some gateways > > > > requires the Experiment name with the inputs only. > > > > > > > > But right now the getAllExperiments() method returns the list of > > > > all the Experiments with all the experiment related attributes > > > > filled ( the whole Experiment Model). > > > > > > > > It's costly to get the whole Experiment objects from the API > > > > rather than getting the required few attributes. > > > > > > > > Any suggestions on how we could achieve this? > > > > > > > > One suggestion would be to have a getAllExperiments method with > > > > the parameters as a list of required fields of the Experiments > > > > and return only those fields. > > > > > > > > -- > > > > Thanks, > > > > Sachith Withana > > > >
