Saminda, This a really good implementation. As you said, this can be a very powerful extension of the API.
shouldn’t we also provide a default experiment summary object ( if no fields are set) containing only the Experiment name, status, submit date and the last status update ? Because these would be what most gateways would require essentially. On Mar 20, 2014, at 8:51 PM, Saminda Wijeratne <[email protected]> wrote: > Emre, that looks exactly like what we want. Thanks for pointing it out. We > were targeting for a simple version of "query" for 0.12, but after looking at > mongodb docs I can see that if we plan ahead this can be a very powerful > extension to the API with regards to data retrieval in future. > > Terri, you have a very valid point there. Right now its an implied "and". We > are trying to keep the data model for SearchQuery as simple as possible. My > current mode of thinking is hirarchical filter fields such as, > > {Experiment.Project, "manhattan"} > => Experiment.Project=="manhattan" > > {{Experiment.Project, "manhattan"}, {Experiment.Created, "03-19-2014",">"}} > > => Experiment.Project=="manhattan" && Experiment.Created >03-19-2014 > > {{Experiment.Project, "manhattan"}, {{Experiment.Status, > "done","OR"},{Experiment.Status, "failed"}}, {Experiment.Created, > "03-19-2014",">"}} > => Experiment.Project=="manhattan" && (Experiment.Status=="done" || > Experiment.Status=="failed") && Experiment.Created >03-19-2014 > > Loosely defined, following could be the grammar of a filter criteria, > > <FIELD_CRITERIA> = {<FIELD_CRITERIA>[,<FIELD_CRITERIA>*]} > <FIELD_CRITERIA> = {<Field_Name> , <Field_Value> > [,Field_Comparison][,Logical_Operator_With_Next_Criteria]} > > wdyt? > > Saminda > > > > On Thu, Mar 20, 2014 at 8:27 PM, Schwartz, Terri <[email protected]> wrote: > Hi Saminda, > > I'm working on a similar issue in the cipres rest api. I'm curious about the > syntax for filter criteria and how general purpose it will be. Is there an > implied 'and' between the individual criterion? Can more elaborate boolean > expressions be used? > > Terri > From: Saminda Wijeratne [[email protected]] > Sent: Thursday, March 20, 2014 2:40 PM > To: dev > Subject: Re: Retrieving Experiment Summaries > > In an offline discussion with Chathuri, we came up with a simple way for > gateway developers to specify retrieving a filtered set of experiment data > based on the requirements of the gateway user. > > eg: > > SearchQuery query = > new SearchQuery({Experiment.Name, Experiment.Status}, > {{Experiment.Owner,"bob"},{Experiment.Project,"manhattan"}{Experiment.Created,"03-19-2014",">"}) > List<Experiment> experiments = thriftAPI.getExperiments(query); > > Sample syntax > sq = new SearchQuery(<list of fields that needs to be filled>, <list of > filter criteria for the data>) > > Further more the SearchQuery will have the capability to specify paging (eg; > experiments from 11 to 20). > > wdyt? > > Saminda > > > > On Tue, Mar 18, 2014 at 3:04 PM, Lahiru Gunathilake <[email protected]> wrote: > On Tue, Mar 18, 2014 at 2:55 PM, Saminda Wijeratne <[email protected]> > wrote: > For performance issues a gateway should only request a subset of data of an > experiment from Airavata server to compile a summary view of the experiment > to the scientist. Based on my current experience I feel the following data is > required to compile a general summary. > > - Exp ID/Name > - Status > - Project > - Owner/Creation time > +1, We can show minimum data and give detailed view on-demand. But I think we > need to support experiment search based on some criteria and develop an index > for each search criteria, because if I ran jobs for 6 months and I would > never want to get all my experiments, even thought we make it super fast will > minimum data. > > ex: I want to search the experiments I ran last week, or with some text base > search. > > We can use the above solution Saminda suggested in searching too. > > Lahiru > > We have seeing a direct relationship between the number of experiment data > records and the turnaround time. Thus we may need some paging when requesting > the experiment data. > > wdyt? Your thoughts are welcome. > > (Using JIRA [1] to track the status of this task) > > A detailed discussion on the topic is on the Architecture mailing list [2]. > > Regards, > Saminda > > 1. https://issues.apache.org/jira/browse/AIRAVATA-995 > 2. http://www.mail-archive.com/[email protected]/msg00080.html > > > > -- > System Analyst Programmer > PTI Lab > Indiana University > >
