Great initiation Saminda, this is a key usability topic and especially when you refresh people’s past lives you hit it right on :)
Firstly lets refer to Airavata Stakeholders diagram [1], I assume we are talking about Gateway Developers who will program against the Airavata Client SDK. Just to confirm, I do not think the concern is if Generic Software Frameworks will meet the diverse goals of science gateways right? With so many use cases applied to Airavata I hope its evident that it can be infact be done. Eran’s summary of 80% of classic use cases and extensibility to meet the 20% corner cases is a good one. Infact, may be we only should say 10% of advanced use cases and Airavata cannot (not worth the additional complexity) meet the last 10% of corner cases. If the discussion is about the best practices to document the use of API (not just API documents, but good use case documents), then here are couple of approaches to consider. To address a similar challenging goals for scientific workflows (a first cousin of gateways) we first identified various usage paradigms [2], then gathered vignettes from community [3] to illustrate these paradigms for real-world use cases. In a different world, we tried to approach this problem writing a online cookbook (trying not to have them outdated for long) where each recipes will describe an approach to build a gateways. So a new gateways comes on, they look at these recipes and find the closest matching one and mimic it. So based on these, what I am proposing for Airavata is: * Lets define various known usage paradigms, * Solicit vignettes from community * Develop and maintain a Airavata Cookbook, with each recipe pointing to sample code and will have a corresponding integration test to make sure the samples work as the software evolves through releases. Comments? Suresh [1] - http://airavata.apache.org/architecture/airavata-stakeholders.html [2] - https://sites.google.com/site/earthcubeworkflow/workflow-background/workflow-use-paradigms [3] - https://sites.google.com/site/earthcubeworkflow/workflow-vignettes [4] - http://www.isi.edu/~gil/EarthCube/EC-WorkflowsRoadmap-September2012.pdf [5] - https://www.xsede.org/web/gateways/gateways-cookbook [6] - https://www.xsede.org/web/gateways/gateways-cookbook/-/wikid/f5zI/Cookbook/CIPRES+Science+Gateway+Recipe On Jan 31, 2014, at 3:00 AM, Saminda Wijeratne <[email protected]> wrote: > On Thu, Jan 30, 2014 at 9:34 PM, Amila Jayasekara > <[email protected]>wrote: > >> We definitely need sample code which demonstrate the use of Airavata API. >> But I am not sure whether to call them patterns or API documentation. >> > Its definitely not API documentation. We so far have solve several gateway > problems using Airavata and and each is different on its own although the > ultimate goal was the same. I just think that it would be very useful to > have these variations documented. I used the word "pattern" because even > with different variations there will be other gateways which may be > following the same variation. Thus the word "pattern". > >> >> We do not need to create a "very" generic API but the API should be >> intuitive enough for a person with high-level picture in mind to understand >> and guess which functions to call to get something done. >> > In my experience these days most developers google for > examples/solutions/algorithms (atleast to get an idea how to do something) > rather than go through API documentations and designing and implementing > from scratch. And the Airavata API (Thrift version or otherwise) inherits a > context of usage which pops the questions like "when to use what and how". > There is so many features just for an Airavata experiment launch which we > have implemented because of user requirements which we cannot expect users > to be intuitive enough to figure-out the importance of using them when, why > and how. > > >> Thanks >> Amila >> >> >> On Thu, Jan 30, 2014 at 11:54 PM, Saminda Wijeratne <[email protected] >>> wrote: >> >>> I'm thinking of the reasons why we are having trouble getting a proper >> API >>> to work smoothly is that, >>> >>> >>> 1. We are trying too hard to generalize when its probably not possible >>> to do >>> 2. When we keep trying to see the big picture we miss the little >> things >>> in the gateways that makes the usage of the API easy or hard for the >>> developer. >>> >>> eg: Not all gateways look at "Experiment Launch" use case as passing an >>> application name and its input to launch the experiment. (I think we've >>> came across many variations for this use case) >>> >>> - the inputs could be in different form (could be actual files) or >> just >>> optional >>> - application name could be just a reference to a parameter of a >> global >>> application >>> - part of the launch process may have runtime dependencies which >> gateway >>> requires to participate >>> - the gateway might want more control over the launch configurations >> at >>> runtime. >>> - ... >>> >>> I think devs see it as a mismatch of the API for their gateway which >>> discourages the effort to use the API. >>> >>> IMO part of our Thrift API effort should be to document these with the >>> solutions similar to Java Patterns. >>> >>> In short, instead of trying to come up with a general API to make >> everyone >>> happy we provide solution patterns for a gateway use case which will have >>> multiple variations. (solution pattern would be use of the API + >> supporting >>> code which handles the variation) >>> >>> wdyt? >>> >>> >>> Regards, >>> >>> Saminda >>> >>
