FYI - an email exchange I had with Michel Dumontier about Taverna and RESTful services. Feedback, comments and further discussions welcome!
---------- Forwarded message ---------- From: Stian Soiland-Reyes <[email protected]> Date: Sat, May 9, 2009 at 22:23 Subject: Re: [Fwd: mygrid] To: Michel Dumontier <[email protected]> Cc: Alexandra Nenadic <[email protected]> On Sat, May 9, 2009 at 20:23, Alexandra Nenadic <[email protected]> wrote: > 1) Does taverna provide an easy interface to query restful services The simplest ones can easily be accessed using the "Fetch web page" local worker. However the URL construction comes down to the user, so it could not be called 'Easy' out of the box. However - you could build a nested workflow per resource type, taking the URL parameters as workflow inputs. You could then provide a collection of these nested workflows on the myExperiment so they could be used as a toolbox. We've been talking about building more customized support for this in Taverna - like some kind of wizard to help you construct the URL and generate a single service that takes customized inputs based on what you indicated were URL parameters. This would still however require a deep knowledge of the RESTful service you are describing - perhaps the description should then be saved somehow so it can be shared with other Taverna users. I know BioCatalogue - http://www.biocatalogue.org/ - have also looked at describing RESTful web services - and they've run into many of the same issues I'm describing in this email - but a joint effort here could most certainly be beneficial. The key to the whole thing is a good selection of example services to start with. Here's an example of using a simple REST service: http://www.myexperiment.org/workflows/528 - this is done just by customizing the "Fetch web page" local worker to access a WABI service at xml.nig.ac.jp Some REST services like the Uniprot one is quite easy to figure out, for instance if you're at http://www.uniprot.org/uniprot/P68250 you can just do instead: http://www.uniprot.org/uniprot/P68250.xml and you get a nice XML document describing P68250 - but as you see hyperlinks are missing from the XML so they'll have to be constructed 'manually' based on guesses or human readable techie documentation, and extracting information from here using say XPath and then constructing the new URLs) is more advanced and probably not something the random bioinformatician would like to spend time on. So it also comes down to how the restful services themselves are - the rules are very lax in the REST world - which is both an advantage and a danger.. > 2) Is there a standard for describing restful services; Unfortunately not - that is part of the problem in supporting RESTful services. Another is that truly RESTful services are not single 'operations' like RPC-like WSDL services - but are just resources linking to other resources - you would also need to build in parsing of the resource representations (and by which schemas would they be described?) and extract the links for the next resources. There are some attempts like WADL and WSDL 2.0 that we have looked at, but we have yet to find any real services actually described using those, so we've not spent much time into building support for these yet. Another thing is that both of these standards or specification formats seem very > 3) Do you know how I can get a grid-enabled application to be deployed > on an open computing grid that would be accessible to taverna? Most installed computing grids are not "open" - but expect payment in some form or another. Starting your own grid would require lots of infrastructure, hardware and support. Unless you are talking about "open" as in that the framework the grid is running on is open source! :-) I know that EBI - http://www.ebi.ac.uk/Tools/ - are providing many useful (bioinformatical) services that have been grid-enabled - and I'm sure that if you have an interesting application it could be worthwhile hearing with them if they would be interesting in hosting it - they already provide various analysis tools at http://www.ebi.ac.uk/Tools/webservices/ which are based on open source software not written by EBI. EBI is also a solid provider with dedicated people and hardware with a good reputation in the community - just search for "EBI" at myExperiment - 66 workflows: http://www.myexperiment.org/workflows/search?query=EBI If you have something that EBI might be interested in hosting, you could contact EBI either at http://www.ebi.ac.uk/support/index.php?query=WebServices or you could even talk directly to Rodrigo Lopez ( http://www.ebi.ac.uk/Information/Staff/person_maint.php?s_person_id=159 ) - he's the Head of External Services. As a programmer I feel your question is a bit vague - a 'grid-enabled application' can be anything - it would have to be 'grid-enabled' for a specific grid infrastructure - which would then say how it is to be deployed. I would start with what grid you are going to deploy your application onto before I 'grid-enable' it. :-) For instance you could have a Globus Toolkit 4-based grid application that you then enabled as a GT4 web service - which normally would work out of the box with Taverna. In general if you get the service to be exposed as a WSDL-based service it should work from Taverna - the question is what you do with security requirements - most grids require certificates and magic - and the question is if you will require the users of your web service to also provide these credentials (something we're currently building support for in Taverna with caGrid-based services) - or the in many cases better solution to hide this from the anonymous user and let the web service itself provide credentials for submitting the underlying grid jobs. You can hear more about our caGrid/Globus enabling work for Taverna by asking on the taverna-hackers list or see http://www.mygrid.org.uk/dev/wiki/display/caGrid/Home and http://cagrid.org/display/knowledgebase/How+to+Create+CaGrid+Workflow+Using+Taverna+2 We have people building support for other grids as well - like gLite. gLite doesn't normally provide web services, but exposes the grid as pure job submission with data access through gridFTP - this obviously requires more in-depth knowledge of the client and typically the user as there's no description of what applications are installed on the grid - or what kind of parameters and input data they take. Ketan Maheshwari is developing the gLite plugin for Taverna - you'll find him on the taverna-hackers list as well. We also have support for using GridSAM from Taverna - this is again quite low-level job submission stuff which goes against quite a lot of typical Taverna workflows - but you could hide this by putting each job submission in a nested workflow - and again provide a toolbox collection of pre-designed nested workflows for your applications. -- Stian Soiland-Reyes, myGrid team School of Computer Science The University of Manchester -- Stian Soiland-Reyes, myGrid team School of Computer Science The University of Manchester ------------------------------------------------------------------------------ Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT is a gathering of tech-side developers & brand creativity professionals. Meet the minds behind Google Creative Lab, Visual Complexity, Processing, & iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian Group, R/GA, & Big Spaceship. http://www.creativitycat.com _______________________________________________ taverna-hackers mailing list [email protected] Web site: http://www.taverna.org.uk Mailing lists: http://www.taverna.org.uk/taverna-mailing-lists/ Developers Guide: http://www.mygrid.org.uk/tools/developer-information
