On 06/10/2010 12:19, Richard Holland wrote: Hello,
> On 6 Oct 2010, at 12:52, Donal K. Fellows wrote: > >> On 06/10/2010 08:27, Richard Holland wrote: [snip] > Agreed. I wouldn't necessarily suggest automatically defining which > tasks should be parallelised. But the ability to be able to > right-click on one in the interface and say 'parallise this because I > know it is slow' would be super. You can do this (sort of) by setting the number of invocations of the service that can be called in parallel. This is done by r-click -> Configure running -> Parallel jobs. That though only works sensibly for external services e.g. WSDL. For services that run locally e.g. beanshells, then there is no way currently to parallelize their invocations. Except... you can use the external tool (also known as UseCase) activity that was developed as part of the KnowArc project. In that you can call external tools that are on remote machines or locally. You can specify a set of machines and the external tool activity will determine which machine to run an invocation on. The external tool mechanism has also been used to run invocations on instances in a cloud. Warning - I may be wrong on the details of this: The external tool definitions currently relate to the tools present on a version of Debian. Instances of that Debian are created in the cloud. The Taverna workflow contains external tool activities that represent runs of those tools. When the workflow is executed, the external tool activity finds out what instances of the Debian are in the cloud and runs the tool on whichever instance is "free". The activity is clever as it does things like try to ensure that minimal data is transferred between Debian instances. We've been talking with Hajo Krabbenhoft and Steffen Moeller (who developed the external tool activity) as to how it can be extended. One thing I am keen on is to make it much easier for users to specify how to invoke their own tools (even if they are only available on the local machine). There is currently a web interface for specifying the tools, but I think it could quite easily be done as configuration of an external tool template. > As would feedback of processing > times so that it could give hints as to which ones you might like to. You do get processing time information in the workflow report view of the run. > You could have an API call for each task indicating whether it is > permitted to be parallel or not - the default would be yes, but the > web service one would be no, and anything marked no would not be able > to be selected as parallelisable (sp?) in the interface (hence > disallowing DDOS). That would be very useful. [snip] >>> 7. Have the ability (maybe via myExperiment) to log every >>> execution of a workflow including a reference to the input data, >>> the structure of the workflow at the time of the execution, and a >>> reference to the results, provenance, etc. This is very useful >>> for lab notebook concepts and also for reproducing work at a >>> later date. As Ian said we have produced experimental implementations of this concept. It also relates to work going on in the eLico and SysMo projects. >> I'd rate the probability of this being done as high. >> >>> 8. Taverna server (App4Andy is a great start by the way) to offer >>> the possibility to upload arbitrary workflows via the desktop >>> client and execute them on the server/grid/cloud, rather than >>> choosing from a predefined selection. On uploading it could make >>> an auto-generated but editable web interface for obtaining >>> workflow input, monitoring progress, downloading >>> results/provenance/etc. The workflow could be a one-off, or could >>> be stored there, and kept private, or shared via user/group >>> notions, etc. This is all in Knime already (except the cloud >>> bit). >> >> We can do some of this now (the auto-interface isn't sophisticated, >> but it does exist) and other things - the integration with the >> workbench in particular - are on the technical feature roadmap. >> This *will* be done. Indeed. Alex is working on an interface to do some of this for the NeISS project. > Thanks for the responses - good to know what is on the map and what > is not. The most important of all these things really is number 1 > (parallelisation) as everything else can be worked-around/hacked by > those who are desperate enough. :) > > cheers Richard Alan ------------------------------------------------------------------------------ Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb _______________________________________________ taverna-hackers mailing list [email protected] Web site: http://www.taverna.org.uk Mailing lists: http://www.taverna.org.uk/about/contact-us/ Developers Guide: http://www.taverna.org.uk/developers/
