Hi Andreas,we have no workflow profiling tools for now. That would be very interesting and should definitelly be a feature of the workflow development environment we are building.
My guess is that your short jobs are being slown down either by the overhead of the submission process or by numerous IO they perform simultaneously on the shared file system.
cheers, Romain Le 20/05/2015 15:56, Andreas Schuh a écrit :
Hi, given my OpenMOLE workflow, I notice that the throughput of jobs being executed on the Condor cluster with more than 200 unclaimed machines is very low. For the main parallel task which spawns several hundreds or thousands of jobs which take about 10min each it seems fine. However, much shorter tasks which post-process these results either on the cluster or locally seem to take way longer than possible. One solution might of course be to group jobs using the "by" keyword of the OpenMOLE DSL. I would be interested to identify on what OpenMOLE spends actually the most time or whether it wastes lots of time being idle, waiting for some event and which events those are. Is it possible to monitor the job queue and to obtain reports on why certain jobs in the queue are not being submitted to the cluster yet ? Cheers, Andreas _______________________________________________ OpenMOLE-users mailing list OpenMOLE-users@iscpif.fr http://fedex.iscpif.fr/mailman/listinfo/openmole-users
smime.p7s
Description: Signature cryptographique S/MIME
_______________________________________________ OpenMOLE-users mailing list OpenMOLE-users@iscpif.fr http://fedex.iscpif.fr/mailman/listinfo/openmole-users
