That would be great as I was hoping to finally be able to run my tasks to get actual results… it’s been 1 month now developing the OpenMOLE workflow :(I’ll be happy to test it in our environment. I have access to our lab dedicated SLURM cluster and the department HTCondor setup. I could also try it on our college HPC which uses SGE and shared storage.I also agree that these options should be part of the environment specification.
Great !
OpenMOLE env works by copying file to storages. In the general case the storage is not shared between the submission machine and the execution machines. In the case of a cluster OpenMOLE copy everything on the shared FS by using ssh transfer to the master node (entry point of the cluster) so it is accessible to all the computing nodes. In the particular case where the submission machine shares it's FS with the computing node I intend to substitute copy operations by simlink creations, in order for this particular case to be handled by the generic submission code of OpenMOLE.I basically agree with you for the file in ~/.openmole: file are transfered to the node through the shared FS. So it has to be copied here. What could be optimized, is the temporary dir location of execution for task. It is also created in this folder and therefore on the sharded FS, which is not actually requiered. This workdir could be optionnaly relocated somewhere using an environment parameter.Not sure if I follow this solution outline, but I’m sure you have a better idea of how things are working right now and need to be modified. Why do files have to be copied to ~/.openmole when the original input files to the workflow (exploration SelectFileDomain), is already located in a shared FS ?That the location of the local and remote temporary directory location can be configured via environment variable would solve the second issue of where temporary files such as wrapper scripts and remote resources are located.The first issue is how to deal with input and output files of tasks which are located on a shared FS already and thus should not require a copy to the temporary directories.
smime.p7s
Description: Signature cryptographique S/MIME
_______________________________________________ OpenMOLE-users mailing list [email protected] http://fedex.iscpif.fr/mailman/listinfo/openmole-users
