Le 01/05/2015 18:55, Andreas Schuh a écrit :
FYI I just refreshed my memory of our college HPC cluster (it’s actually using PBS, not SGE as mentioned before).From their intro document, the following information may be useful while revising the OpenMOLE storage handling: On the HPC system, there are two file stores available to the user: HOME and WORK . HOME has a relatively small quota of 10GB and is intended for storing binaries, source and modest amounts of data. It should not be written to directly by jobs. WORK is a larger area which is intended for staging files between jobs and for long term data These areas should be referred to using the environment variables $HOME and $WORK as their absolute locations are subject to change. Additionally,$TMPDIR. Jobs requiring scratch space at run time should write to $TMPDIR.On 1 May 2015, at 11:57, Andreas Schuh <[email protected]> wrote:On 1 May 2015, at 11:49, Romain Reuillon <[email protected]> wrote:That would be great as I was hoping to finally be able to run my tasks to get actual results… it’s been 1 month now developing the OpenMOLE workflow :( I’ll be happy to test it in our environment. I have access to our lab dedicated SLURM cluster and the department HTCondor setup. I could also try it on our college HPC which uses SGE and shared storage. I also agree that these options should be part of the environment specification.Great !I basically agree with you for the file in ~/.openmole: file are transfered to the node through the shared FS. So it has to be copied here. What could be optimized, is the temporary dir location of execution for task. It is also created in this folder and therefore on the sharded FS, which is not actually requiered. This workdir could be optionnaly relocated somewhere using an environment parameter.Not sure if I follow this solution outline, but I’m sure you have a better idea of how things are working right now and need to be modified. Why do files have to be copied to ~/.openmole when the original input files to the workflow (exploration SelectFileDomain), is already located in a shared FS ? That the location of the local and remote temporary directory location can be configured via environment variable would solve the second issue of where temporary files such as wrapper scripts and remote resources are located. The first issue is how to deal with input and output files of tasks which are located on a shared FS already and thus should not require a copy to the temporary directories.OpenMOLE env works by copying file to storages. In the general case the storage is not shared between the submission machine and the execution machines. In the case of a cluster OpenMOLE copy everything on the shared FS by using ssh transfer to the master node (entry point of the cluster) so it is accessible to all the computing nodes. In the particular case where the submission machine shares it's FS with the computing node I intend to substitute copy operations by simlink creations, in order for this particular case to be handled by the generic submission code of OpenMOLE.Ok, got it, and sounds like a good solution. So the optional symbolic links (“link” option of “addInputFile” and “addResource”) from the temporary directory/workingDir of each individual task are pointing to the storage on the master node of the execution machines. That is why I encounter currently an unexpected copy of my files. When the storage used by the execution machines themselves, however, uses symbolic links to the storage of the submission machine (as all machines share the same FS), no files are actually copied. What would have been when I had executed the OpenMOLE console on the master node of the environment ? Would then OpenMOLE already know that submission machine and execution machine are actually identical and thus inherently share the same storage ?
smime.p7s
Description: Signature cryptographique S/MIME
_______________________________________________ OpenMOLE-users mailing list [email protected] http://fedex.iscpif.fr/mailman/listinfo/openmole-users
