Hi, I recently discovered OpenMOLE and a question arises to me : How do you manage tools that read/write files on a local file system ? We tried to adapt legacy image processing tools to work in a distributed environment (Hadoop/MapReduce with HDFS).Those tools usually follow an internal workflow and save their current state into files rather than in memory. We cut the processing into different map/reduce steps in an Oozie workflow, and from there, we may adopt two strategies:
- The tools are developed in Java and we can adapt the code so as to write the file directly into HDFS instead of the local file system. - Another strategy is to push the local result file from each processing step into HDFS, so as to make it available to the next step in the workflow that might run on another node in the cluster. Regards, Pascal GILLET
_______________________________________________ OpenMOLE-users mailing list [email protected] http://fedex.iscpif.fr/mailman/listinfo/openmole-users
