> The problem as I see it is the "tearing it down" bit, I don't want the jobs > shutting down before the user has had a chance to get the resulting data, but > I suspect if we let users shut them down themselfes a lot of them will sit > around for no reason.
I think it's important to provide the user with a simple tool to synchronize any data file in their workspace (e.g. their home folder in the docker container) with a private S3 / Google Storage bucker / folder. Ideally the sync would happen in the background (e.g. like dropbox does) and the idle detection system would never tear down a container / VM as long as the workspace folder sync has not completed. It's possible to use Apache Libcloud to build a vendor agnostic cloud storage client: https://libcloud.readthedocs.org/en/latest/storage/examples.html#create-a-backup-of-a-directory-and-directly-stream-it-to-cloudfiles However I do not know any ready made equivalent to dropbox that is vendor agnostic. -- Olivier ------------------------------------------------------------------------------ Slashdot TV. Video for Nerds. Stuff that matters. http://tv.slashdot.org/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
