Hi Greg,
I only had time to skim through it. However, one thing I find is missing
in the Data Management part is something like this :
- If you have tens of thousands of files (or more), you are probably
doing it wrong. Look for archiving those files in chunks, or combining
them in some way.
On a cluster, having tens of thousands of files will be a killer, but
even on a personnal computer, it will make your data management a
nightmare.
I have seen users who think that having 2.5M files in a directory is "ok"...
Thanks,
Maxime
Le 2015-11-17 06:10, Greg Wilson a écrit :
Hi everyone,
I've started drafting a short guide to "good enough" practices in
scientific computing, and would be grateful for your input. You can
view the outline at
http://swcarpentry.github.io/good-enough-practices-in-scientific-computing/,
and leave comments at
https://github.com/swcarpentry/good-enough-practices-in-scientific-computing
(either as issues or pull requests - if you don't have permission,
just email me and I'll post it).
Thanks,
Greg
_______________________________________________
Discuss mailing list
[email protected]
http://lists.software-carpentry.org/mailman/listinfo/discuss_lists.software-carpentry.org
