Hello, I was wondering what ideas are in Lustre's roadmap to address the issues with serialization and management that appear when dealing with large amounts of files in parallel. Clusters are getting more and more cores, with some of them having hundreds of thousands already, file systems, however, continue with the paradigm of individual files, even on massively parallel contexts. Wouldn't it be better for parallel file system like Lustre to somehow bundle files together for a group of tasks? If I recall correctly, there is a proposal to extend POSIX to achieve some of this.
I would love to hear some insight about the topic. Thanks, Alvaro.
_______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
