Re: [lustre-discuss] Data migration software

2023-03-22 Thread Stephane Thiell via lustre-discuss
Hi Anna, Weโ€™re about to deploy Lustre/HSM with Phobos for a new large research data archival system at Stanford (200PB). https://github.com/phobos-storage Phobos is open source and a Lustre copytool is available. Archiving policies can be set up via Robinhood like with other HSMs. Robinhood

[lustre-discuss] Data migration software

2023-03-22 Thread Anna Fuchs via lustre-discuss
Dear all, if you have a large Lustre storage and a large tape archive and maybe even additionally some in-house cloud storage, which software do you use for more or less automatic data migration, that has good scaling? Ideally it somehow supports Lustre project quota and more important a

Re: [lustre-discuss] Lustre project quotas and project IDs

2023-03-22 Thread Andreas Dilger via lustre-discuss
Of course my preference would be a contribution to improving the name-projid mapping in the "lfs project" command under LU-13335 so this would also help other Lustre users manage their project IDs. One proposal I had in LU-13335 that I would welcome feedback on was if a name or projid did not

[lustre-discuss] Improving the performance of Lustre small files, Maybe aggregation small files

2023-03-22 Thread ็Ž‹็ƒๆ–Œ via lustre-discuss
Hi~ Regarding improving the performance of Lustre small files, are there any aggregation schemes for small files? If so, what schemes can be used? ---Shuobin___ lustre-discuss mailing list lustre-discuss@lists.lustre.org