[lustre-discuss] Call for Submissions IO500 ISC23

2023-03-30 Thread IO500 Committee
Stabilization Period: Monday, April 3rd - Friday, April 14th, 2023 Submission Deadline: Tuesday, May 16st, 2023 AoE The IO500 is now accepting and encouraging submissions for the upcoming 12th semi-annual IO500 list, in conjunction with ISC23. Once again, we are also accepting submissions to th

Re: [lustre-discuss] [EXTERNAL] Re: Joining files

2023-03-30 Thread Vicker, Darby J. (JSC-EG111)[Jacobs Technology, Inc.] via lustre-discuss
> Instead, my recommendation would be to use an ext4 filesystem image to hold > the many small files (during create, if from a single client, or aggregated > after they are created). Later, this filesystem image could be mounted > read-only on multiple clients for access. Also, the whole image f

Re: [lustre-discuss] Joining files

2023-03-30 Thread Andreas Dilger via lustre-discuss
Based on your use case, I don't think file join will be a suitable solution. There is a limit on the number of files that can be joined (about 2000) and this would make for an unusual file format (something like a tar file, but would need special tools to access). It would also be very Lustre-sp

Re: [lustre-discuss] Joining files

2023-03-30 Thread Patrick Farrell via lustre-discuss
Sven, It would not be possible to implement purely in userspace today, no. The layout manipulation primitives provided by Lustre don't currently include what would be needed for a join. We would to provide some sort of new layout primitive(s) and there would be some userspace work to hook the

Re: [lustre-discuss] Lustre HA

2023-03-30 Thread Dominika Wanat
Hi, we have the HA stack based on Pacemaker/Corosync with ZFS and Lustre resource agents on production, with setup provided by Laura and ZFS multi-mount protection. The main advantage is that the resources are moved automatically when there is a problem with the server or Lustre RPC. The main dis