On Thu, Feb 09, 2023 at 08:56:24AM +0900, Michael Paquier wrote: > On Wed, Feb 08, 2023 at 02:25:54PM -0800, Nathan Bossart wrote: >> These are all good points. Perhaps there could be a base archiver >> implementation that shell_archive uses (and that other modules could use if >> desired, which might be important for backward compatibility with the >> existing callbacks). But if you want to do something fancier than >> archiving sequentially, you could write your own. > > Which is basically the kind of things you can already achieve with a > background worker and a module of your own?
IMO one of the big pieces that's missing is a way to get the next N files to archive. Right now, you'd have to trawl through archive_status on your own if you wanted to batch/parallelize. I think one advantage of what Robert is suggesting is that we could easily provide a supported way to get the next set of files to archive, and we can asynchronously mark them "done". Otherwise, each module has to implement this. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com
