Hi, I just had a closer look at a plugin implementation (libcloud for processing S3 buckets) and some of the plugin interface and core code.
The idea was to simply skip over all directories if the rctime based check indicates that the directory has not been changed since the last backup. So a depth first traversal would only hit changed directories. But I do not see a way to skip the entire directory content without traversing it at least. The incremental jobs indicate the right state for files and directories (FT_NOCHG/FT_DIRNOCHG), but if no information is given at all for former entries, bareos assumes that the file/directory was deleted. So if a plugin skips a directory completely, its content will be marked as deleted and pruned according to the retention setup. If this is correct, it is not possible to skip the traversal of unchanged subdirectories (either in plugins or the fd core itself). Best regards, Burkhard andreas.rogge schrieb am Donnerstag, 3. November 2022 um 17:26:03 UTC+1: > Hi Burkhard, > > Am 02.11.22 um 15:30 schrieb Burkhard Linke: > > 1. Is it possible to run a script within a job and skip the job > > depending on exit code? > Running scripts is only possible with RunScript right now. Using that > you could fail the job, but that's probably not what you want. > > > 2. Is it possible to pass the timestamp of the last job run to such a > > script? > When you run a job it will have a so-called "since-time" which is the > time when the previous job started (when you run a differential job, it > is the time of the full that is based on). > When you configure a RunScript you can use "%s" as a placeholder that > will be replaced by the since-time. > > > 3. Can a skip method be implemented in the FD itself? Check whether a > > directory is using cephfs, check whether recursive stats are available, > > skip if rctime is older than last job run? > That would be doable, but the xattr code is pretty far away from the > code that traverses the directory tree, so it is probably not as > straightforward as it might seem at first. > > > 4. Use a FD plugin instead of extending the FD core code? > That is probably the way to go - at least to get a working proof of > concept running. > We are currently working on a change that will improve plugin > performance a lot (see PR #1297 [1]). With that change, a plugin can now > pass open file descriptors to the core allowing it to achieve > near-native backup performance with a plugin. > > In a plugin you also have much more freedom to adapt to the needs of the > system you're backing up. In case of cephfs you could easily take a > snapshot and then rewrite the paths to remove the /.snap/your-snapshot/ > so restore will just write to the right location. > > Having said all that, I wonder how you want to proceed. > We can probably provide some limited guidance to get you started > implementing a plugin yourself, or I can get you in touch with sales to > discuss funded development for this. > > Best Regards, > Andreas > > [1] https://github.com/bareos/bareos/pull/1297/ > > -- > Andreas Rogge andrea...@bareos.com > Bareos GmbH & Co. KG Phone: +49 221-630693-86 <+49%20221%2063069386> > http://www.bareos.com > > Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646 > Komplementär: Bareos Verwaltungs-GmbH > Geschäftsführer: S. Dühr, M. Außendorf, J. Steffens, Philipp Storz > -- You received this message because you are subscribed to the Google Groups "bareos-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-devel/0b84383f-2a3d-4657-9d92-18ca44d7d998n%40googlegroups.com.
