Am Mon, 29 Mar 2021 19:30:34 +0300 schrieb Andrei Borzenkov <arvidj...@gmail.com>:
> On 29.03.2021 16:16, Claudius Heine wrote: > > Hi, > > > > I am currently investigating the possibility to use `btrfs-stream` > > files (generated by `btrfs send`) for deploying a image based > > update to systems (probably embedded ones). > > > > One of the issues I encountered here is that btrfs-send does not > > use any diff algorithm on files that have changed from one snapshot > > to the next. > > btrfs send works on block level. It sends blocks that differ between > two snapshots. > > > One way to implement this would be to add some sort of 'patch' > > command to the `btrfs-stream` format. > > > > This would require reading complete content of both snapshots instead > if just computing block diff using metadata. Unless I misunderstand > what you mean. On embedded systems it is common to update complete "firmware" images as opposed to package based partial updates. You often have two root filesystems to be able to always fall back to a working state in case of any sort or error. Take the picture from https://sbabic.github.io/swupdate/overview.html#double-copy and assume that "Application software" is a full blown OS with everything that makes your device. That approach offers great "control" but unfortunately can also lead to great downloads required for an update. The basic idea is to download the binary-diff between the future and the current rootfs only. Given a filesystem supports snapshots, it would be great to "send/receive" them as diffs. Today most people that do such things with other fss script around with xdelta etc. But btrfs is more "integrated", so when considering it for such embedded usecases native support would most likely be better than hacks on top. We have several use-cases in mind with btrfs. - ro-base with rw overlays - binary diff updates against such a ro-base - backup/restore with snapshots of certain subvolumes - factory reset with wiping certain submodules regards, Henning > > Is this something upstream would be interested in? > > > > Lets say we introduce a new `btrfs-send` format, lets call it > > `btrfs-delta-stream`, which could can be created from a > > `btrfs-stream`: > > > > 1. For all `write` commands, check the requirements: > > - Does the file already exists in the old snapshot? > > - Is the file smaller than xMiB (this depends on the diff-algo > > and the available resources) > > 2. If the file fulfills those requirements, replace 'write' command > > with 'patch' command, and calculate the binary delta. Also check > > if the delta is actually smaller than the data of the new file. > > Possible add the used binary diff algo as well as a checksum of the > > 'old' file to the command as well. > > > > This file format can of course be converted back to `btrfs-stream` > > and then applied with `btrfs-receive`. > > > > I would probably start with `bsdiff` for the diff algorithm, but > > maybe we want to be flexible here. > > > > Of course if `btrfs-delta-stream` is implemented in `btrfs-progs` > > then, we can create and apply this format directly. > > > > regards, > > Claudius >
