On Mon, Jun 4, 2012 at 2:39 PM, Alex Lyakas
<alex.bolshoy.bt...@gmail.com> wrote:
> Hi Jan, Alex,
>
> I have seen some discussions about btrfs send/receive functionality
> being developed by you. I have also been interested in this. I spent
> some time coding a prototype doing something like Alex described in
> http://www.spinics.net/lists/linux-btrfs/msg16175.html, i.e., walking
> over FS tree and pulling those items that have transid/generation
> larger than a particular value. I realized though, that there are many
> issues with that approach, and also probably there are many issues I
> am not aware of. Some of the issues I realized:
Well, you are, like me in the beginning, on a wrong track ;) Using the
transid only is not the way send/receive will be implemented when it's
done.
>
> # How does one track changes in generic INODE_ITEM properties, like
> "mode" or "uid/gid"? Whenever such property gets changed, INODE_ITEM
> gets stamped with a new transid, but do we need to compare it with the
> previous version on the receive side to realize what has changed?
> # File size - is it required, again, to compare vs previous size, to
> realize file truncation? (file grow perhaps can be realized via new
> EXTENT_DATAs)
> # What should be done if INODE_ITEM::flags change (e.g., inode gets
> nodatacow/nodatasum flags set). What should be done at receive side?
> # How does one track deletion of INODE_ITEMs? Or, deletion and
> re-creation of a INODE_ITEM with the same inode number? (I saw that
> inode_cache mount option allows to re-use inode numbers, so I think it
> can happen.) Does this mean that on receive side, it is required to
> compare contents of each directory vs previous version?
> # What should be done with INODE_ITEMs like block/char device, FIFO or a
> socket?
> # XATTR_ITEMs: although they have a transid stamp, again, need to
> track deletion/re-creation of them. Again by comparing?
> # INODE_REFs: these seem most tough to me, because they don't have
> transid stamps. How such scenario can be handled: an INODE_ITEM had
> two INODE_REFs with names N1 and N2. But now on the send side, both
> those INODE_REFs were deleted and INODE_REFs N3 and N4 were created.
> Does that mean we need to always compare all INODE_REFs for each
> INODE_ITEM, or we perhaps can use DIR_ITEMs/DIR_INDEXs of parent
> INODE_ITEM to detect changes in INODE_REFs?
>
> All in all, it looks like the approach of navigating the FS tree and
> trying to *understand* specifically which modifications were
> performed, is quite error-prone. And I am sure there are modifications
> I am not aware about.
The problem with the transid only approach is, that there is no way
to find out "what" has changed in an inode. You only know which inode
has changed. You could probably determine which extents have
changed, but this is unreliable as you've already read in my older mail.
There is also absolutely no way to detect deleted/moved files/dirs.
When send/receive is released and working, I may try to implement
a mode that only relies on the transid, but this has low priority for me
and also needs some changes to other parts of btrfs. If I implement
that, it would however still be unable to detect what has changed and
would also miss deleted/moved dirs. You could compare it to rsync
without the --delete option (which I use frequently to transfer VM
images).
>
> I was wondering, what state your work is in? Is it possible to look at
> some code or prototype, to understand what approach have you taken, or
> perhaps an overall description of the approach?
Currently, most things work as expected. But, the code is not in a state
to be released. Jan, Arne, David and Stefan are currently reviewing my
code and I have a lot of TODO's due to the suggestions they all made.
>
> Jan, I saw that you provided some new code for backref resolving. Can
> you give a hint of how is that related to the send/receive
> functionality?
It is not only related to the send/receive code. It is currently mainly related
to the upcoming qgroups patches that Jan is preparing. It may also be
related to other parts of btrfs (as far as I know, scrub is also using the
backref resolving code). His patches are however a requirement for
send/receive to work properly when I release my first patches).
>
> Thanks,
> Alex.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
