Re: ERROR: parent determination failed (btrfs send-receive)
On Tue, Sep 19, 2017 at 3:37 PM, Andrei Borzenkovwrote: > 18.09.2017 09:10, Dave пишет: >> I use snap-sync to create and send snapshots. >> >> GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external >> drive >> https://github.com/wesbarnett/snap-sync >> > > Are you trying to backup top-level subvolume? No. I never mount the top-level subvolume. > I just reproduced this > behavior with this tool. The problem is, snapshots > of top-level > subvolume do not have parent UUID (I am not even sure if UUID exists at > all TBH). If you mount any other subvolume, it will work. Well, that's not exactly my issue but it still helps me gain additional understanding. It may help me reproduce my original problem too. Thank you. > > > As I told you in the first reply, showing output of "btrfs su li -qu > /path/to/src" would explain your problem much earlier. > > Actually if snap-sync used "btrfs send -p" instead of "btrfs send -c" it > would work as well, as then no parent search would be needed (and as I > mentioned in another mail both commands are functionally equivalent). > But this becomes really off-topic on this list. As already suggested, > open issue for snap-sync. Thank you for this also. I will simply edit my current copy of the snap-sync script to use "-p" now. That's a simple change I am implement immediately. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ERROR: parent determination failed (btrfs send-receive)
18.09.2017 09:10, Dave пишет: > I use snap-sync to create and send snapshots. > > GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external > drive > https://github.com/wesbarnett/snap-sync > Are you trying to backup top-level subvolume? I just reproduced this behavior with this tool. The problem is, snapshots of top-level subvolume do not have parent UUID (I am not even sure if UUID exists at all TBH). If you mount any other subvolume, it will work. On openSUSE root is always mounted as subvolume (actually, the very first snapshot) which explains why I did not see it before. I.e. mkfs -t btrfs /dev/sdb1 mount /dev/sdb1 /test snapper -c test create-config /test attempt to "snap-sync -c test" will fail second time. But btrfs sub create /test/@ umount /test mount -o subvol=@ /dev/sdb1 /test snapper -c test create-config /test ... will work. As I told you in the first reply, showing output of "btrfs su li -qu /path/to/src" would explain your problem much earlier. Actually if snap-sync used "btrfs send -p" instead of "btrfs send -c" it would work as well, as then no parent search would be needed (and as I mentioned in another mail both commands are functionally equivalent). But this becomes really off-topic on this list. As already suggested, open issue for snap-sync. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ERROR: parent determination failed (btrfs send-receive)
On Mon, Sep 18, 2017 at 12:23 AM, Andrei Borzenkovwrote: > > 18.09.2017 05:31, Dave пишет: > > Sometimes when using btrfs send-receive, I get errors like this: > > > > ERROR: parent determination failed for > > > > When this happens, btrfs send-receive backups fail. And all subsequent > > backups fail too. > > > > The issue seems to stem from the fact that an automated cleanup > > process removes certain earlier subvolumes. (I'm using Snapper.) > > > > I'd like to understand exactly what is happening so that my backups do > > not unexpectedly fail. > > > > You try to send incremental changes but you deleted subvolume to compute > changes against. It is hard to tell more without seeing subvolume list > with uuid/parent uuid. > Thanks to Duncan <1i5t5.dun...@cox.net> I have a bit better understanding of -c and -p now. My backup tool is using only the -c option. (The tool is GitHub - wesbarnett/snap-sync https://github.com/wesbarnett/snap-sync .) No subvolume at the destination had ever been deleted. At the source, a number (about 30 in this case) preceding snapshots (subvolumes) exist, but some others have been cleaned up with Snapper's timeline algorithm. I think I understand that with the -c option, it is **not** strictly necessary that the specified snapshots exist on both the source and destination. It seems I had sufficient subvolumes available for the incremental send to work with the -c option. Therefore, it still isn't clear why I got the error: ERROR: parent determination failed. Further general comments will be helpful. I understand the limits in getting too specific in replies because I cannot give subvolume lists with uuid's. As mentioned, I cannot give that info because I tried to fix the above error by sending a subvolume from the destination back to the target. This resulted in my source volume having a "Received UUID" which wrecked all subsequent backups. I now understand that (for my use cases) a source subvolume for a send-receive should **never** have a Received UUID. (If that is indeed a general rule, it seems btrfs tools should check it. Or possibly something about this the pitfalls of a "Received UUID" in a source could be listed on the BTRFS incremental backup wiki page?) I was previously referred to the FAQ here: https://github.com/digint/btrbk/blob/master/doc/FAQ.md#im-getting-an-error-aborted-received-uuid-is-set But in the end, I decided the safest strategy was to delete all prior backup subvolumes so I could be sure my backups were valid going forward. I am asking these questions now to avoid getting into a situation like this again (hopefully). Any general comments are appreciated. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ERROR: parent determination failed (btrfs send-receive)
18.09.2017 11:45, Graham Cobb пишет: > On 18/09/17 07:10, Dave wrote: >> For my understanding, what are the restrictions on deleting snapshots? >> >> What scenarios can lead to "ERROR: parent determination failed"? > > The man page for btrfs-send is reasonably clear on the requirements > btrfs imposes. If you want to use incremental sends (i.e. the -c or -p > options) then the specified snapshots must exist on both the source and > destination. If you don't have a suitable existing snapshot then don't > use -c or -p and just do a full send. > Well, I do not immediately see why -c must imply incremental send. We want to reduce amount of data that is transferred, so reuse data from existing snapshots, but it is really orthogonal to whether we send full subvolume or just changes since another snapshot. >> I use snap-sync to create and send snapshots. >> >> GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external >> drive >> https://github.com/wesbarnett/snap-sync > > I am not familiar with this tool. Your question should be sent to the > author of the tool, if that is what is deciding what -p and -c options > are being used. > I am not sure how it could come to this error. I looked on more or less default installation of openSUSE here and all snapper snapshots have as parent UUID the subvolume that is mounted as root (by default only one configuration for root subvolume exists). So it is not possible to remove this subvolume, unless some rollback to another snapshot was performed. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ERROR: parent determination failed (btrfs send-receive)
On 18/09/17 07:10, Dave wrote: > For my understanding, what are the restrictions on deleting snapshots? > > What scenarios can lead to "ERROR: parent determination failed"? The man page for btrfs-send is reasonably clear on the requirements btrfs imposes. If you want to use incremental sends (i.e. the -c or -p options) then the specified snapshots must exist on both the source and destination. If you don't have a suitable existing snapshot then don't use -c or -p and just do a full send. > I use snap-sync to create and send snapshots. > > GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external > drive > https://github.com/wesbarnett/snap-sync I am not familiar with this tool. Your question should be sent to the author of the tool, if that is what is deciding what -p and -c options are being used. Personally I use and recommend btrbk. I have never had this issue and the configuration options let me limit the snapshots it saves on both the source and destination disks separately (so I keep fewer on the source than on the backup disk). Graham -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ERROR: parent determination failed (btrfs send-receive)
On Mon, Sep 18, 2017 at 12:23 AM, Andrei Borzenkovwrote: > 18.09.2017 05:31, Dave пишет: >> Sometimes when using btrfs send-receive, I get errors like this: >> >> ERROR: parent determination failed for >> >> When this happens, btrfs send-receive backups fail. And all subsequent >> backups fail too. >> >> The issue seems to stem from the fact that an automated cleanup >> process removes certain earlier subvolumes. (I'm using Snapper.) >> >> I'd like to understand exactly what is happening so that my backups do >> not unexpectedly fail. >> > > You try to send incremental changes but you deleted subvolume to compute > changes against. It is hard to tell more without seeing subvolume list > with uuid/parent uuid. I do not have a current subvolume list to provide UUID's. To ensure integrity of my backups, I was forced to delete all backup snapshots and start over. (After this initial parent determination error, my attempted solution created a new problem related to Received UUID's, so removing all backups was the best solution in the end.) For my understanding, what are the restrictions on deleting snapshots? What scenarios can lead to "ERROR: parent determination failed"? After all, it should not be necessary to retain every snapshot ever made. We have to delete snapshots periodically. In my case, I still retained every snapshot on the target volume. None had ever been deleted (yet). And I also retained around 30 recent snapshots on the source, according to the snapper timeline cleanup config. Yet I still ran into "ERROR: parent determination failed". > >> In my scenario, no parent subvolumes have been deleted from the >> target. Some subvolumes have been deleted from the source, but why >> does that matter? I am able to take a valid snapshot at this time and >> every snapshot ever taken continues to reside at the target backup >> destination (seemingly meaning that a parent subvolume can be found at >> the target). >> >> This issue seems to make btrfs send-receive a very fragile backup >> solution. > > btrfs send/receive is not backup solution - it is low level tool that > does exactly what it is told to do. You may create backup solution that > is using btrfs send/receive to transfer data stream, but then do not > blame tool for incorrect usage. > > To give better advice how to fix your situation you need to describe > your backup solution - how exactly you select/create snapshots. I use snap-sync to create and send snapshots. GitHub - wesbarnett/snap-sync: Use snapper snapshots to backup to external drive https://github.com/wesbarnett/snap-sync > > I hope, instead, there is some knowledge I'm missing, that >> when learned, will make this a robust backup solution. >> >> Thanks >> -- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ERROR: parent determination failed (btrfs send-receive)
18.09.2017 05:31, Dave пишет: > Sometimes when using btrfs send-receive, I get errors like this: > > ERROR: parent determination failed for > > When this happens, btrfs send-receive backups fail. And all subsequent > backups fail too. > > The issue seems to stem from the fact that an automated cleanup > process removes certain earlier subvolumes. (I'm using Snapper.) > > I'd like to understand exactly what is happening so that my backups do > not unexpectedly fail. > You try to send incremental changes but you deleted subvolume to compute changes against. It is hard to tell more without seeing subvolume list with uuid/parent uuid. > In my scenario, no parent subvolumes have been deleted from the > target. Some subvolumes have been deleted from the source, but why > does that matter? I am able to take a valid snapshot at this time and > every snapshot ever taken continues to reside at the target backup > destination (seemingly meaning that a parent subvolume can be found at > the target). > > This issue seems to make btrfs send-receive a very fragile backup > solution. btrfs send/receive is not backup solution - it is low level tool that does exactly what it is told to do. You may create backup solution that is using btrfs send/receive to transfer data stream, but then do not blame tool for incorrect usage. To give better advice how to fix your situation you need to describe your backup solution - how exactly you select/create snapshots. I hope, instead, there is some knowledge I'm missing, that > when learned, will make this a robust backup solution. > > Thanks > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
ERROR: parent determination failed (btrfs send-receive)
Sometimes when using btrfs send-receive, I get errors like this: ERROR: parent determination failed for When this happens, btrfs send-receive backups fail. And all subsequent backups fail too. The issue seems to stem from the fact that an automated cleanup process removes certain earlier subvolumes. (I'm using Snapper.) I'd like to understand exactly what is happening so that my backups do not unexpectedly fail. In my scenario, no parent subvolumes have been deleted from the target. Some subvolumes have been deleted from the source, but why does that matter? I am able to take a valid snapshot at this time and every snapshot ever taken continues to reside at the target backup destination (seemingly meaning that a parent subvolume can be found at the target). This issue seems to make btrfs send-receive a very fragile backup solution. I hope, instead, there is some knowledge I'm missing, that when learned, will make this a robust backup solution. Thanks -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html