Re: question about the recursive algorithm
> I don't believe the transferring part of rsync will jump around. How about the deleting part? > It will transfer files it deems need it in the order it finds them > which will be 1 dir at a time. Though when it enters a child dir that > doesn't mean it is done with the parent dir. In the sense it might later do other child dirs of its parent dir? -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: question about the recursive algorithm
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I don't believe the transferring part of rsync will jump around. It will transfer files it deems need it in the order it finds them which will be 1 dir at a time. Though when it enters a child dir that doesn't mean it is done with the parent dir. On Thu, 16 Jan 2025, [email protected] wrote: Date: Thu, 16 Jan 2025 15:16:02 + From: [email protected] To: Kevin Korb Cc: [email protected] Subject: Re: question about the recursive algorithm Thanks for your message again. I appreciate your answers. I don't mind about the order of the files, and neither really about the order of the directories: I'm interested about whether rsync might transfer some files into a directory, then transfer some files into another which is outside of the first one, and then transfer more files into the first directory again. And with the --del option, which afaik means --delete-during, whether it might delete some files from a directory, then delete some files from another which lies outside of the first dir, and then delete some more files from the first directory again. This behaviour would be at odds in my mind with the word recursion, which in my mind implies walking through the directory tree with some criteria, rather than jumping around the directory tree. And in your message you use the word "directory indexing", which in my mind also suggests "focusing exclusively on such directory", at least as far as only the lines starting by *delete are concerned, or only the lines starting by > are concerned... On Thu, Jan 16 2025, Kevin Korb wrote: There is no guarantee that rsync will do anything in any order since it is going by the filesystem without any sorting. With --delete-during the directory indexing and therefore the deletions are running in parallel with file transfers. This means it is quite common for it to delete things in directories it isn't transferring files in. -BEGIN PGP SIGNATURE- iF0EARECAB0WIQSHERqysePm7S8yuR9UoLWOVtABBwUCZ4kkewAKCRBUoLWOVtAB B7BqAJ9Xa5sz8HRl/YbZCFlcPgZQJv2FegCfXLo+l0kNOjs5aEp4cPAeiis7W/g= =si9a -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: question about the recursive algorithm
Thanks for your message again. I appreciate your answers. I don't mind about the order of the files, and neither really about the order of the directories: I'm interested about whether rsync might transfer some files into a directory, then transfer some files into another which is outside of the first one, and then transfer more files into the first directory again. And with the --del option, which afaik means --delete-during, whether it might delete some files from a directory, then delete some files from another which lies outside of the first dir, and then delete some more files from the first directory again. This behaviour would be at odds in my mind with the word recursion, which in my mind implies walking through the directory tree with some criteria, rather than jumping around the directory tree. And in your message you use the word "directory indexing", which in my mind also suggests "focusing exclusively on such directory", at least as far as only the lines starting by *delete are concerned, or only the lines starting by > are concerned... On Thu, Jan 16 2025, Kevin Korb wrote: > There is no guarantee that rsync will do anything in any order since > it is going by the filesystem without any sorting. With > --delete-during the directory indexing and therefore the deletions are > running in parallel with file transfers. This means it is quite > common for it to delete things in directories it isn't transferring > files in. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: question about the recursive algorithm
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 There is no guarantee that rsync will do anything in any order since it is going by the filesystem without any sorting. With --delete-during the directory indexing and therefore the deletions are running in parallel with file transfers. This means it is quite common for it to delete things in directories it isn't transferring files in. On Thu, 16 Jan 2025, BP25 wrote: Date: Thu, 16 Jan 2025 14:47:27 + From: BP25 To: Kevin Korb Cc: [email protected] Subject: Re: question about the recursive algorithm Thanks Kevin, but I don't understand your message, or at least how it answers my "real question" (last paragraph)... and by the way --delete defaults to --delete-during for current versions of rsync as far as I know... rsync doesn't really give much control over the order it does things in. If you want to control when the deletions happen there is only - --delete-before or --delete-during but both are slower than the default - --delete. -BEGIN PGP SIGNATURE- iF0EARECAB0WIQSHERqysePm7S8yuR9UoLWOVtABBwUCZ4kd5AAKCRBUoLWOVtAB B8lIAJ4m+yB7YWl/DVwvlWo4lRjDPqBe8wCeJlSMsd+5smY1qpKbQqr0/nqg2u0= =5Wst -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: question about the recursive algorithm
Thanks Kevin, but I don't understand your message, or at least how it answers my "real question" (last paragraph)... and by the way --delete defaults to --delete-during for current versions of rsync as far as I know... > rsync doesn't really give much control over the order it does things > in. If you want to control when the deletions happen there is only - > --delete-before or --delete-during but both are slower than the > default - --delete. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: question about the recursive algorithm
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 rsync doesn't really give much control over the order it does things in. If you want to control when the deletions happen there is only - --delete-before or --delete-during but both are slower than the default - --delete. On Thu, 16 Jan 2025, BP via rsync wrote: Date: Thu, 16 Jan 2025 13:53:04 + From: BP via rsync To: [email protected] Subject: question about the recursive algorithm Hello, I always run rsync as follows: "sudo rsync -PaSHAXvi --del DIR1/ DIR2". I would think that whenever I see in the output of this rsync command a few lines of the form A/B/... and then further down in the output again a few lines of the form A/B/... (dots are dirs or files), then every line between such two lines is also of this form. I then discovered that this is incorrect (incidentally, the order of the deleted files here below is not alphabetical?!): f+ A/B/folder1/file1.mkv f+ A/B/folder1/file2.mkv *deleting A/C/file3.tex *deleting A/C/file4.pdf *deleting A/C/file5.txt cd+ A/B/folder2/ f+ A/B/folder2/file6.html Is this expected? Reading the manual (section about inductive recursion) I would think that the only case where rsync jumps unfinished dirs would be the creation of empty dirs. But here's the real question: is my hypothesis correct when we restrict to just lines starting by >, and also to just lines which start by *deleting? In other words, does the sender never jump folder and back, and same the receiver (but combined they might, as shown above)? I would grately appreciate insights. Please, let me know! Thanks, BP -BEGIN PGP SIGNATURE- iF0EARECAB0WIQSHERqysePm7S8yuR9UoLWOVtABBwUCZ4kUigAKCRBUoLWOVtAB B9zRAJ9QzHtrrEFTP9ME/oFg2DmQoK23hgCfYxx+fpg4lLKjIFXsomPXw0dVYM8= =lGQP -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: question about the recursive algorithm
On Thu, 2025-Jan-16, BP via rsync wrote: I always run rsync as follows: "sudo rsync -PaSHAXvi --del DIR1/ DIR2". I would think that whenever I see in the output of this rsync command a few lines of the form A/B/... and then further down in the output again a few lines of the form A/B/... (dots are dirs or files), then every line between such two lines is also of this form. I then discovered that this is incorrect (incidentally, the order of the deleted files here below is not alphabetical?!): f+ A/B/folder1/file1.mkv f+ A/B/folder1/file2.mkv *deleting A/C/file3.tex *deleting A/C/file4.pdf *deleting A/C/file5.txt cd+ A/B/folder2/ f+ A/B/folder2/file6.html Is this expected? Reading the manual (section about inductive recursion) I would think that the only case where rsync jumps unfinished dirs would be the creation of empty dirs. But here's the real question: is my hypothesis correct when we restrict to just lines starting by >, and also to just lines which start by *deleting? In other words, does the sender never jump folder and back, and same the receiver (but combined they might, as shown above)? I would grately appreciate insights. Please, let me know! Please see ... https://bugzilla.samba.org/show_bug.cgi?id=6741 Marc. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
