Re: rsync hangs during --remove-source-files
Wayne Davison wrote: On Fri, Oct 17, 2008 at 03:34:03PM +0800, Thomas Gutzler wrote: I had another rsync hang. I've attached a text file with the stack traces you asked for and some other info. [...] So, the strange thing is that the receiver is waiting around for more files to process, which raises the question of why the generator doesn't know that the files it initiated are all done (or make me wonder if some pipe data didn't get delivered, but that is less likely). Did your transfer report any errors? No, there are no errors in the log file expect the one when I killed the process. I've just switched back to rsync version 2.6.9 - see if that works. Happy to test/debug stuff, you just have to tell me what to do. Tom -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync hangs during --remove-source-files
On Fri, Oct 17, 2008 at 03:34:03PM +0800, Thomas Gutzler wrote: I had another rsync hang. I've attached a text file with the stack traces you asked for and some other info. The generator backtrace shows that it is waiting for the receiver to finish some files before going on. This attempts to sprinkle the source removes more evenly during the transfer instead of having them all get bunched up at the end. So, the strange thing is that the receiver is waiting around for more files to process, which raises the question of why the generator doesn't know that the files it initiated are all done (or make me wonder if some pipe data didn't get delivered, but that is less likely). Did your transfer report any errors? ..wayne.. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync hangs during --remove-source-files
Hi, I had another rsync hang. I've attached a text file with the stack traces you asked for and some other info. There were three rsync processes running at the time. I haven't heard anything from Wayne yet - maybe this generates more interest. Cheers, Tom Matt McCutchen wrote: On Mon, 2008-10-13 at 21:19 +0800, Thomas Gutzler wrote: I'm using the following command to move files from a to b: rsync -a --remove-source-files --ignore-existing /path/to/dir1/ /path/to/dir2 logfile dir2 isn't empty so I only move files from dir1 to dir2 that don't exist already. Every so often rsync stops in the middle and doesn't continue with the result that I have to kill it. Any suggestions how to debug this so that I can find out what's going on? Unfortunately I haven't found a way to reproduce it. Every time I kill and execute the same command again, it works. The next time rsync hangs, before you kill it, attach gdb to each of the three rsync processes (gdb /PATH/TO/rsync PID) and get a stack trace (bt). Wayne should have more suggestions for what information would be helpful for debugging. Matt [EMAIL PROTECTED]:~# ps ax | grep rsync 5249 ?S 0:00 sh -c /usr/local/rsync3/bin/rsync -a --remove-source-files --ignore-existing /home/backup/snapshots/frequent_2008-10-09_18-00-01/ /home/backup/snapshots/daily_2008-10-17_05-00-01 /home/backup/logs/frequent_2008-10-17_12-00-01.log 21 5250 ?S 0:02 /usr/local/rsync3/bin/rsync -a --remove-source-files --ignore-existing /home/backup/snapshots/frequent_2008-10-09_18-00-01/ /home/backup/snapshots/daily_2008-10-17_05-00-01 5251 ?S 0:00 /usr/local/rsync3/bin/rsync -a --remove-source-files --ignore-existing /home/backup/snapshots/frequent_2008-10-09_18-00-01/ /home/backup/snapshots/daily_2008-10-17_05-00-01 5252 ?S 0:02 /usr/local/rsync3/bin/rsync -a --remove-source-files --ignore-existing /home/backup/snapshots/frequent_2008-10-09_18-00-01/ /home/backup/snapshots/daily_2008-10-17_05-00-01 [EMAIL PROTECTED]:~# lsof | grep rsync | grep backup COMMAND PIDUSER FD TYPE DEVICESIZE NODE NAME rsync 5250root cwd DIR 254,04096 15290437 /home/backup/snapshots/frequent_2008-10-09_18-00-01 rsync 5250root1w REG 254,0 1258828 76689435 /home/backup/logs/frequent_2008-10-17_12-00-01.log rsync 5250root2w REG 254,0 1258828 76689435 /home/backup/logs/frequent_2008-10-17_12-00-01.log rsync 5251root cwd DIR 254,04096 15294467 /home/backup/snapshots/daily_2008-10-17_05-00-01 rsync 5251root2w REG 254,0 1258828 76689435 /home/backup/logs/frequent_2008-10-17_12-00-01.log rsync 5252root cwd DIR 254,04096 15294467 /home/backup/snapshots/daily_2008-10-17_05-00-01 rsync 5252root2w REG 254,0 1258828 76689435 /home/backup/logs/frequent_2008-10-17_12-00-01.log [EMAIL PROTECTED]:/home/backup/logs# tail -2 frequent_2008-10-17_12-00-01.log touch_up_dirs: home/io/obel/acuratolo/.winnt/profile/Cookies (20) set modtime of home/io/obel/acuratolo/.winnt/profile/Cookies to (1223546878) Thu Oct 9 18:07:58 2008 [EMAIL PROTECTED]:/home/backup/snapshots# ll -d daily_2008-10-17_05-00-01/home/io/obel/acuratolo/.winnt/profile/Cookies frequent_2008-10-09_18-00-01/home/io/obel/acuratolo/.winnt/profile/Cookies drwx-- 2 1161 1097 4096 2008-10-09 18:07 daily_2008-10-17_05-00-01/home/io/obel/acuratolo/.winnt/profile/Cookies/ drwx-- 2 1161 1097 4096 2008-10-17 12:15 frequent_2008-10-09_18-00-01/home/io/obel/acuratolo/.winnt/profile/Cookies/ [EMAIL PROTECTED]:~# gdb /usr/local/rsync3/bin/rsync 5250 GNU gdb 6.4.90-debian Copyright (C) 2006 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type show copying to see the conditions. There is absolutely no warranty for GDB. Type show warranty for details. This GDB was configured as i486-linux-gnu...Using host libthread_db library /lib/tls/i686/cmov/libthread_db.so.1. Attaching to program: /usr/local/rsync3/bin/rsync, process 5250 Reading symbols from /lib/tls/i686/cmov/libc.so.6...done. Loaded symbols for /lib/tls/i686/cmov/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/tls/i686/cmov/libnss_compat.so.2...done. Loaded symbols for /lib/tls/i686/cmov/libnss_compat.so.2 Reading symbols from /lib/tls/i686/cmov/libnsl.so.1...done. Loaded symbols for /lib/tls/i686/cmov/libnsl.so.1 Reading symbols from /lib/tls/i686/cmov/libnss_nis.so.2...done. Loaded symbols for /lib/tls/i686/cmov/libnss_nis.so.2 Reading symbols from /lib/tls/i686/cmov/libnss_files.so.2...done. Loaded symbols for
rsync hangs during --remove-source-files
Hi, I'm using the following command to move files from a to b: rsync -a --remove-source-files --ignore-existing /path/to/dir1/ /path/to/dir2 logfile dir2 isn't empty so I only move files from dir1 to dir2 that don't exist already. Every so often rsync stops in the middle and doesn't continue with the result that I have to kill it. The last time it happened, rsync had successfully moved 1320 files, there were 2943 files in dir2 and 1101 files in dir1. lsof showed no open files inside dir1 or dir2, the log file was opened 4 times, dir1 itself showed up and dir2 twice. This is on a dual core CPU with rsync version 3.0.4 protocol version 30. The log file shows no errors. Last 4 lines say: renaming dir/res/.svn/text-base/.somefile.svn-base.mDBdUb to dir/res/.svn/text-base/somefile.svn-base sender removed dir/.svn/text-base/somefile.svn-base touch_up_dirs: dir/.svn/prop-base (84) set modtime of dir/res/.svn/prop-base to (1223212308) Sun Oct 5 21:11:48 2008 Any suggestions how to debug this so that I can find out what's going on? Unfortunately I haven't found a way to reproduce it. Every time I kill and execute the same command again, it works. Cheers, Tom -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync hangs during --remove-source-files
On Mon, 2008-10-13 at 21:19 +0800, Thomas Gutzler wrote: I'm using the following command to move files from a to b: rsync -a --remove-source-files --ignore-existing /path/to/dir1/ /path/to/dir2 logfile dir2 isn't empty so I only move files from dir1 to dir2 that don't exist already. Every so often rsync stops in the middle and doesn't continue with the result that I have to kill it. Any suggestions how to debug this so that I can find out what's going on? Unfortunately I haven't found a way to reproduce it. Every time I kill and execute the same command again, it works. The next time rsync hangs, before you kill it, attach gdb to each of the three rsync processes (gdb /PATH/TO/rsync PID) and get a stack trace (bt). Wayne should have more suggestions for what information would be helpful for debugging. Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
