Re: Aw: Re: rsync 3.0.9 segmentation fault
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Using --no-inc-recursive would increase memory usage not reduce it. That option forces rsync to keep the entire tree in memory. On 03/19/2015 03:10 PM, devz...@web.de wrote: Hi Aron, i hope it`s ok for you if i bring this back on-list. Your issue and the way or possible fix to resolve it may be interesting for others too (that may include future searches etc) so with 3.1.1 we are a step further i don`t really have a clue what`s happening here but my next step would be taking a closer look on how the memory usage of rsync on the client and server grows. you could log it like this: while true;do ps -eo vsz,rss,sz,rsync|grep cron;sleep 10;done logfile does it grow continuously? does the oom situation reproducibly happen at a certain size ? what`s the client and server platform? how many files? (- https://rsync.samba.org/FAQ.html#5 ! ) regards roland *Gesendet:* Donnerstag, 19. März 2015 um 12:24 Uhr *Von:* Aron Rotteveel rotteveel.a...@gmail.com *An:* devz...@web.de *Betreff:* Re: rsync 3.0.9 segmentation fault In addition to my last message: * Client (sender) has 16GB's or RAM, of which only 6.5GB is used during peak. * I tried using --no-inc-recursive, but it does not solve the issue. What currrently is puzzling me is the question of why I am receiving these errors when my server seems to have plenty of memory to spare. -- Best regards / Met vriendelijke groet, Aron Rotteveel 2015-03-19 11:52 GMT+01:00 Aron Rotteveel rotteveel.a...@gmail.com: Hi Roland, I just upgrade both the client and host to 3.1.1 and seem to memory related issues now: ERROR: out of memory in make_file [sender] rsync error: error allocating core memory buffers (code 22) at util2.c(102) [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c, line=102): about to call exit(22) [Receiver] _exit_cleanup(code=22, file=io.c, line=1633): about to call exit(22) rsnapshot encountered an error! The program was invoked with these options: /usr/bin/rsnapshot -c \ /home/remotebackup/hosts/redacted/rsnapshot.conf sync -- Best regards / Met vriendelijke groet, Aron Rotteveel 2015-03-18 23:43 GMT+01:00 devz...@web.de: Hi, rsync 3.0.9 is quite ancient, more than 3 years old. A lot of bugs have been fixed since then. Is there a chance to update to the latest rsync version and retry with that ? regards Roland *Gesendet:* Dienstag, 17. März 2015 um 11:51 Uhr *Von:* Aron Rotteveel rotteveel.a...@gmail.com *An:* rsync@lists.samba.org *Betreff:* rsync 3.0.9 segmentation fault Hi, I am experiencing segfaults when transferring files via rsync though sudo. Setup: - Backupserver initiates the rsync command with --delete -vvv --no-inc-recursive --numeric-ids --delete-excluded --relative --rsync-path=/home/backupuser/rsync-wrapper.sh - rsync-wrapper.sh (on the client) contains /usr/bin/sudo /usr/bin/rsync $@; - user backupuser has sudo access to the rsync command - Both host and client are running 3.0.9 The transfer starts and some files are actually transferred. Once a certain file is reached (plain PHP file, no special characters or any other peculiarities) it segfaults. rsync host output: [sender] make_file(redacted/libraries/phputf8/mbstring/strlen.php,*,2) rsync: connection unexpectedly closed (51261222 bytes received so far) [Receiver] rsync error: unexplained error (code 139) at io.c(605) [Receiver=3.0.9] [Receiver] _exit_cleanup(code=12, file=io.c, line=605): about to call exit(139) rsnapshot encountered an error! The program was invoked with these options: /usr/bin/rsnapshot -c \ /home/remotebackup/hosts/redacted/rsnapshot.conf sync ERROR: /usr/bin/rsync returned 139 while processing backupuser@redacted:/backup/ Client output when using gdb to debug the coredump: warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff015fd000 Core was generated by `/usr/bin/rsync --server --sender -vvvlogDtprRe.Lsf --numeric-ids . /backup'. Program terminated with signal 11, Segmentation fault. #0 0x0035cda7b441 in memcpy () from /lib64/libc.so.6 Any help would be greatly appreciated. Please let me know if additional info is required to properly debug this issue. -- Best regards / Met vriendelijke groet, Aron Rotteveel -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html - --
Aw: Re: rsync 3.0.9 segmentation fault
Hi Aron, i hope its ok for you if i bring this back on-list. Your issue and the way or possible fix to resolve it may be interesting for others too (that may include future searches etc) so with 3.1.1 we are a step further i dont really have a clue whats happening here but my next step would be taking a closer look on how the memory usage of rsync on the client and server grows. you could log it like this: while true;do ps -eo vsz,rss,sz,rsyncgrep cron;sleep 10;done logfile does it grow continuously? does the oom situation reproducibly happen at a certain size ? whats the client and server platform? how many files? (- https://rsync.samba.org/FAQ.html#5 ! ) regards roland Gesendet:Donnerstag, 19. Mrz 2015 um 12:24 Uhr Von:Aron Rotteveel rotteveel.a...@gmail.com An:devz...@web.de Betreff:Re: rsync 3.0.9 segmentation fault In addition to my last message: Client (sender) has 16GBs or RAM, of which only 6.5GB is used during peak. I tried using --no-inc-recursive, but it does not solve the issue. What currrently is puzzling me is the question of why I am receiving these errors when my server seems to have plenty of memory to spare. -- Best regards / Met vriendelijke groet, Aron Rotteveel 2015-03-19 11:52 GMT+01:00 Aron Rotteveel rotteveel.a...@gmail.com: Hi Roland, I just upgrade both the client and host to 3.1.1 and seem to memory related issues now: ERROR: out of memory in make_file [sender] rsync error: error allocating core memory buffers (code 22) at util2.c(102) [sender=3.1.1] [sender] _exit_cleanup(code=22, file=util2.c, line=102): about to call exit(22) [Receiver] _exit_cleanup(code=22, file=io.c, line=1633): about to call exit(22) rsnapshot encountered an error! The program was invoked with these options: /usr/bin/rsnapshot -c /home/remotebackup/hosts/redacted/rsnapshot.conf sync -- Best regards / Met vriendelijke groet, Aron Rotteveel 2015-03-18 23:43 GMT+01:00 devz...@web.de: Hi, rsync 3.0.9 is quite ancient, more than 3 years old. A lot of bugs have been fixed since then. Is there a chance to update to the latest rsync version and retry with that ? regards Roland Gesendet:Dienstag, 17. Mrz 2015 um 11:51 Uhr Von:Aron Rotteveel rotteveel.a...@gmail.com An:rsync@lists.samba.org Betreff:rsync 3.0.9 segmentation fault Hi, I am experiencing segfaults when transferring files via rsync though sudo. Setup: - Backupserver initiates the rsync command with--delete -vvv --no-inc-recursive --numeric-ids --delete-excluded --relative --rsync-path=/home/backupuser/rsync-wrapper.sh - rsync-wrapper.sh (on the client) contains/usr/bin/sudo /usr/bin/rsync @; - user backupuser has sudo access to the rsync command - Both host and client are running 3.0.9 The transfer starts and some files are actually transferred. Once a certain file is reached (plain PHP file, no special characters or any other peculiarities) it segfaults. rsync host output: [sender] make_file(redacted/libraries/phputf8/mbstring/strlen.php,*,2) rsync: connection unexpectedly closed (51261222 bytes received so far) [Receiver] rsync error: unexplained error (code 139) at io.c(605) [Receiver=3.0.9] [Receiver] _exit_cleanup(code=12, file=io.c, line=605): about to call exit(139) rsnapshot encountered an error! The program was invoked with these options: /usr/bin/rsnapshot -c /home/remotebackup/hosts/redacted/rsnapshot.conf sync ERROR: /usr/bin/rsync returned 139 while processing backupuser@redacted:/backup/ Client output when using gdb to debug the coredump: warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff015fd000 Core was generated by /usr/bin/rsync --server --sender -vvvlogDtprRe.Lsf --numeric-ids . /backup. Program terminated with signal 11, Segmentation fault. #0 0x0035cda7b441 in memcpy () from /lib64/libc.so.6 Any help would be greatly appreciated. Please let me know if additional info is required to properly debug this issue. -- Best regards / Met vriendelijke groet, Aron Rotteveel -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
