ulimit:
time(seconds)unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes)8192
coredump(blocks) 0
memory(kbytes) unlimited
locked memory(kbytes) 64
process 64098
nofiles 65536
vmemory(kbytes) unlimited
locksunlimited
Rsync call at the end, so i = 1600
modified test script:
---8<---
import os
import subprocess
#import psutil
F = 'test'
OPENS = 1600
cmd = [
#'gdb', '--args',
#'strace', '-f', '-olog',
'./rsync',
'-aviH',
'/etc/passwd',
'/tmp/passwd'
]
for i in xrange(OPENS):
fd = os.open(F, os.O_WRONLY | os.O_CREAT)
os.system('ulimit -a')
#proc = psutil.Process()
#print len(proc.open_files())
subprocess.check_call(cmd)
---8<---
strace:
https://pastebin.com/Xu9Pq4R9
If system limit excided it may fall with EMFILE, I think
2017-04-16 9:43 GMT+03:00 <devz...@web.de>:
> What's the value of "i" when this happens and what are the system ulimit
> values for the user running that?
>
> Roland
>
>
>
>> Gesendet: Freitag, 14. April 2017 um 19:22 Uhr
>> Von: "Boris Savelev via rsync" <rsync@lists.samba.org>
>> An: rsync@lists.samba.org
>> Betreff: rsync buffer overflow detected
>>
>> Hello!
>>
>> I use rsync from python on my Debian Jessie amd64 and get this error:
>> *** buffer overflow detected ***: /rsync terminated
>> === Backtrace: =
>> /lib/x86_64-linux-gnu/libc.so.6(+0x731af)[0x778971af]
>> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7791caa7]
>> /lib/x86_64-linux-gnu/libc.so.6(+0xf6cc0)[0x7791acc0]
>> /lib/x86_64-linux-gnu/libc.so.6(+0xf8a17)[0x7791ca17]
>> /rsync(+0x30c78)[0x55584c78]
>> /rsync(+0x31cfe)[0x55585cfe]
>> /rsync(+0x31ef6)[0x55585ef6]
>> /rsync(+0x336ed)[0x555876ed]
>> /rsync(+0x22417)[0x55576417]
>> /rsync(+0x2395e)[0x5557795e]
>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x77845b45]
>> /rsync(+0x7f89)[0xbf89]
>>
>> I guess that problem is about too many open fds
>> STR for this is a small script on python:
>> import os
>> import subprocess
>>
>> F = 'test'
>> OPENS = 1600
>>
>> cmd = [
>> #'gdb', '--args',
>> './rsync',
>> '-aviH',
>> '/etc/passwd',
>> '/tmp/passwd'
>> ]
>>
>> for i in xrange(OPENS):
>> fd = os.open(F, os.O_WRONLY | os.O_CREAT)
>> print(cmd)
>> subprocess.check_call(cmd)
>>
>> I rebuild rsync-3.1.1 from Debian source with debug and -O1 and get bt from
>> gdb:
>> (gdb) bt
>> #0 0x77859067 in __GI_raise (sig=sig@entry=6) at
>> ../nptl/sysdeps/unix/sysv/linux/raise.c:56
>> #1 0x7785a448 in __GI_abort () at abort.c:89
>> #2 0x778971b4 in __libc_message (do_abort=do_abort@entry=2,
>> fmt=fmt@entry=0x77989cb3 "*** %s ***: %s terminated\n")
>> at ../sysdeps/posix/libc_fatal.c:175
>> #3 0x7791caa7 in __GI___fortify_fail
>> (msg=msg@entry=0x77989c4a "buffer overflow detected") at
>> fortify_fail.c:31
>> #4 0x7791acc0 in __GI___chk_fail () at chk_fail.c:28
>> #5 0x7791ca17 in __fdelt_chk (d=d@entry=1606) at fdelt_chk.c:25
>> #6 0x55584c78 in safe_read (fd=fd@entry=1606,
>> buf=buf@entry=0x7fffa810 "\037", len=len@entry=4) at io.c:245
>> #7 0x55585cfe in read_buf (f=f@entry=1606,
>> buf=buf@entry=0x7fffa810 "\037", len=len@entry=4) at io.c:1815
>> #8 0x55585ef6 in read_int (f=f@entry=1606) at io.c:1711
>> #9 0x555876ed in setup_protocol (f_out=1605, f_in=1606) at
>> compat.c:158
>> #10 0x55576417 in client_run (f_in=1606, f_out=1605,
>> pid=24793, argc=1, argv=0x557d5240) at main.c:1128
>> #11 0x5557795e in start_client (argv=0x557d5240, argc=1)
>> at main.c:1423
>> #12 main (argc=2, argv=0x557d5240) at main.c:1651
>>
>> It looks like a bug, but I'm not sure)
>>
>> --
>> Boris
>>
>> --
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>>
--
Boris
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
