ulimit: time(seconds) unlimited file(blocks) unlimited data(kbytes) unlimited stack(kbytes) 8192 coredump(blocks) 0 memory(kbytes) unlimited locked memory(kbytes) 64 process 64098 nofiles 65536 vmemory(kbytes) unlimited locks unlimited
Rsync call at the end, so i = 1600 modified test script: ---8<--- import os import subprocess #import psutil F = 'test' OPENS = 1600 cmd = [ #'gdb', '--args', #'strace', '-f', '-olog', './rsync', '-aviH', '/etc/passwd', '/tmp/passwd' ] for i in xrange(OPENS): fd = os.open(F, os.O_WRONLY | os.O_CREAT) os.system('ulimit -a') #proc = psutil.Process() #print len(proc.open_files()) subprocess.check_call(cmd) ---8<--- strace: https://pastebin.com/Xu9Pq4R9 If system limit excided it may fall with EMFILE, I think 2017-04-16 9:43 GMT+03:00 <devz...@web.de>: > What's the value of "i" when this happens and what are the system ulimit > values for the user running that? > > Roland > > > >> Gesendet: Freitag, 14. April 2017 um 19:22 Uhr >> Von: "Boris Savelev via rsync" <rsync@lists.samba.org> >> An: rsync@lists.samba.org >> Betreff: rsync buffer overflow detected >> >> Hello! >> >> I use rsync from python on my Debian Jessie amd64 and get this error: >> *** buffer overflow detected ***: <snip>/rsync terminated >> ======= Backtrace: ========= >> /lib/x86_64-linux-gnu/libc.so.6(+0x731af)[0x7ffff78971af] >> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7ffff791caa7] >> /lib/x86_64-linux-gnu/libc.so.6(+0xf6cc0)[0x7ffff791acc0] >> /lib/x86_64-linux-gnu/libc.so.6(+0xf8a17)[0x7ffff791ca17] >> <snip>/rsync(+0x30c78)[0x555555584c78] >> <snip>/rsync(+0x31cfe)[0x555555585cfe] >> <snip>/rsync(+0x31ef6)[0x555555585ef6] >> <snip>/rsync(+0x336ed)[0x5555555876ed] >> <snip>/rsync(+0x22417)[0x555555576417] >> <snip>/rsync(+0x2395e)[0x55555557795e] >> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7ffff7845b45] >> <snip>/rsync(+0x7f89)[0x55555555bf89] >> >> I guess that problem is about too many open fds >> STR for this is a small script on python: >> import os >> import subprocess >> >> F = 'test' >> OPENS = 1600 >> >> cmd = [ >> #'gdb', '--args', >> './rsync', >> '-aviH', >> '/etc/passwd', >> '/tmp/passwd' >> ] >> >> for i in xrange(OPENS): >> fd = os.open(F, os.O_WRONLY | os.O_CREAT) >> print(cmd) >> subprocess.check_call(cmd) >> >> I rebuild rsync-3.1.1 from Debian source with debug and -O1 and get bt from >> gdb: >> (gdb) bt >> #0 0x00007ffff7859067 in __GI_raise (sig=sig@entry=6) at >> ../nptl/sysdeps/unix/sysv/linux/raise.c:56 >> #1 0x00007ffff785a448 in __GI_abort () at abort.c:89 >> #2 0x00007ffff78971b4 in __libc_message (do_abort=do_abort@entry=2, >> fmt=fmt@entry=0x7ffff7989cb3 "*** %s ***: %s terminated\n") >> at ../sysdeps/posix/libc_fatal.c:175 >> #3 0x00007ffff791caa7 in __GI___fortify_fail >> (msg=msg@entry=0x7ffff7989c4a "buffer overflow detected") at >> fortify_fail.c:31 >> #4 0x00007ffff791acc0 in __GI___chk_fail () at chk_fail.c:28 >> #5 0x00007ffff791ca17 in __fdelt_chk (d=d@entry=1606) at fdelt_chk.c:25 >> #6 0x0000555555584c78 in safe_read (fd=fd@entry=1606, >> buf=buf@entry=0x7fffffffa810 "\037", len=len@entry=4) at io.c:245 >> #7 0x0000555555585cfe in read_buf (f=f@entry=1606, >> buf=buf@entry=0x7fffffffa810 "\037", len=len@entry=4) at io.c:1815 >> #8 0x0000555555585ef6 in read_int (f=f@entry=1606) at io.c:1711 >> #9 0x00005555555876ed in setup_protocol (f_out=1605, f_in=1606) at >> compat.c:158 >> #10 0x0000555555576417 in client_run (f_in=1606, f_out=1605, >> pid=24793, argc=1, argv=0x5555557d5240) at main.c:1128 >> #11 0x000055555557795e in start_client (argv=0x5555557d5240, argc=1) >> at main.c:1423 >> #12 main (argc=2, argv=0x5555557d5240) at main.c:1651 >> >> It looks like a bug, but I'm not sure) >> >> -- >> Boris >> >> -- >> Please use reply-all for most replies to avoid omitting the mailing list. >> To unsubscribe or change options: >> https://lists.samba.org/mailman/listinfo/rsync >> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >> -- Boris -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html