Re: out of memory errors seen on several AnonCVS servers
On Sun, Mar 03, 2013 at 22:17, Stuart Henderson wrote: On 2013/03/03 17:11, Ted Unangst wrote: On Sun, Mar 03, 2013 at 16:16, Stuart Henderson wrote: Summary of off-list mail exchange: anoncvs.usa consistently has a realloc failure in xenocara/font/misc-misc/18x18ja.bdf, some other servers have a failure the first time checking this file out when it's part of the whole directory, but resuming or checking out individually does work. amd64? Use i386. :) or opencvs. The problems in opencvs are more insidious. You only have to check the file out once, then it works. As far as I know. Haven't done a full x11 checkout on amd64 for a while.
Re: out of memory errors seen on several AnonCVS servers
On 2013/03/04 10:47, Ted Unangst wrote: On Sun, Mar 03, 2013 at 22:17, Stuart Henderson wrote: On 2013/03/03 17:11, Ted Unangst wrote: On Sun, Mar 03, 2013 at 16:16, Stuart Henderson wrote: Summary of off-list mail exchange: anoncvs.usa consistently has a realloc failure in xenocara/font/misc-misc/18x18ja.bdf, some other servers have a failure the first time checking this file out when it's part of the whole directory, but resuming or checking out individually does work. amd64? Use i386. :) or opencvs. The problems in opencvs are more insidious. You only have to check the file out once, then it works. As far as I know. Haven't done a full x11 checkout on amd64 for a while. The client arch and software doesn't make a difference, the problem is on the server side. Problems seen when using opencvs server-side include giving out the wrong file version, and giving a checkout which can't reliably be used against a server running original CVS.
Re: out of memory errors seen on several AnonCVS servers
On Mon, Mar 04, 2013 at 15:55, Stuart Henderson wrote: The client arch and software doesn't make a difference, the problem is on the server side. Problems seen when using opencvs server-side include giving out the wrong file version, and giving a checkout which can't reliably be used against a server running original CVS. hmmm, it's been a long time, but when I ran into this problem (like 2005), the client arch made all the difference. Or maybe back then the servers were all i386, too, and both client and server needed to be 32 bit? i.e., what used to be a client problem is now a server and client problem.
Re: out of memory errors seen on several AnonCVS servers
On 2013/03/04 11:13, Ted Unangst wrote: On Mon, Mar 04, 2013 at 15:55, Stuart Henderson wrote: The client arch and software doesn't make a difference, the problem is on the server side. Problems seen when using opencvs server-side include giving out the wrong file version, and giving a checkout which can't reliably be used against a server running original CVS. hmmm, it's been a long time, but when I ran into this problem (like 2005), the client arch made all the difference. Or maybe back then the servers were all i386, too, and both client and server needed to be 32 bit? i.e., what used to be a client problem is now a server and client problem. yes, that's older - stsp fixed one problem with it since then... - PatchSet 254 Date: 2009/12/14 21:15:55 Author: stsp Branch: HEAD Tag: OPENBSD_4_7_BASE Log: Fix cvs [update aborted]: out of memory; can not reallocate 5242880 bytes when checking out xenocara from a server running OpenBSD/amd64. While processing RCS deltas, don't allocate twice as much memory as needed when copying a line vector to a vector which has less lines. Also, when switching back from a branch to trunk while searching an RCS file for a revision, free the trunklines vector immediately after lines saved in it have been copied back into the currentlines vector. Somehow, these two changes together make the problem go away. ok tobias@, this has been a serious annoyance sthen@, sure deraadt@ Members: src/rcs.c:1.22-1.23 - Hmm. Looking at the number of bytes in the error message which matches what's seen on anoncvs.usa and differs to my server... Todd, does the copy in your chroot pre-date this commit?
Re: out of memory errors seen on several AnonCVS servers
On Mon, 04 Mar 2013 16:19:56 GMT, Stuart Henderson wrote: Hmm. Looking at the number of bytes in the error message which matches what's seen on anoncvs.usa and differs to my server... Todd, does the copy in your chroot pre-date this commit? The cvs binary may not have had that fix on anoncvs.usa. I've updated and rebuilt the cvs binary which should fix it. - todd
Re: out of memory errors seen on several AnonCVS servers
On Mon, Mar 04, 2013 at 16:19, Stuart Henderson wrote: yes, that's older - stsp fixed one problem with it since then... Oh, missed that. Good to know. Hmm. Looking at the number of bytes in the error message which matches what's seen on anoncvs.usa and differs to my server... Todd, does the copy in your chroot pre-date this commit? Sweet. One problem solved. Sounds like the remaining issue is internal fragmentation, possibly exacerbated by randomization. How old is your cvs binary? I'm wondering if the realloc fixes I made last release have any effect. And of course, I think you can always up the ulimit.
Re: out of memory errors seen on several AnonCVS servers
On Mon, Mar 04, 2013 at 11:13:22AM -0500, Ted Unangst wrote: On Mon, Mar 04, 2013 at 15:55, Stuart Henderson wrote: The client arch and software doesn't make a difference, the problem is on the server side. Problems seen when using opencvs server-side include giving out the wrong file version, and giving a checkout which can't reliably be used against a server running original CVS. hmmm, it's been a long time, but when I ran into this problem (like 2005), the client arch made all the difference. Ditto. Ken Or maybe back then the servers were all i386, too, and both client and server needed to be 32 bit? i.e., what used to be a client problem is now a server and client problem.
Re: out of memory errors seen on several AnonCVS servers
I believe this problem ought to be fixed over the next few hours as mirrors pull updated files.
Re: out of memory errors seen on several AnonCVS servers
Summary of off-list mail exchange: anoncvs.usa consistently has a realloc failure in xenocara/font/misc-misc/18x18ja.bdf, some other servers have a failure the first time checking this file out when it's part of the whole directory, but resuming or checking out individually does work. $ cvs -d anon...@anoncvs.spacehopper.org:/cvs get xenocara/font/misc-misc U xenocara/font/misc-misc/10x20.bdf U xenocara/font/misc-misc/12x13ja.bdf cvs [server aborted]: out of memory; can not reallocate 3833880 bytes $ cvs -d anon...@anoncvs.spacehopper.org:/cvs get xenocara/font/misc-misc U xenocara/font/misc-misc/18x18ja.bdf U xenocara/font/misc-misc/18x18ko.bdf.xaa U xenocara/font/misc-misc/18x18ko.bdf.xab snip But against anoncvs.usa it always fails: $ cvs -d anon...@anoncvs.usa.openbsd.org:/cvs get xenocara/font/misc-misc/18x18ja.bdf cvs [server aborted]: out of memory; can not reallocate 5242880 bytes This only happens with the network protocol (either via pserver or 'cvs server' via ssh) not with local checkouts. Forcing a coredump when realloc fails results in the backtrace below. This file *can* be successfully checked out in a newer version of cvs (I built a vanilla copy from upstream's sources), I made a start at porting our patches across, but got stuck in a few places (and of course such a change would require *very* careful checking considering what we use cvs for). (gdb) bt #0 0x1ce596f65c1d in xrealloc (ptr=0x0, bytes=3833880) at /usr/src/gnu/usr.bin/cvs/src/subr.c:72 #1 0x1ce596f52953 in linevector_copy (to=0x7f7c7f60, from=0x7f7c7f50) at /usr/src/gnu/usr.bin/cvs/src/rcs.c:6762 #2 0x1ce596f53524 in RCS_deltas (rcs=0x1ce79d124f00, fp=0x1ce799b1ca00, rcsbuf=0x7f7c7f00, version=0x1ce79a67dfc0 1.1.1.1, op=RCS_FETCH, text=0x7f7c8228, len=0x7f7c8220, log=0x7f7c8218, loglen=0x7f7c8210) at /usr/src/gnu/usr.bin/cvs/src/rcs.c:7172 #3 0x1ce596f4e3e2 in RCS_checkout (rcs=0x1ce79d124f00, workfile=0x0, rev=0x1ce79a67dfc0 1.1.1.1, nametag=0x1ce79a67dfd0 1.1.1.1, options=0x1ce79a67d590 , sout=0x0, pfn=0x1ce596f6bea3 checkout_to_buffer, callerdat=0x1ce79d124400) at /usr/src/gnu/usr.bin/cvs/src/rcs.c:4114 #4 0x1ce596f6b7ae in checkout_file (finfo=0x7f7c8600, vers_ts=0x1ce79d124b00, adding=0, merging=0, update_server=1) at /usr/src/gnu/usr.bin/cvs/src/update.c:1367 #5 0x1ce596f6a768 in update_fileproc (callerdat=0x0, finfo=0x7f7c8600) at /usr/src/gnu/usr.bin/cvs/src/update.c:776 #6 0x1ce596f57ccc in do_file_proc (p=0x1ce7a1fad000, closure=0x7f7c8630) at /usr/src/gnu/usr.bin/cvs/src/recurse.c:823 #7 0x1ce596f30f61 in walklist (list=0x1ce79bc21000, proc=0x1ce596f57b4e do_file_proc, closure=0x7f7c8630) at /usr/src/gnu/usr.bin/cvs/src/hash.c:370 #8 0x1ce596f57a06 in do_recursion (frame=0x7f7c86b0) at /usr/src/gnu/usr.bin/cvs/src/recurse.c:727 #9 0x1ce596f582f5 in do_dir_proc (p=0x1ce7a1fad500, closure=0x7f7c8840) at /usr/src/gnu/usr.bin/cvs/src/recurse.c:1087 #10 0x1ce596f30f61 in walklist (list=0x1ce7a0e25800, proc=0x1ce596f57cfe do_dir_proc, closure=0x7f7c8840) at /usr/src/gnu/usr.bin/cvs/src/hash.c:370 #11 0x1ce596f57af9 in do_recursion (frame=0x7f7c88e0) at /usr/src/gnu/usr.bin/cvs/src/recurse.c:754 #12 0x1ce596f573c7 in start_recursion ( fileproc=0x1ce596f6a24a update_fileproc, filesdoneproc=0x1ce596f6a956 update_filesdone_proc, direntproc=0x1ce596f6aa89 update_dirent_proc, dirleaveproc=0x1ce596f6ae1a update_dirleave_proc, callerdat=0x0, argc=0, argv=0x1ce79a67d100, local=0, which=3, aflag=0, readlock=1, update_preload=0x1ce79bf776e0 xenocara/font/misc-misc, dosrcs=1) at /usr/src/gnu/usr.bin/cvs/src/recurse.c:351 #13 0x1ce596f6a212 in do_update (argc=0, argv=0x0, xoptions=0x0, xtag=0x0, xdate=0x0, xforce=1, local=0, xbuild=1, xaflag=0, xprune=0, xpipeout=0, which=3, xjoin_rev1=0x0, xjoin_rev2=0x0, preload_update_dir=0x1ce79bf776e0 xenocara/font/misc-misc, xdotemplate=1) at /usr/src/gnu/usr.bin/cvs/src/update.c:511 #14 0x1ce596f1767f in checkout_proc (argc=1, argv=0x1ce79a67d410, where_orig=0x0, mwhere=0x0, mfile=0x0, shorten=0, local_specified=0, omodule=0x1ce79bf77d20 xenocara/font/misc-misc, msg=0x1ce597093f6f Updating) at /usr/src/gnu/usr.bin/cvs/src/checkout.c:1014 #15 0x1ce596f425eb in do_module (db=0x1ce79bf77f60, mname=0x1ce79bf77d20 xenocara/font/misc-misc, m_type=CHECKOUT, msg=0x1ce597093f6f Updating, callback_proc=0x1ce596f1687f checkout_proc, where=0x0, shorten=0, local_specified=0, run_module_prog=1, build_dirs=1, extra_arg=0x0) at /usr/src/gnu/usr.bin/cvs/src/modules.c:317 #16 0x1ce596f1656f in checkout (argc=1, argv=0x1ce79bf775f0) at /usr/src/gnu/usr.bin/cvs/src/checkout.c:376 #17 0x1ce596f5fc52 in do_cvs_command (cmd_name=0x1ce5970a2fdb checkout,
Re: out of memory errors seen on several AnonCVS servers
On Sun, Mar 03, 2013 at 16:16, Stuart Henderson wrote: Summary of off-list mail exchange: anoncvs.usa consistently has a realloc failure in xenocara/font/misc-misc/18x18ja.bdf, some other servers have a failure the first time checking this file out when it's part of the whole directory, but resuming or checking out individually does work. amd64? Use i386. :) or opencvs.
Re: out of memory errors seen on several AnonCVS servers
On 2013/03/03 17:11, Ted Unangst wrote: On Sun, Mar 03, 2013 at 16:16, Stuart Henderson wrote: Summary of off-list mail exchange: anoncvs.usa consistently has a realloc failure in xenocara/font/misc-misc/18x18ja.bdf, some other servers have a failure the first time checking this file out when it's part of the whole directory, but resuming or checking out individually does work. amd64? Use i386. :) or opencvs. The problems in opencvs are more insidious.
Re: out of memory errors seen on several AnonCVS servers
On Sun, Mar 03, 2013 at 04:16:30PM +, Stuart Henderson wrote: Summary of off-list mail exchange: anoncvs.usa consistently has a realloc failure in xenocara/font/misc-misc/18x18ja.bdf, some other servers have a failure the first time checking this file out when it's part of the whole directory, but resuming or checking out individually does work. $ cvs -d anon...@anoncvs.spacehopper.org:/cvs get xenocara/font/misc-misc U xenocara/font/misc-misc/10x20.bdf U xenocara/font/misc-misc/12x13ja.bdf cvs [server aborted]: out of memory; can not reallocate 3833880 bytes I got this error consistently with anoncvs.openbsd.org as well, on both amd64 and mips64el. My use case is cvs -d$CVSROOT up -Pd for each of src, xenocara and ports (normal tracking of -stable). I believe I got the error on various other CVS servers until I started using spacehopper. I haven't seen the problem again. Thanks Stuart. /jl -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachments / \ Code Blue or Go Home! Encrypted email preferred PGP Key 2048R/DA65BC04
