Ah, zmq_ctx_set ZMQ_MAX_SOCKETS defaults to 1024... I'm not sure what the setting for "infinite" is, it's not documented. Probably should be -1 to be consistent with zmq_poll.
On Sat, Nov 9, 2013 at 8:32 PM, Bruno D. Rodrigues <[email protected]> wrote: > I’ve patched the code to do as you said, creating sockets until a null, and > then closing them all, printing a lot to debug it for now, and what I see on > the Mac is the following: > > - the sockets are happily created up to 1024, irrespectively of the ulimit -n > value being 256, 512, 1024 or anything up to 8192. (need to grep the code and > understand where is this 1024 limit coming from, as it surely ain’t > respecting the ulimit -n) > > - the test fails when it tries to close the sockets. Depending on the value > of ulimit, it will not be able to close all sockets, and will either hang > forever (limit = 8192) or fail in the middle with an assertion and a stack > smash affecting the close cycle > - for ulimit -n 8192, the last lines are Socket “…\nclose: 112\nSocket clos” > and hangs forever > - for ulimit -n 1024, 512 and 256, it doesn’t hang, but reports a "Bad file > descriptor (signaler.cpp:110)” and the counter mixes itself up. > - For example, for 256: (…) means order is correct, at 794 the assertion, > then i becomes 695 to 691, then an empty line (with?), then back at 788 > (where are the 793 to 789), then dies at 696: > Socket close: 795 > Socket close: 794Bad file descriptor (signaler.cpp:110) > Socket close: 695 > Socket close: 694 > (…) > Socket close: 691 > > Socket close: 788 > Socket close: 787 > (…) > Socket close: 696 > (nothing else) > > I’m gonna put the kid to bed, and then compare the compilation and behavior > on linux and try to understand a little bit more on the code, but if anyone > has any idea already about: > > - why is it possible to create 1024 sockets irrespectively of the file > handler limits. is this 1024 hardcoded somewhere on the code? > - where could be the “stack overflow” that affects the close in such a random > manner. > > brb > > > index d7d85d7..b0c4ce6 100644 > --- a/tests/test_many_sockets.cpp > +++ b/tests/test_many_sockets.cpp > @@ -22,7 +22,8 @@ > #include <stdio.h> > #include <stdlib.h> > > -const int no_of_sockets = 5000; > +const int no_of_sockets = 1024 * 1024; > + > > int main(void) > { > @@ -36,15 +37,19 @@ int main(void) > for ( int i = 0; i < no_of_sockets; ++i ) > { > sockets[i] = zmq_socket(ctx, ZMQ_PAIR); > - if (sockets[i]) > - ++sockets_created; > + if (!sockets[i]) > + break; > + printf("Socket created: %d\n", i); > + ++sockets_created; > } > > - assert(sockets_created < no_of_sockets); > + printf("Socket limit: %d\n", sockets_created); > > - for ( int i = 0; i < no_of_sockets; ++i ) > - if (sockets[i]) > - zmq_close (sockets[i]); > + for ( int i = sockets_created-1; i != 0; --i ) > + { > + printf("Socket close: %d\n", i); > + zmq_close (sockets[i]); > + } > > zmq_ctx_destroy (ctx); > return 0; > > > > On Nov 9, 2013, at 12:42, Pieter Hintjens <[email protected]> wrote: > >> The test_many_sockets should IMO create sockets in a loop, without >> limit, until it receives a NULL return, and then exit happily. The >> goal being to check that libzmq does not crash or assert when this >> condition hits. >> >> On Sat, Nov 9, 2013 at 1:19 PM, Bruno D. Rodrigues >> <[email protected]> wrote: >>> So if the limits are raised should the test fail or still pass albeit >>> is not testing anything? >>> >>> I don't think it's a good idea to have tests depending on >>> externalities as they should run consistently no matter what the >>> ulimits -n is (as long as it's sane), but can I assume a default of >>> 1024 (is it the default/minimum from Linux?), and if so I'll try to >>> have a look at them later. >>> >>> 1. Assume ulimits 1024? Why does the doc say 1200? >>> >>> 2. That test shall pass or fail if limits are raised? >>> >>> >>> >>> -- >>> Bruno Rodrigues >>> Sent from my iPhone >>> >>> No dia 09/11/2013, às 11:25, Pieter Hintjens <[email protected]> escreveu: >>> >>>> XFAIL is intentional failure, which is normal for the two tests that have >>>> it. >>>> >>>> The test_many_sockets test is meant to exceed system limits and check >>>> libzmq deals with it correctly. It shouldn't need raising process >>>> handles to pass. The code may still be flaky on OS/X. >>>> >>>> >>>> On Sat, Nov 9, 2013 at 12:18 PM, Bruno D. Rodrigues >>>> <[email protected]> wrote: >>>>> With the current master, I get all pass on macosx as long as I run ulimit >>>>> -n 8192 before. >>>>> >>>>> The test_many_sockets fails because it creates 5K sockets. The shutdown >>>>> may fail for similar reasons. >>>>> >>>>> ============================================================================ >>>>> Testsuite summary for zeromq 4.1.0 >>>>> ============================================================================ >>>>> # TOTAL: 46 >>>>> # PASS: 44 >>>>> # SKIP: 0 >>>>> # XFAIL: 2 >>>>> # FAIL: 0 >>>>> # XPASS: 0 >>>>> # ERROR: 0 >>>>> >>>>> (I assume the 2 XFAIL are okish?) >>>>> >>>>> >>>>>> On Nov 9, 2013, at 9:38, Pieter Hintjens <[email protected]> wrote: >>>>>> >>>>>>> On Sat, Nov 9, 2013 at 8:17 AM, Matt Connolly <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> test_many_sockets.log: >>>>>>> Assertion failed: nbytes == sizeof (dummy) (signaler.cpp:149) >>>>>> >>>>>> Some unhandled error condition on line 140; presumably specific to OS/X. >>>>>> >>>>>> -Pieter >>>>>> _______________________________________________ >>>>>> zeromq-dev mailing list >>>>>> [email protected] >>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>> >>>>> _______________________________________________ >>>>> zeromq-dev mailing list >>>>> [email protected] >>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>> >>>> >>>> >>>> -- >>>> - >>>> Pieter Hintjens >>>> CEO of iMatix.com >>>> Founder of ZeroMQ community >>>> blog: http://hintjens.com >>>> _______________________________________________ >>>> zeromq-dev mailing list >>>> [email protected] >>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> >> -- >> - >> Pieter Hintjens >> CEO of iMatix.com >> Founder of ZeroMQ community >> blog: http://hintjens.com >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev -- - Pieter Hintjens CEO of iMatix.com Founder of ZeroMQ community blog: http://hintjens.com _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
