I tried using valgrind's massif tool with the following command valgrind --trace-children=yes --tool=massif --pages-as-heap=yes --detailed-freq=1000000 /zmqtestapp
Again, all my test app does is create context, create socket, destroy socket, destroy context. Here's what I saw as the output. I'm not very familiar with valgrind but I'm guessing some pthread_create call is allocating 16MB -------------------------------------------------------------------------------- n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B) -------------------------------------------------------------------------------- 37 2,877,217 3,629,056 3,629,056 0 0 38 2,967,318 3,764,224 3,764,224 0 0 39 2,985,374 3,768,320 3,768,320 0 0 40 3,013,543 12,156,928 12,156,928 0 0 41 3,034,072 20,545,536 20,545,536 0 0 42 3,082,222 22,642,688 22,642,688 0 0 43 3,082,241 21,934,080 21,934,080 0 0 100.00% (21,934,080B) (page allocation syscalls) mmap/mremap/brk, --alloc-fns, etc. ->82.84% (18,169,856B) 0x49756F7: mmap (in /lib/libc-2.17.so) | ->76.49% (16,777,216B) 0x49F48D6: pthread_create@@GLIBC_2.4 (in /lib/ libpthread-2.17.so) | | | ->06.33% (1,388,544B) 0x491CE96: new_heap (in /lib/libc-2.17.so) | | | ->00.02% (4,096B) in 1+ places, all below ms_print's threshold (01.00%) | ->15.80% (3,465,216B) 0x4018127: mmap (in /lib/ld-2.17.so) | ->15.13% (3,317,760B) 0x4006B7A: _dl_map_object_from_fd (in /lib/ ld-2.17.so) | | | ->00.67% (147,456B) in 1+ places, all below ms_print's threshold (01.00%) | ->01.36% (299,008B) in 2 places, all below massif's threshold (01.00%) On Thu, Jul 24, 2014 at 4:21 PM, Pieter Hintjens <[email protected]> wrote: > Perhaps there's a way to run under valgrind or some other checker, and > see where that memory is being allocated? Working blind is not going > to be easy. > > On Thu, Jul 24, 2014 at 7:23 PM, Philip Dizon <[email protected]> > wrote: > > Sorry to resurrect this old thread as I've been working on a more urgent > > project. > > > > Anyway, it seems that setting the max sockets has no effect on the > amount of > > virtual memory used. I tried setting max sockets to 1 and the memory > still > > jumped up by 17MB after the first socket was created. > > > > Here's my sample code using just plain zmq and not czmq. > > void *acontext = zmq_ctx_new(); > > zmq_ctx_set (acontext, ZMQ_MAX_SOCKETS, maxSockets); > > printf("ZMQ_MAX_SOCKETS = %d\n", zmq_ctx_get (acontext, > ZMQ_MAX_SOCKETS)); > > // 3MB virtual memory used at this point > > void *foo = zmq_socket (acontext, ZMQ_REP); // 20MB virtual memory used > at > > this point > > > > > > On Mon, Jun 23, 2014 at 10:44 PM, Pieter Hintjens <[email protected]> wrote: > >> > >> In czmq master this is now configurable via zsys for the zsock API, > which > >> doesn't use contexts. > >> > >> On Jun 24, 2014 6:55 AM, "Michel Pelletier" <[email protected] > > > >> wrote: > >>> > >>> Yep, you can see a comment in the Python code I posted that > demonstrates > >>> there is no underlying context until a socket is created. > >>> > >>> Why not create a dummy socket and the destroy it? It's not pretty but > it > >>> works. > >>> > >>> -Michel > >>> > >>> > >>> On Mon, Jun 23, 2014 at 9:16 PM, Steve Murphy <[email protected]> > wrote: > >>>> > >>>> Michel-- > >>>> > >>>> Just read a little deeper on zctx_underlying... > >>>> > >>>> The doc says: > >>>> > >>>> will be NULL before first socket > >>>> // is created. Use with care. > >>>> > >>>> And, yet, in this use case, we would have to set > >>>> the MAX_SOCKETS before the first socket is created.... > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> so, zctx_underlying() won't be useful. > >>>> > >>>> > >>>> murf > >>>> > >>>> > >>>> > >>>> On Mon, Jun 23, 2014 at 9:41 PM, Steve Murphy <[email protected]> > >>>> wrote: > >>>>> > >>>>> Thanks, Michel-- > >>>>> > >>>>> missed that function in the zctx set! > >>>>> Many thanks! > >>>>> > >>>>> murf > >>>>> > >>>>> > >>>>> > >>>>> On Mon, Jun 23, 2014 at 8:56 PM, Michel Pelletier > >>>>> <[email protected]> wrote: > >>>>>> > >>>>>> You can use zctx_underlying to get the low level context object. > Here > >>>>>> is an example using pyczmq: > >>>>>> > >>>>>> >>> from pyczmq import zctx, zsocket, zmq > >>>>>> >>> s = zctx.new() > >>>>>> >>> zctx.underlying(s) # None until a socket is made > >>>>>> >>> p = zsocket.new(s, zmq.PUSH) > >>>>>> >>> zmq.ctx_set(zctx.underlying(s), zmq.MAX_SOCKETS, 10) > >>>>>> 0 > >>>>>> >>> zmq.ctx_get(zctx.underlying(s), zmq.MAX_SOCKETS) > >>>>>> 10 > >>>>>> >>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Mon, Jun 23, 2014 at 7:26 PM, Steve Murphy <[email protected]> > >>>>>> wrote: > >>>>>>> > >>>>>>> Philip-- > >>>>>>> > >>>>>>> In the code you provided, you are mixing czmq > >>>>>>> and zmq lib calls, and as stated, it won't work. > >>>>>>> > >>>>>>> The zctx_t that zctx_new provides is NOT a void* > >>>>>>> that zmq_ctx_new() would give you, and the calls to zmq_ctx_set() > and > >>>>>>> .._get() > >>>>>>> will not work properly if given a zctx_t. If you replace your call > >>>>>>> to zctx_new() with zmq_ctx_new(), you will get better results. > >>>>>>> > >>>>>>> Now, Pieter mentioned that: > >>>>>>> > >>>>>>> "You can lower the max sockets per context, before creating your > >>>>>>> first > >>>>>>> context. See zmq_ctx_set ()." > >>>>>>> > >>>>>>> But I think he meant to say "before creating your first socket." > >>>>>>> > >>>>>>> (which, btw, is not in the ZMQ ref manual.) > >>>>>>> > >>>>>>> I double checked the zctx page in the CZMQ spec, and no function > >>>>>>> is available to get/set the context options... at least the > >>>>>>> MAX_SOCKETS, that is. > >>>>>>> So, if you need to play with MAX_SOCKETS, you have to abandon CZMQ, > >>>>>>> as there is no > >>>>>>> way to slip from the zctx_t world to the void* world. > >>>>>>> > >>>>>>> And a quick look at max sockets on a new context shows that the > >>>>>>> default for > >>>>>>> a new context is 1023. in the 4.1 stuff, there is also a > >>>>>>> ZMQ_SOCKET_LIMIT, > >>>>>>> which is the absolute maximum you can set in a set() call, but this > >>>>>>> isn't > >>>>>>> in the 4.0 versions. > >>>>>>> > >>>>>>> murf > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Mon, Jun 23, 2014 at 10:25 AM, Philip Dizon > >>>>>>> <[email protected]> wrote: > >>>>>>>> > >>>>>>>> Is it possible to set it ZMQ_MAX_SOCKETS for a czmq context object > >>>>>>>> because I'm getting an assert failure when I do a zmq_ctx_get? > >>>>>>>> > >>>>>>>> e.g. > >>>>>>>> client->ctx = zctx_new(); > >>>>>>>> assert(client->ctx); > >>>>>>>> > >>>>>>>> int max_sockets = 256; > >>>>>>>> zmq_ctx_set (client->ctx, ZMQ_MAX_SOCKETS, max_sockets); > >>>>>>>> assert (zmq_ctx_get (client->ctx, ZMQ_MAX_SOCKETS) == > >>>>>>>> max_sockets); > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Thu, Jun 19, 2014 at 4:05 PM, Pieter Hintjens <[email protected]> > >>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>> You can lower the max sockets per context, before creating your > >>>>>>>>> first > >>>>>>>>> context. See zmq_ctx_set (). > >>>>>>>>> > >>>>>>>>> On Fri, Jun 20, 2014 at 12:39 AM, Philip Dizon > >>>>>>>>> <[email protected]> wrote: > >>>>>>>>> > Hi, > >>>>>>>>> > > >>>>>>>>> > I noticed that just creating a new zmq context and socket bumps > >>>>>>>>> > my memory > >>>>>>>>> > usage by 18MB, and this is a big problem on my embedded system > >>>>>>>>> > which only > >>>>>>>>> > contains about 100MB. I determined this by using top command > and > >>>>>>>>> > comparing > >>>>>>>>> > the difference in mem usage. > >>>>>>>>> > Is there any sort of option I can use to lower the memory > usage? > >>>>>>>>> > > >>>>>>>>> > Thanks > >>>>>>>>> > > >>>>>>>>> > _______________________________________________ > >>>>>>>>> > zeromq-dev mailing list > >>>>>>>>> > [email protected] > >>>>>>>>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >>>>>>>>> > > >>>>>>>>> _______________________________________________ > >>>>>>>>> zeromq-dev mailing list > >>>>>>>>> [email protected] > >>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> _______________________________________________ > >>>>>>>> zeromq-dev mailing list > >>>>>>>> [email protected] > >>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> > >>>>>>> Steve Murphy > >>>>>>> ParseTree Corporation > >>>>>>> 57 Lane 17 > >>>>>>> Cody, WY 82414 > >>>>>>> ✉ murf at parsetree dot com > >>>>>>> ☎ 307-899-5535 > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> zeromq-dev mailing list > >>>>>>> [email protected] > >>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >>>>>>> > >>>>>> > >>>>>> > >>>>>> _______________________________________________ > >>>>>> zeromq-dev mailing list > >>>>>> [email protected] > >>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> > >>>>> Steve Murphy > >>>>> ParseTree Corporation > >>>>> 57 Lane 17 > >>>>> Cody, WY 82414 > >>>>> ✉ murf at parsetree dot com > >>>>> ☎ 307-899-5535 > >>>>> > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> > >>>> Steve Murphy > >>>> ParseTree Corporation > >>>> 57 Lane 17 > >>>> Cody, WY 82414 > >>>> ✉ murf at parsetree dot com > >>>> ☎ 307-899-5535 > >>>> > >>>> > >>>> > >>>> _______________________________________________ > >>>> zeromq-dev mailing list > >>>> [email protected] > >>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >>>> > >>> > >>> > >>> _______________________________________________ > >>> zeromq-dev mailing list > >>> [email protected] > >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >>> > >> > >> _______________________________________________ > >> zeromq-dev mailing list > >> [email protected] > >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >> > > > > > > _______________________________________________ > > zeromq-dev mailing list > > [email protected] > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
