Hi Frank, Sounds like it may be this bug:
https://issues.apache.org/jira/browse/PROTON-892 what is the value of data->capacity when the pni_data_grow method is called? -K ----- Original Message ----- > From: "Frank Quinn" <fquinn...@gmail.com> > To: proton@qpid.apache.org > Sent: Tuesday, September 8, 2015 6:52:25 PM > Subject: Qpid Proton pn_data_put_ubyte crash updating pn_message_properties > data > > Hi Folks, > > I'm seeing crashing in qpid proton (qpid-proton-c-devel-0.9-3 on fc22) when > running my publisher at a decent rate (1000 msg/s) for a minute or so. > > Does something like this look familiar? Somehow got into a state where > pn_data_grow is called, but the data node coming back from it points to an > invalid memory location. > > (gdb) > bt > [31/1873] > #0 0x0000000006bf55df in pn_data_new (data=data@entry=0x5e6a090) at > /usr/src/debug/qpid-proton-0.9/proton-c/src/codec/codec.c:1126 > #1 0x0000000006bf5de8 in pn_data_add (data=data@entry=0x5e6a090) at > /usr/src/debug/qpid-proton-0.9/proton-c/src/codec/codec.c:1397 > #2 0x0000000006bf5fa8 in pn_data_put_ubyte (data=data@entry=0x5e6a090, > ub=<optimized out>) at > /usr/src/debug/qpid-proton-0.9/proton-c/src/codec/codec.c:1485 > #3 0x00000000069c4c14 in qpidBridgeMsgCodec_pack (bridgeMessage=0x5e61fb0, > target=target@entry=0x5e6b2c0, protonMessage=protonMessage@entry > =0xffefff528) > at mama/c_cpp/src/c/bridge/qpid/codec.c:119 > #4 0x00000000069c5fd2 in qpidBridgePublisherImpl_enqueueMessageForAddress > (msg=msg@entry=0x5e6b2c0, url=<optimized out>, impl=impl@entry=0x5e61f30) > at mama/c_cpp/src/c/bridge/qpid/publisher.c:629 > #5 0x00000000069c63cd in qpidBridgeMamaPublisher_send > (publisher=0x5e61f30, msg=0x5e6b2c0) at > mama/c_cpp/src/c/bridge/qpid/publisher.c:295 > #6 0x0000000004e6be97 in mamaPublisher_send (publisher=0x5e61dc0, > msg=msg@entry=0x5e6b2c0) at mama/c_cpp/src/c/publisher.c:258 > #7 0x000000000040577d in publishMessageRdtsc (pubIndex=<optimized out>, > msgSample=<optimized out>, nowTsc=<optimized out>) at > mama/c_cpp/src/testtools/performance/c/mamaproducerc_v2.c:1620 > #8 0x0000000000403b20 in main (argc=<optimized out>, argv=<optimized out>) > at mama/c_cpp/src/testtools/performance/c/mamaproducerc_v2.c:782 > (gdb) l > 1121 { > 1122 if (data->capacity <= data->size) { > 1123 pn_data_grow(data); > 1124 } > 1125 pni_node_t *node = pn_data_node(data, ++(data->size)); > 1126 node->next = 0; > 1127 node->down = 0; > 1128 node->children = 0; > 1129 return node; > 1130 } > (gdb) print *data > $11 = {nodes = 0x0, buf = 0x5e6a5d0, decoder = 0x5e6a6c0, encoder = > 0x5e6a790, error = 0x5e6a850, str = 0x5e6a8c0, capacity = 0, size = 32769, > parent = 1, current = 0, base_parent = 0, > base_current = 0} > (gdb) print node > $12 = (pni_node_t *) 0x240000 <---- invalid memory location > > I ran it through valgrind as well and couldn't see any invalid memory > trouncing leading up to the event: > > ==2332== Invalid write of size 2 > ==2332== at 0x6BF55DF: pn_data_new (codec.c:1126) > ==2332== by 0x6BF5DE7: pn_data_add (codec.c:1397) > ==2332== by 0x6BF5FA7: pn_data_put_ubyte (codec.c:1485) > ==2332== by 0x69C4C13: qpidBridgeMsgCodec_pack (codec.c:119) > ==2332== by 0x69C5FD1: qpidBridgePublisherImpl_enqueueMessageForAddress > (publisher.c:629) > ==2332== by 0x69C63CC: qpidBridgeMamaPublisher_send (publisher.c:295) > ==2332== by 0x4E6BE96: mamaPublisher_send (publisher.c:258) > ==2332== by 0x40577C: publishMessageRdtsc (mamaproducerc_v2.c:1620) > ==2332== by 0x403B1F: main (mamaproducerc_v2.c:782) > ==2332== Address 0x240034 is not stack'd, malloc'd or (recently) free'd > ==2332== > ==2332== > ==2332== Process terminating with default action of signal 11 (SIGSEGV): > dumping core > ==2332== Access not within mapped region at address 0x240034 > ==2332== at 0x6BF55DF: pn_data_new (codec.c:1126) > ==2332== by 0x6BF5DE7: pn_data_add (codec.c:1397) > ==2332== by 0x6BF5FA7: pn_data_put_ubyte (codec.c:1485) > ==2332== by 0x69C4C13: qpidBridgeMsgCodec_pack (codec.c:119) > ==2332== by 0x69C5FD1: qpidBridgePublisherImpl_enqueueMessageForAddress > (publisher.c:629) > ==2332== by 0x69C63CC: qpidBridgeMamaPublisher_send (publisher.c:295) > ==2332== by 0x4E6BE96: mamaPublisher_send (publisher.c:258) > ==2332== by 0x40577C: publishMessageRdtsc (mamaproducerc_v2.c:1620) > ==2332== by 0x403B1F: main (mamaproducerc_v2.c:782) > ==2332== If you believe this happened as a result of a stack > ==2332== overflow in your program's main thread (unlikely but > ==2332== possible), you can try to increase the size of the > ==2332== main thread stack using the --main-stacksize= flag. > ==2332== The main thread stack size used in this run was 8388608. > > > Cheers, > Frank > -- -K
