Hi Pieter,

On Mon, Oct 12, 2015 at 01:22:48AM +0200, PiBa-NL wrote:
> >>>>>>#1  0x0000000000417388 in buffer_slow_realign (buf=0x7d3c90) at
> >>>>>>src/buffer.c:166
> >>>>>>           block1 = -3306
> >>>>>>           block2 = 0

I'm puzzled by this above, no block should have a negative size.

> >>>>>>#2  0x0000000000480c42 in http_wait_for_request (s=0x80247d600,
> >>>>>>req=0x80247d610, an_bit=4)
> >>>>>>       at src/proto_http.c:2686
> >>>>>>           cur_idx = -6336
> >>>>>>           sess = (struct session *) 0x80241e400
> >>>>>>           txn = (struct http_txn *) 0x802bb2140
> >>>>>>           msg = (struct http_msg *) 0x802bb21a0
> >>>>>>           ctx = {line = 0x2711079 <Address 0x2711079 out of 
> >>>>>>bounds>, idx = 3, val = 0, vlen = 7, tws = 0, del = 33, prev = 0}

And this above, similarly cur_idx shouldn't be negative.

> >Seems that buffer_slow_realign() isn't used regularly during normal 
> >haproxy operation, and it crashes first time that specific function 
> >gets called.
> >Reproduction is pretty consistent with chrome browser refreshing stats 
> >every second.
> >Then starting: wrk -c 200 -t 2 -d 10
> >I tried adding some Alert(); items in the code to see what parameters 
> >are set at what step, but am not understanding the exact internals of 
> >that code..
> >
> >This negative bh=-7800 is not supposed to be there i think? It is from 
> >one of the dprintf statements, how are those supposed generate output?..
> >[891069718] http_wait_for_request: stream=0x80247d600 b=0x80247d610, 
> >exp(r,w)=0,0 bf=00c08200 bh=-7800 analysers=34
> >
> >Anything else i can check or provide to help get this fixed?
> >
> >Best regards,
> >PiBa-NL
> Just a little 'bump' to this issue..
> Anyone know when/how this buffer_slow_realign() is suppose to work?

Yes, it's supposed to be used only when a request or response is wrapped
in the request or response buffer. It uses memcpy(), hence the "slow"
aspect of the realign.

> I suspect it either contains a bug, or is called with bogus parameters..

It's very sensitive to the consistency of the buffer being realigned. So
errors such as buf->i + buf->o > buf->size, or buf->p > buf->data + buf->size,
or buf->p < buf->data etc... can lead to crashes. But these must never happen
at all otherwise it proves that there's a bug somewhere else.

Here since block1 is -3306 and block2 = 0, I suspect that they were assigned
at line 159 from buf->i, which definitely means that the buffer was already

> How can we/i determine which it is?

The difficulty consists in finding what can lead to a corrupted buffer :-/
In the past we had such issues when trying to forward more data than was
available in the buffer, due to option send-name-header. I wouldn't be
surprized that it can happen here on corner cases when building a message
from Lua if the various message pointers are not all perfectly correct.

> Even though with a small change in the config (adding a backend) i cant 
> reproduce it, that doesnt mean there isn't a problem with the fuction.. 
> As the whole function doesn't seem to get called in that circumstance..

It could be related to an uninitialized variable somewhere as well. You
can try to start haproxy with "-dM" to see if it makes the issues 100%
reproducible or not. This poisons all buffers (fills them with a constant
byte 0x50 after malloc) so that we don't rely on an uninitialized zero byte


Reply via email to