Re: Infinite loop after 39cc020af BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both()

2019-04-15 Thread Willy Tarreau
On Mon, Apr 15, 2019 at 12:46:05PM -0400, Richard Russo wrote:
> After the weekend, the test machine looks fine. Thanks!

Thank you for this positive feedback, Richard, much appreciated!

Willy



Re: Infinite loop after 39cc020af BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both()

2019-04-15 Thread Richard Russo
After the weekend, the test machine looks fine. Thanks!

-- 
  Richard Russo
  to...@enslaves.us

On Fri, Apr 12, 2019, at 10:33 AM, Richard Russo wrote:
> Thank you; I had missed the context from 1.9.6.  I've updated my test 
> machine and will report back on Monday (or earlier, if it runs into 
> trouble)
> 
> -- 
>   Richard Russo
>   to...@enslaves.us
> 
> On Fri, Apr 12, 2019, at 4:17 AM, Olivier Houchard wrote:
> > Hi,
> > 
> > On Fri, Apr 12, 2019 at 08:37:10AM +0200, Maciej Zdeb wrote:
> > > Hi Richard,
> > > 
> > > Those patches from Olivier (in streams) are related to my report from
> > > thread "[1.9.6] One of haproxy processes using 100% CPU", but as it turned
> > > out it wasn't a single bug and issue is not entirely fixed yet.
> > > 
> > > Currently I'm testing some additional patches from Olivier which hopefully
> > > fix the issue definitely.
> > > 
> > 
> > Indeed, the rmoeval of SI_FL_ERR in si_update_both() was bogus, and covered
> > misuses of it.
> > With the great help of Maciej, we investigated this, and I just pushed what
> > we fixed so far. Richard, I'd be really interested in knowing if you still
> > have issues with the latest master.
> > 
> > Thanks !
> > 
> > Olivier
> > 
> > > pt., 12 kwi 2019 o 00:01 Richard Russo  napisał(a):
> > > 
> > > > It seems that after applying 39cc020af, if a stream gets the SI_FL_ERR
> > > > flag, process_stream can keep going back to redo around stream.c:line 
> > > > 2503:
> > > >
> > > > if (si_f->state == SI_ST_DIS || si_f->state != si_f_prev_state ||
> > > > si_b->state == SI_ST_DIS || si_b->state != si_b_prev_state ||
> > > > ((si_f->flags | si_b->flags) & SI_FL_ERR) ||
> > > > (((req->flags ^ rqf_last) | (res->flags ^ rpf_last)) &
> > > > CF_MASK_ANALYSER))
> > > >  goto redo;
> > > >
> > > > Now that si_update_both no longer clears the SI_FL_ERR flag, and nothing
> > > > else does, the goto will get called forever. I don't understand this
> > > > section enough to try to reproduce this, but I found several processes
> > > > stuck here on a machine testing from yesterday's HEAD.
> > > >
> > > > Richard
> > > >
> > > > --
> > > >   Richard Russo
> > > >   to...@enslaves.us
> > > >
> > > >
> >
> 
>



Re: Infinite loop after 39cc020af BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both()

2019-04-12 Thread Richard Russo
Thank you; I had missed the context from 1.9.6.  I've updated my test machine 
and will report back on Monday (or earlier, if it runs into trouble)

-- 
  Richard Russo
  to...@enslaves.us

On Fri, Apr 12, 2019, at 4:17 AM, Olivier Houchard wrote:
> Hi,
> 
> On Fri, Apr 12, 2019 at 08:37:10AM +0200, Maciej Zdeb wrote:
> > Hi Richard,
> > 
> > Those patches from Olivier (in streams) are related to my report from
> > thread "[1.9.6] One of haproxy processes using 100% CPU", but as it turned
> > out it wasn't a single bug and issue is not entirely fixed yet.
> > 
> > Currently I'm testing some additional patches from Olivier which hopefully
> > fix the issue definitely.
> > 
> 
> Indeed, the rmoeval of SI_FL_ERR in si_update_both() was bogus, and covered
> misuses of it.
> With the great help of Maciej, we investigated this, and I just pushed what
> we fixed so far. Richard, I'd be really interested in knowing if you still
> have issues with the latest master.
> 
> Thanks !
> 
> Olivier
> 
> > pt., 12 kwi 2019 o 00:01 Richard Russo  napisał(a):
> > 
> > > It seems that after applying 39cc020af, if a stream gets the SI_FL_ERR
> > > flag, process_stream can keep going back to redo around stream.c:line 
> > > 2503:
> > >
> > > if (si_f->state == SI_ST_DIS || si_f->state != si_f_prev_state ||
> > > si_b->state == SI_ST_DIS || si_b->state != si_b_prev_state ||
> > > ((si_f->flags | si_b->flags) & SI_FL_ERR) ||
> > > (((req->flags ^ rqf_last) | (res->flags ^ rpf_last)) &
> > > CF_MASK_ANALYSER))
> > >  goto redo;
> > >
> > > Now that si_update_both no longer clears the SI_FL_ERR flag, and nothing
> > > else does, the goto will get called forever. I don't understand this
> > > section enough to try to reproduce this, but I found several processes
> > > stuck here on a machine testing from yesterday's HEAD.
> > >
> > > Richard
> > >
> > > --
> > >   Richard Russo
> > >   to...@enslaves.us
> > >
> > >
>



Re: Infinite loop after 39cc020af BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both()

2019-04-12 Thread Olivier Houchard
Hi,

On Fri, Apr 12, 2019 at 08:37:10AM +0200, Maciej Zdeb wrote:
> Hi Richard,
> 
> Those patches from Olivier (in streams) are related to my report from
> thread "[1.9.6] One of haproxy processes using 100% CPU", but as it turned
> out it wasn't a single bug and issue is not entirely fixed yet.
> 
> Currently I'm testing some additional patches from Olivier which hopefully
> fix the issue definitely.
> 

Indeed, the rmoeval of SI_FL_ERR in si_update_both() was bogus, and covered
misuses of it.
With the great help of Maciej, we investigated this, and I just pushed what
we fixed so far. Richard, I'd be really interested in knowing if you still
have issues with the latest master.

Thanks !

Olivier

> pt., 12 kwi 2019 o 00:01 Richard Russo  napisał(a):
> 
> > It seems that after applying 39cc020af, if a stream gets the SI_FL_ERR
> > flag, process_stream can keep going back to redo around stream.c:line 2503:
> >
> > if (si_f->state == SI_ST_DIS || si_f->state != si_f_prev_state ||
> > si_b->state == SI_ST_DIS || si_b->state != si_b_prev_state ||
> > ((si_f->flags | si_b->flags) & SI_FL_ERR) ||
> > (((req->flags ^ rqf_last) | (res->flags ^ rpf_last)) &
> > CF_MASK_ANALYSER))
> >  goto redo;
> >
> > Now that si_update_both no longer clears the SI_FL_ERR flag, and nothing
> > else does, the goto will get called forever. I don't understand this
> > section enough to try to reproduce this, but I found several processes
> > stuck here on a machine testing from yesterday's HEAD.
> >
> > Richard
> >
> > --
> >   Richard Russo
> >   to...@enslaves.us
> >
> >



Re: Infinite loop after 39cc020af BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both()

2019-04-12 Thread Maciej Zdeb
Hi Richard,

Those patches from Olivier (in streams) are related to my report from
thread "[1.9.6] One of haproxy processes using 100% CPU", but as it turned
out it wasn't a single bug and issue is not entirely fixed yet.

Currently I'm testing some additional patches from Olivier which hopefully
fix the issue definitely.

pt., 12 kwi 2019 o 00:01 Richard Russo  napisał(a):

> It seems that after applying 39cc020af, if a stream gets the SI_FL_ERR
> flag, process_stream can keep going back to redo around stream.c:line 2503:
>
> if (si_f->state == SI_ST_DIS || si_f->state != si_f_prev_state ||
> si_b->state == SI_ST_DIS || si_b->state != si_b_prev_state ||
> ((si_f->flags | si_b->flags) & SI_FL_ERR) ||
> (((req->flags ^ rqf_last) | (res->flags ^ rpf_last)) &
> CF_MASK_ANALYSER))
>  goto redo;
>
> Now that si_update_both no longer clears the SI_FL_ERR flag, and nothing
> else does, the goto will get called forever. I don't understand this
> section enough to try to reproduce this, but I found several processes
> stuck here on a machine testing from yesterday's HEAD.
>
> Richard
>
> --
>   Richard Russo
>   to...@enslaves.us
>
>


Infinite loop after 39cc020af BUG/MEDIUM: streams: Don't remove the SI_FL_ERR flag in si_update_both()

2019-04-11 Thread Richard Russo
It seems that after applying 39cc020af, if a stream gets the SI_FL_ERR flag, 
process_stream can keep going back to redo around stream.c:line 2503:

if (si_f->state == SI_ST_DIS || si_f->state != si_f_prev_state ||
si_b->state == SI_ST_DIS || si_b->state != si_b_prev_state ||
((si_f->flags | si_b->flags) & SI_FL_ERR) ||
(((req->flags ^ rqf_last) | (res->flags ^ rpf_last)) & CF_MASK_ANALYSER))
 goto redo;

Now that si_update_both no longer clears the SI_FL_ERR flag, and nothing else 
does, the goto will get called forever. I don't understand this section enough 
to try to reproduce this, but I found several processes stuck here on a machine 
testing from yesterday's HEAD.

Richard

-- 
  Richard Russo
  to...@enslaves.us