Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Henrik Nordström
sön 2012-12-02 klockan 22:27 -0700 skrev Alex Rousskov: My understanding is that the bug is in Squid store rebuilding(?) code that assumes that Squid will have time to do some I/O before the next timed event is fired, but actually there is a lot of non-I/O work to do, In such case the bug is

Re: ICAP vectoring points

2012-12-03 Thread Steve Hill
On 30.11.12 17:18, Alex Rousskov wrote: but categorizing a response before it enters the cache does make a lot of sense in many cases. It does, except you don't necessarily know what categorisation is required at that point. Ideally it would be nice to categorise a request in

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Alex Rousskov
On 12/03/2012 01:44 AM, Henrik Nordström wrote: The store rebuilding does eventAdd(storeRebuild, RebuildStep, rb, 0.01, 1); which says both delay = 0.01, avoid saturating things too much weight = 1, this is a heavy job, slow down event.cc processing when this is seen. Only

Re: [PATCH] SSL server certificate validator implementation

2012-12-03 Thread Tsantilas Christos
On 11/24/2012 04:23 PM, Tsantilas Christos wrote: A new patch. This patch includes most (/all I hope) of Amos requested changes. If there is no objection I will commit the latest patch (cert_validator-v3.patch) to trunk. On 11/22/2012 02:10 PM, Amos Jeffries wrote: On 22/11/2012 9:14

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Henrik Nordström
mån 2012-12-03 klockan 09:07 -0700 skrev Alex Rousskov: Yes. IIRC, it has been broken before my AsyncCall changes to the main loop. The sawActivity code is from the AsyncCall merge in rev 8812. This will poll non-comm event loops until they are completely idle having nothing to do.

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Alex Rousskov
On 12/03/2012 02:01 PM, Henrik Nordström wrote: mån 2012-12-03 klockan 09:07 -0700 skrev Alex Rousskov: I was going to agree with that wholeheartedly, but then I thought about signals. If we visit each non-waiting engine once, then we will only process signals once per main loop step. Is that

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Henrik Nordström
mån 2012-12-03 klockan 15:53 -0700 skrev Alex Rousskov: True (although the current small maximum select(2) delay is a result of bugs elsewhere in the code and should not really be there). If we can do the right thing here easily, we should (instead of adding more future problems). And I think

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Amos Jeffries
On 04.12.2012 05:07, Alex Rousskov wrote: On 12/03/2012 01:44 AM, Henrik Nordström wrote: The store rebuilding does eventAdd(storeRebuild, RebuildStep, rb, 0.01, 1); which says both delay = 0.01, avoid saturating things too much weight = 1, this is a heavy job, slow down event.cc

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Henrik Nordström
mån 2012-12-03 klockan 15:53 -0700 skrev Alex Rousskov: We just need to make sure that heavy events (certain timed events and signals) interrupt the sawActivity loop. This will fix the issue you are facing without introducing new problems. Another question on this.. how to fit the concept of

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Henrik Nordström
mån 2012-12-03 klockan 09:07 -0700 skrev Alex Rousskov: Could you please fix this properly by adding AsyncEngine::timeTillNextEvent() or similar API and removing loop_delay manipulation from checkEvents() as discussed above? The attached patch adds AsyncEngine::timeTillNextEvent() API. It

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Alex Rousskov
On 12/03/2012 04:16 PM, Henrik Nordström wrote: mån 2012-12-03 klockan 15:53 -0700 skrev Alex Rousskov: Ah, I see another problem with the each non-waiting engine runs once approach. One engine may create work for other engines or for itself, including async calls and run this now lightweight

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Alex Rousskov
On 12/03/2012 04:36 PM, Henrik Nordström wrote: mån 2012-12-03 klockan 15:53 -0700 skrev Alex Rousskov: We just need to make sure that heavy events (certain timed events and signals) interrupt the sawActivity loop. This will fix the issue you are facing without introducing new problems.

Re: Multiple issues in Squid-3.2.3 SMP + rock + aufs + a bit of load

2012-12-03 Thread Alex Rousskov
On 12/03/2012 04:18 PM, Amos Jeffries wrote: Would it make sense to do one of: * have the AsyncCall queue drain with a timeout. So that no sequence of calls can lag any of the sensitive engines too much. That along with the event engines scheduling their heavyweight actions as Calls instead