Re: [naviserver-devel] ns_proxy hang

2017-05-29 Thread David Osborne
Thanks very much Gustaf! We'll schedule a build and let you know how it goes. On 25 May 2017 at 11:32, Gustaf Neumann wrote: > > but i will be looking into this when times allows. > David, There are now some updates in the repository, removing falso > positives about zombies etc. for eval-timout

Re: [naviserver-devel] ns_proxy hang

2017-05-25 Thread Gustaf Neumann
> but i will be looking into this when times allows. David, There are now some updates in the repository, removing falso positives about zombies etc. for eval-timouted nsproxy cases -g -- Check out the vibrant tech commun

Re: [naviserver-devel] ns_proxy hang

2017-05-17 Thread Gustaf Neumann
Am 17.05.17 um 11:58 schrieb David Osborne: > Thanks Gustaf. > We've tested and rolled this change out now and it's looking good. Great, good to know! i am not happy about the warnings/error messages, as it looks to me, getting rid of these would require some API changes to report a more detailed

Re: [naviserver-devel] ns_proxy hang

2017-05-17 Thread David Osborne
Thanks Gustaf. We've tested and rolled this change out now and it's looking good. Regards On 15 May 2017 at 21:01, Gustaf Neumann wrote: > Am 15.05.17 um 17:54 schrieb David Osborne: > > In some cases (but not all), Ns_WaitProcess is logging an error > > immediately after the reaper sends a sig

Re: [naviserver-devel] ns_proxy hang

2017-05-15 Thread Gustaf Neumann
Am 15.05.17 um 17:54 schrieb David Osborne: > In some cases (but not all), Ns_WaitProcess is logging an error > immediately after the reaper sends a signal 15. > > It's under these circumstances ReleaseProxy seems to find the proxy > Busy without a slave. > > [15/May/2017:16:38:39][23802.7f2f0d7

Re: [naviserver-devel] ns_proxy hang

2017-05-15 Thread David Osborne
In some cases (but not all), Ns_WaitProcess is logging an error immediately after the reaper sends a signal 15. It's under these circumstances ReleaseProxy seems to find the proxy Busy without a slave. [15/May/2017:16:38:39][23802.7f2f0d7fa700][-nsproxy:reap-] Warning: [testpool]: pid 5174 won't

Re: [naviserver-devel] ns_proxy hang

2017-05-15 Thread David Osborne
Thanks - that stops the seg fault... it does seem to leave a lot of zombies though. This is the output during the ns_proxy test: [15/May/2017:15:03:47][7028.75c3b700][-command-] Notice: releasing busy proxy testpool-8 [15/May/2017:15:03:48][7028.7fffc67fc700][-nsproxy:reap-] Warning: [testpool

Re: [naviserver-devel] ns_proxy hang

2017-05-15 Thread Gustaf Neumann
Am 15.05.17 um 15:15 schrieb David Osborne: > Thanks. > > This does indeed seem to fix the test case I sent - good > however I'm now getting a segmentation fault during the "make test". bad i've made the code more robust although it is not clear to me yet, how the proxy can be "Busy" without

Re: [naviserver-devel] ns_proxy hang

2017-05-15 Thread David Osborne
Thanks. This does indeed seem to fix the test case I sent - however I'm now getting a segmentation fault during the "make test". It's during the second ns_proxy 5.10 test (there are two 5.10s) - "test ns_proxy-5.10 {check killing active proxy}". ns_close is being called on an invalid SlavePtr du

Re: [naviserver-devel] ns_proxy hang

2017-05-13 Thread Gustaf Neumann
Hi David, i've committed a version to bitbucket, that should address the problem. Here is what's seems to happen: a) in your example, you are sending exec commands with huge output and a eval-timeout of 1ms b) NaviServer stops it side the eval more or less immediately (after one ms) c) the slave

Re: [naviserver-devel] ns_proxy hang

2017-05-13 Thread Maurizio Martignano
flow.7] pointer arithmetic overflow on + in ptr + n: SUCCESS From: David Osborne [mailto:da...@qcode.co.uk] Sent: 12 May 2017 18:19 To: naviserver-devel@lists.sourceforge.net Subject: Re: [naviserver-devel] ns_proxy hang Not sure if this helps you - it may be entirely expected behaviour, but I at

Re: [naviserver-devel] ns_proxy hang

2017-05-12 Thread David Osborne
Not sure if this helps you - it may be entirely expected behaviour, but I attached gbp to the nsproxy process and got a backtrace from the 2 test cases I mentioned, immediately after the "ns_proxy eval...". In the case with no hang the nsproxy process is waiting in RecvBuf: #0 0x76593250

Re: [naviserver-devel] ns_proxy hang

2017-05-10 Thread Gustaf Neumann
I'll look at it at the weekend - unless someone else can fix this before this. -g Am 10.05.17 um 16:00 schrieb David Osborne: Increasing waittimeout doesn't seem to have any effect on this problem. I have backtraces of all threads at the point of the hang here: https://gist.github.com/davidqc

Re: [naviserver-devel] ns_proxy hang

2017-05-10 Thread David Osborne
Increasing waittimeout doesn't seem to have any effect on this problem. I have backtraces of all threads at the point of the hang here: https://gist.github.com/davidqc/ebee38528b0a40a0b8d028981ad933e6 Thread 19 I think is the culprit: Thread 19 (Thread 0x7fffaaffd700 (LWP 17652)): #0 0x7ff

Re: [naviserver-devel] ns_proxy hang

2017-05-10 Thread Gustaf Neumann
We don't see such hangs either. Does increasing the waittimeout solve this issue? ... not as a fix, but to narrow the problem down. -gn Am 10.05.17 um 13:38 schrieb David Osborne: The manifestation for us in production is quite insidious and difficult to spot the root cause of. Certainly not ea

Re: [naviserver-devel] ns_proxy hang

2017-05-10 Thread David Osborne
The manifestation for us in production is quite insidious and difficult to spot the root cause of. Certainly not easy to see via the logs. One place we are experiencing it is in code which generates outgoing emails. Sometimes, when the outgoing email is particularly large, nsproxy times out while

Re: [naviserver-devel] ns_proxy hang

2017-05-10 Thread Gustaf Neumann
Dear David, i have not been able to look into this yet. However, i've checked our logs on several servers using nsproxy intensively, but did not see any problems. How is the proxy setup (config section and "ns_proxy configure" command) in your environment? -g Am 10.05.17 um 10:50 schrieb Dav

Re: [naviserver-devel] ns_proxy hang

2017-05-10 Thread David Osborne
Has anyone got any ideas on debugging this issue? Turns out it's causing us many more operational issues that we first thought... Regards, -- David On 18 April 2017 at 15:44, David Osborne wrote: > Hi, > > We're encountering an occasional problem which means calls to ns_proxy > hang forever. >