Re: idleTimeout values are getting reset

2018-04-26 Thread Sandeep Moré
You can use the class ProxyInboundSocket.java, this was the one included
with the original commit.
In there find the onClientOpen i.e.

 @OnOpen
  public void onClientOpen(final javax.websocket.Session
backendSession) {


and update 'backendSession' as follows

backendSession.setMaxIdleTimeout(6 * 60 * 24);

I would highly recommend to upgrade since your custom settings will be lost
once you upgrade and it will be difficult to keep track as this class does
not exist any more.

Best,
Sandeep


On Thu, Apr 26, 2018 at 1:27 AM, Ashok V Jose <ashoj...@in.ibm.com> wrote:

> Hi Sandeep,
>
> The knox version (0.12.0.2.6.2.0-205) I am using did not have this file
> ProxyInboundClient.java
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_knox_blob_92e2ec59a5940a9e7c67ec5cd29044f811dee40a_gateway-2Dserver_src_main_java_org_apache_knox_gateway_websockets_ProxyInboundClient.java-23L61=DwMFaQ=jf_iaSHvJObTbx-siA1ZOg=IwA-Q1lswMrNoI5t1-MA_ksLuVkVEL61cf3yFoZAQhs=DdBwaq-A5CjsAq6juBCXi4pLgWL1KtMZOv3_IB7x0FQ=ZDrjApvcdgx2U9mY2vzxvz_20rLGqy0268qKih_frAc=>,
> is there a way I can put in somewhere in my verison ?
>
>
> Thanks & Regards,
> Ashok Jose
>
> **
>
> BigInsights Quality
> EGL C Block, 6th Floor
> IBM India Software Labs,
> Mail-Id : ashoj...@in.ibm.com
>
>
>
>
> - Original message -
> From: "Sandeep Moré" <moresand...@gmail.com>
> To: user@knox.apache.org
> Cc:
> Subject: Re: idleTimeout values are getting reset
> Date: Tue, Apr 24, 2018 12:02 AM
>
> Hello Ashok,
>
> Unfortunately, there is only one setting as of now, you can try adding
> session.setMaxIdleTimeout(98);
> at https://github.com/apache/knox/blob/92e2ec59a5940a9e7c67ec5cd29044
> f811dee40a/gateway-server/src/main/java/org/apache/knox/
> gateway/websockets/ProxyInboundClient.java#L61
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_knox_blob_92e2ec59a5940a9e7c67ec5cd29044f811dee40a_gateway-2Dserver_src_main_java_org_apache_knox_gateway_websockets_ProxyInboundClient.java-23L61=DwMFaQ=jf_iaSHvJObTbx-siA1ZOg=IwA-Q1lswMrNoI5t1-MA_ksLuVkVEL61cf3yFoZAQhs=DdBwaq-A5CjsAq6juBCXi4pLgWL1KtMZOv3_IB7x0FQ=ZDrjApvcdgx2U9mY2vzxvz_20rLGqy0268qKih_frAc=>
>
> and see if that helps.
>
> Best,
> Sandeep
>
>
> On Sun, Apr 22, 2018 at 9:17 PM, Ashok V Jose <ashoj...@in.ibm.com>
> wrote:
>
> Hi Sandeep,
>
> Is there a parameter that I can set the timeouts for both 
> WebSocketServerConnection
> and WebSocketClientConnection to a larger value from the knox settings ?
> One of these is gateway.websocket.idle.timeout, is there an other one ?
> Or are these set from the client program ?
>
>
> Thanks & Regards,
> Ashok Jose
>
> **
>
> BigInsights Quality
> EGL C Block, 6th Floor
> IBM India Software Labs,
> Mail-Id : ashoj...@in.ibm.com
>
>
>
>
> - Original message -
> From: "Sandeep Moré" <moresand...@gmail.com>
> To: user@knox.apache.org
> Cc:
> Subject: Re: idleTimeout values are getting reset
> Date: Thu, Apr 19, 2018 7:15 PM
>
> Hello Ashok,
>
> This is interesting, Knox opens up two connections for Websockets, one on
> the client side (browser/client etc.) and other on the backend (service
> host).
> The timeouts you are seeing are for the two
> 1. First debug line - WebSocketServerConnection - opens up websocket
> connection to the backend
> 2. Second debug line - WebSocketClientConnection - opens up websocket to
> the browser
>
> As far as the timeouts are concerned the shortest one wins (and can be
> configured), in this case 98, since the connection will be closed and the
> error value relayed (or should be).
>
> I am curious why you are reducing the timeout value if you are getting
> timeout errors, shouldn't you be increasing them, or perhaps you are just
> debugging.
>
> Best,
> Sandeep
>
>
>
> On Thu, Apr 19, 2018 at 7:04 AM, Ashok V Jose <ashoj...@in.ibm.com>
> wrote:
>
> Hi Knox Experts,
>
> Needed a help incase you are aware of this issue.
>
> We have this scenario where in we are creating 20 python kernels
> concurrently, we see that of these around 6-7 kernels fail with "Exception:
> Remote host closed websocket" because of timeout.
>
> When we tried the same testcase outside of the knox, we did not see the
> issue. All the kernels got created ok.
>
> So we suspected something in knox which is causing the issue, since the
> failure is because of the timeout imposed, from the ambari console we
> changed these 2 parameters  “gateway.idle.timeout" and
> 

Re: idleTimeout values are getting reset

2018-04-25 Thread Ashok V Jose
Hi Sandeep,The knox version (0.12.0.2.6.2.0-205) I am using did not have this file  ProxyInboundClient.java, is there a way I can put in somewhere in my verison ? 
 
 
Thanks & Regards,Ashok Jose**BigInsights QualityEGL C Block, 6th FloorIBM India Software Labs,Mail-Id : ashoj...@in.ibm.com 
 
 
- Original message -From: "Sandeep Moré" To: user@knox.apache.orgCc:Subject: Re: idleTimeout values are getting resetDate: Tue, Apr 24, 2018 12:02 AM 
Hello Ashok,
 
Unfortunately, there is only one setting as of now, you can try adding session.setMaxIdleTimeout(98); 

at https://github.com/apache/knox/blob/92e2ec59a5940a9e7c67ec5cd29044f811dee40a/gateway-server/src/main/java/org/apache/knox/gateway/websockets/ProxyInboundClient.java#L61
 
and see if that helps.
 
Best,
Sandeep
 
 
On Sun, Apr 22, 2018 at 9:17 PM, Ashok V Jose  wrote:

Hi Sandeep,
 
Is there a parameter that I can set the timeouts for both WebSocketServerConnection and WebSocketClientConnection to a larger value from the knox settings ? One of these is gateway.websocket.idle.timeout, is there an other one ?
Or are these set from the client program ? 
 
 
Thanks & Regards,Ashok Jose**BigInsights QualityEGL C Block, 6th FloorIBM India Software Labs,Mail-Id : ashoj...@in.ibm.com 
 
 
- Original message -From: "Sandeep Moré" To: user@knox.apache.orgCc:Subject: Re: idleTimeout values are getting resetDate: Thu, Apr 19, 2018 7:15 PM 
Hello Ashok,
 
This is interesting, Knox opens up two connections for Websockets, one on the client side (browser/client etc.) and other on the backend (service host). 
The timeouts you are seeing are for the two 
1. First debug line - WebSocketServerConnection - opens up websocket connection to the backend
2. Second debug line - WebSocketClientConnection - opens up websocket to the browser 
 
As far as the timeouts are concerned the shortest one wins (and can be configured), in this case 98, since the connection will be closed and the error value relayed (or should be).
 
I am curious why you are reducing the timeout value if you are getting timeout errors, shouldn't you be increasing them, or perhaps you are just debugging. 
 
Best,
Sandeep
 
 
 
On Thu, Apr 19, 2018 at 7:04 AM, Ashok V Jose  wrote:

Hi Knox Experts,
 
Needed a help incase you are aware of this issue.
 
We have this scenario where in we are creating 20 python kernels concurrently, we see that of these around 6-7 kernels fail with "Exception: Remote host closed websocket" because of timeout.
 
When we tried the same testcase outside of the knox, we did not see the issue. All the kernels got created ok.
 
So we suspected something in knox which is causing the issue, since the failure is because of the timeout imposed, from the ambari console we changed these 2 parameters  “gateway.idle.timeout" and "gateway.websocket.idle.timeout" to different values and ran the testcase.
 
However we see in the knox logs that although we have set the "gateway.websocket.idle.timeout" to 98 it is being reset to 30.
 
We see that idleTimeout is 98 , however it resets and becomes 30 later on.
 
So basically it looks like whatever we set is being overwritten to 30, is this something you would know ?
 
Here are the 2 lines which shows the values of idleTimeout to 98 and 30.
2018-04-13 09:17:38,266 DEBUG events.AbstractEventDriver (AbstractEventDriver.java:openSession(223)) - openSession(WebSocketSession[websocket=JettyListenerEventDriver[org.apache.hadoop.gateway.websockets.ProxyWebSocketAdapter],behavior=SERVER,connection=WebSocketServerConnection@3016d0df[IDLE,DecryptedEndPoint@65fc705{/xx.xxx.xxx.xxx:x<->8443,Open,in,out,-,-,116/98,WebSocketServerConnection}->SelectChannelEndPoint@2dd64f54{/xx.xxx.xxx.xxx:x<->8443,Open,in,out,-,-,1/98,SslConnection}{io=0,kio=0,kro=1}]{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@faa724f[ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@430a35ac[behavior=SERVER,maxTextMessageSize=2147483647,maxTextMessageBufferSize=32768,maxBinaryMessageSize=2147483647,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=6,idleTimeout=98,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@63617fae[batching=true],incoming=JettyListenerEventDriver[org.apache.hadoop.gateway.websockets.ProxyWebSocketAdapter],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]])
 
2018-04-13 09:17:40,535 DEBUG component.ContainerLifeCycle (ContainerLifeCycle.java:addBean(324)) - 

Re: idleTimeout values are getting reset

2018-04-22 Thread Ashok V Jose
Hi Sandeep,
 
Is there a parameter that I can set the timeouts for both WebSocketServerConnection and WebSocketClientConnection to a larger value from the knox settings ? One of these is gateway.websocket.idle.timeout, is there an other one ?
Or are these set from the client program ? 
 
 
Thanks & Regards,Ashok Jose**BigInsights QualityEGL C Block, 6th FloorIBM India Software Labs,Mail-Id : ashoj...@in.ibm.com 
 
 
- Original message -From: "Sandeep Moré" To: user@knox.apache.orgCc:Subject: Re: idleTimeout values are getting resetDate: Thu, Apr 19, 2018 7:15 PM 
Hello Ashok,
 
This is interesting, Knox opens up two connections for Websockets, one on the client side (browser/client etc.) and other on the backend (service host). 
The timeouts you are seeing are for the two 
1. First debug line - WebSocketServerConnection - opens up websocket connection to the backend
2. Second debug line - WebSocketClientConnection - opens up websocket to the browser 
 
As far as the timeouts are concerned the shortest one wins (and can be configured), in this case 98, since the connection will be closed and the error value relayed (or should be).
 
I am curious why you are reducing the timeout value if you are getting timeout errors, shouldn't you be increasing them, or perhaps you are just debugging. 
 
Best,
Sandeep
 
 
 
On Thu, Apr 19, 2018 at 7:04 AM, Ashok V Jose  wrote:

Hi Knox Experts,
 
Needed a help incase you are aware of this issue.
 
We have this scenario where in we are creating 20 python kernels concurrently, we see that of these around 6-7 kernels fail with "Exception: Remote host closed websocket" because of timeout.
 
When we tried the same testcase outside of the knox, we did not see the issue. All the kernels got created ok.
 
So we suspected something in knox which is causing the issue, since the failure is because of the timeout imposed, from the ambari console we changed these 2 parameters  “gateway.idle.timeout" and "gateway.websocket.idle.timeout" to different values and ran the testcase.
 
However we see in the knox logs that although we have set the "gateway.websocket.idle.timeout" to 98 it is being reset to 30.
 
We see that idleTimeout is 98 , however it resets and becomes 30 later on.
 
So basically it looks like whatever we set is being overwritten to 30, is this something you would know ?
 
Here are the 2 lines which shows the values of idleTimeout to 98 and 30.
2018-04-13 09:17:38,266 DEBUG events.AbstractEventDriver (AbstractEventDriver.java:openSession(223)) - openSession(WebSocketSession[websocket=JettyListenerEventDriver[org.apache.hadoop.gateway.websockets.ProxyWebSocketAdapter],behavior=SERVER,connection=WebSocketServerConnection@3016d0df[IDLE,DecryptedEndPoint@65fc705{/xx.xxx.xxx.xxx:x<->8443,Open,in,out,-,-,116/98,WebSocketServerConnection}->SelectChannelEndPoint@2dd64f54{/xx.xxx.xxx.xxx:x<->8443,Open,in,out,-,-,1/98,SslConnection}{io=0,kio=0,kro=1}]{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[SERVER,validating],p=Parser@faa724f[ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@430a35ac[behavior=SERVER,maxTextMessageSize=2147483647,maxTextMessageBufferSize=32768,maxBinaryMessageSize=2147483647,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=6,idleTimeout=98,inputBufferSize=4096]]},remote=WebSocketRemoteEndpoint@63617fae[batching=true],incoming=JettyListenerEventDriver[org.apache.hadoop.gateway.websockets.ProxyWebSocketAdapter],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.eclipse.jetty.websocket.server.WebSocketServerConnection]])
 
2018-04-13 09:17:40,535 DEBUG component.ContainerLifeCycle (ContainerLifeCycle.java:addBean(324)) - WebSocketSession[websocket=JsrAnnotatedEventDriver[websocket=org.apache.hadoop.gateway.websockets.ProxyInboundSocket@66217bd6],behavior=CLIENT,connection=WebSocketClientConnection@1fcb3472[IDLE,SelectChannelEndPoint@627d9a4b{chs-ksj-013-mn003.bi.services.us-south.bluemix.net/172.16.12.7:<->38356,Open,in,out,-,-,1/30,UpgradeConnection}{io=0,kio=0,kro=1}]{f=Flusher[queueSize=0,aggregateSize=0,failure=null],g=Generator[CLIENT,validating],p=Parser@1eafcb18[ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@4e802544[behavior=CLIENT,maxTextMessageSize=2147483647,maxTextMessageBufferSize=32768,maxBinaryMessageSize=2147483647,maxBinaryMessageBufferSize=32768,asyncWriteTimeout=6,idleTimeout=30,inputBufferSize=4096]]},remote=null,incoming=JsrAnnotatedEventDriver[websocket=org.apache.hadoop.gateway.websockets.ProxyInboundSocket@66217bd6],outgoing=ExtensionStack[queueSize=0,extensions=[],incoming=org.eclipse.jetty.websocket.jsr356.JsrSession,outgoing=org.eclipse.jetty.websocket.client.io.WebSocketClientConnection]] added 

Re: idleTimeout values are getting reset

2018-04-19 Thread Sandeep Moré
Hello Ashok,

This is interesting, Knox opens up two connections for Websockets, one on
the client side (browser/client etc.) and other on the backend (service
host).
The timeouts you are seeing are for the two
1. First debug line - WebSocketServerConnection - opens up websocket
connection to the backend
2. Second debug line - WebSocketClientConnection - opens up websocket to
the browser

As far as the timeouts are concerned the shortest one wins (and can be
configured), in this case 98, since the connection will be closed and the
error value relayed (or should be).

I am curious why you are reducing the timeout value if you are getting
timeout errors, shouldn't you be increasing them, or perhaps you are just
debugging.

Best,
Sandeep



On Thu, Apr 19, 2018 at 7:04 AM, Ashok V Jose  wrote:

> Hi Knox Experts,
>
> Needed a help incase you are aware of this issue.
>
> We have this scenario where in we are creating 20 python kernels
> concurrently, we see that of these around 6-7 kernels fail with "Exception:
> Remote host closed websocket" because of timeout.
>
> When we tried the same testcase outside of the knox, we did not see the
> issue. All the kernels got created ok.
>
> So we suspected something in knox which is causing the issue, since the
> failure is because of the timeout imposed, from the ambari console we
> changed these 2 parameters  “gateway.idle.timeout" and
> "gateway.websocket.idle.timeout" to different values and ran the testcase.
>
> However we see in the knox logs that although we have set the
> "gateway.websocket.idle.timeout" to 98 it is being reset to 30.
>
> We see that idleTimeout is 98 , however it resets and becomes 30 later
> on.
>
> So basically it looks like whatever we set is being overwritten to 30,
> is this something you would know ?
>
> Here are the 2 lines which shows the values of idleTimeout to 98 and
> 30.
>
> 2018-04-13 09:17:38,266 DEBUG events.AbstractEventDriver
> (AbstractEventDriver.java:openSession(223)) -
> openSession(WebSocketSession[websocket=JettyListenerEventDriver[org.
> apache.hadoop.gateway.websockets.ProxyWebSocketAdapter],
> behavior=SERVER,connection=WebSocketServerConnection@3016d0df[IDLE,
> DecryptedEndPoint@65fc705{/xx.xxx.xxx.xxx:x<->8443,Open,
> in,out,-,-,116/98,WebSocketServerConnection}->
> SelectChannelEndPoint@2dd64f54{/xx.xxx.xxx.xxx:x<->8443,
> Open,in,out,-,-,1/98,SslConnection}{io=0,kio=0,kro=
> 1}]{f=Flusher[queueSize=0,aggregateSize=0,failure=null],
> g=Generator[SERVER,validating],p=Parser@faa724f[
> ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@430a35ac[
> behavior=SERVER,maxTextMessageSize=2147483647,maxTextMessageBufferSize=
> 32768,maxBinaryMessageSize=2147483647,maxBinaryMessageBufferSize=
> 32768,asyncWriteTimeout=6,idleTimeout=98,inputBufferSize=4096]]},
> remote=WebSocketRemoteEndpoint@63617fae[batching=true],incoming=
> JettyListenerEventDriver[org.apache.hadoop.gateway.websockets.
> ProxyWebSocketAdapter],outgoing=ExtensionStack[queueSize=0,extensions=[],
> incoming=org.eclipse.jetty.websocket.common.WebSocketSession,outgoing=org.
> eclipse.jetty.websocket.server.WebSocketServerConnection]])
>
> 2018-04-13 09:17:40,535 DEBUG component.ContainerLifeCycle
> (ContainerLifeCycle.java:addBean(324)) - WebSocketSession[websocket=
> JsrAnnotatedEventDriver[websocket=org.apache.hadoop.gateway.websockets.
> ProxyInboundSocket@66217bd6],behavior=CLIENT,connection=
> WebSocketClientConnection@1fcb3472[IDLE,SelectChannelEndPoint@627d9a4b{
> chs-ksj-013-mn003.bi.services.us-south.bluemix.net/172.16.12.7:
> <->38356,Open,in,out,-,-,1/30,UpgradeConnection}{io=0,kio=0,
> kro=1}]{f=Flusher[queueSize=0,aggregateSize=0,failure=null],
> g=Generator[CLIENT,validating],p=Parser@1eafcb18[
> ExtensionStack,s=START,c=0,len=0,f=null,p=WebSocketPolicy@4e802544[
> behavior=CLIENT,maxTextMessageSize=2147483647,maxTextMessageBufferSize=
> 32768,maxBinaryMessageSize=2147483647,maxBinaryMessageBufferSize=
> 32768,asyncWriteTimeout=6,idleTimeout=30,inputBufferSize=4096]]},
> remote=null,incoming=JsrAnnotatedEventDriver[websocket=org.apache.hadoop.
> gateway.websockets.ProxyInboundSocket@66217bd6],outgoing=ExtensionStack[
> queueSize=0,extensions=[],incoming=org.eclipse.jetty.
> websocket.jsr356.JsrSession,outgoing=org.eclipse.jetty.websocket.client.io
> .WebSocketClientConnection]] added {ExtensionStack[queueSize=0,
> extensions=[],incoming=org.eclipse.jetty.websocket.
> jsr356.JsrSession,outgoing=org.eclipse.jetty.websocket.client.io.
> WebSocketClientConnection],AUTO}
>
> This are the versions  : Apache Knox: 0.12.0.2.6.2.0-205 (
> c8ad94e5bd29a6925f5a667660a50cfc6de69cc3)
>   HDP-2.6.2.0-205
>
> Please let me know if anything more would help
>
>
> Thanks & Regards,
> Ashok Jose
>
> **
>
> BigInsights Quality
> EGL C Block, 6th Floor
> IBM India Software Labs,
> Mail-Id :