Re: HTTP communication
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Maxim, On 2/1/16 4:53 AM, Maxim Neshcheret wrote: > There is such option in my app (heartbeats from server to client) > but I’m curious why connection becomes stale if I don’t send any > heartbeats. Any networking component can decide that a connection has been open for too long. It could be the OS's TCP/IP stack, a software firewall on either end, etc. Or the client could have gone to sleep (e.g. OS sleep mode). Your heartbeats keep it from going to sleep , maybe? If you want a 10-day connection to stay open, heartbeats are the only way to do it. Presumably, you can catch the heartbeat failure? Under that condition, take appropriate action. - -chris > On 29/01/16 16:04, "Christoph Nenning" > wrote: > >>> I have a problem with my java application related to HTTP >>> communication. >>> >>> Application description: >>> >>> 1. Client – server. Server is running in servlet >>> container. We use Tomcat. >>> >>> Client use java HTTP library to communicate with the server. >>> >>> 2. When client establish connection to the server it sends >>> GET request (keepalive) and server creates AsyncContext for the >>> client with 10 days timeout. >>> >>> 3. After connection established server periodically sends >>> data to the client using AsyncContext and on the client side >>> there is special thread which reads and processes the data. >>> >>> Client reading thread example: >>> >>> private class GetRunnable implements Runnable { >>> >>> private final HttpURLConnection httpConn; private final >>> MyConnection myConn; >>> >>> public GetRunnable(final MyConnection myConn) throws >>> IOException, AppException { this.myConn = myConn; final String >>> jSession = myConn.getJSession(); httpConn = >>> openHttpUrlConnection(false, httpPostTimeout, jSession); >>> httpConn.connect(); final int respCode = >>> httpConn.getResponseCode(); if (respCode != >>> HttpURLConnection.HTTP_OK) { String info = "Incorrect response >>> from server, responseCode:" + respCode + ", message:" + >> httpConn.getResponseMessage(); >>> log.error(info); throw new AppException(info); } else { >>> log.trace("GET connected"); } } >>> >>> @Override public void run() { try (final BufferedReader reader >>> = new BufferedReader (new >>> InputStreamReader(httpConn.getInputStream(), "UTF-8")) ) { >>> log.info("doGet STARTED"); final Thread t = >>> Thread.currentThread(); while (!t.isInterrupted()) { >>> log.trace("before readline"); final String line = >>> reader.readLine(); log.trace("after readline: [" + line + >>> "]"); >>> >>> //INFO: DATA PROCESSING HERE } } catch (IOException e) { >>> log.error("Error while doGet"); } catch (Throwable e) { >>> log.debug("doGet STOPED..."); log.error("Error while read input >>> msg, do logoff", >> e); >>> logoff(); throw e; } log.debug("doGet STOPED..."); >>> myCallback.onGetClose(myConn, connInfo); } } >>> >>> Server side code to push data looks like this: >>> >>> protected int sendMsgs(final MyConnection myConn, final String >>> info) { >>> >>> int res; try { final AsyncContext lAc = _ac; final >>> ServletRequest req = lAc.getRequest(); if >>> (!req.isAsyncStarted()) { log.debug("AsyncStarted=false on >>> begin, uid:" + uid + ", sid:" + sid); return -1; } >>> >>> final ServletResponse res = lAc.getResponse(); >>> ServletOutputStream os = null; final String text = >>> getData(myConn); >>> >>> if (os == null) os = res.getOutputStream(); >>> >>> write2stream(os, text, 0x4000); os.println(); >>> >>> if (os != null) os.flush(); } catch(IOException ex) { >>> log.error("Notify failed"); } catch (InterruptedException e) { >>> log.error("Notify failed"); >>> Thread.currentThread().interrupt(); } catch (Throwable e) { >>> log.error("Notify failed"); logoff(true); } >>> >>> return res; } >>> >>> public static void write2stream(final OutputStream >>> outputStream, final String text, int bufferSize) throws >>> IOException { final CharsetEncoder encoder = >>> getHttpEncodingCharset ().newEn
Re: HTTP communication
Hi Olaf, Just repeat here what I have already replied to André’s comment. “First about network. Client and server are in the same segment and there are no firewalls in between them. Second I expect connection to be alive because during initiation I specify 'keep-alive' property of HTTP request. Also Tomcat and WLS are configured to support keepalive connections. From what I noticed, I monitored TCP connection using TCPView and Wireshark (on the client side), TCP connection is still alive even Tomcat or WLS cannot write to the socket. Just to mention, I have in my code an option to send periodical heartbeats to the client if there is no data. It is configurable and when I use this option everything works fine. But if I don’t send heartbeats as I wrote before connection becomes stale and I want to understand why it happens so that I can initiate, for example, automatic client logoff and cleanup resources.” On 29/01/16 16:13, "Olaf Kock" wrote: >I'll second Andre's answer: Just because you declare a 10d timeout, you >can't rely on the connection to stay up for that long. You can't even >rely on a connection to stay up during the download of a simple gif >(although that's so quick that the odds for connection termination are a >lot lower). > >This is without looking at the code - just relying on a connection to >stay up is not a proper assumption. You'll have to work around anything >in between to fail and reconnect if necessary. Declaring a 10d timeout - >if it works - will cause the appropriately configured endpoint to assume >it's open. It has no effect on any of the components in between the >endpoints. > >Olaf > >Am 29.01.2016 um 09:33 schrieb André Warnier (tomcat): >> On 28.01.2016 18:38, Maxim Neshcheret wrote: >>> I have a problem with my java application related to HTTP >>>communication. >>> >>> Application description: >>> >>> 1. Client – server. Server is running in servlet container. We >>> use Tomcat. >>> >>> Client use java HTTP library to communicate with the server. >>> >>> 2. When client establish connection to the server it sends GET >>> request (keepalive) and server creates AsyncContext for the client >>> with 10 days timeout. >>> >>> 3. After connection established server periodically sends data >>> to the client using AsyncContext and on the client side there is >>> special thread which reads and processes the data. >>> >> [ snip ...] >>> >>> Usually this code works fine but it there is no data from server to >>> client for 1 day and after 24 hours (can be 16 or 12) data appears, >>> server cannot send data to the client. >>> In the log there is no error. It is written that everything flushed >>> but client still waiting for data in “final String line = >>> reader.readLine();” >>> When 2nd portion of data is sent by the server, then during flush I >>> see the following error >>> >>> 2016-01-26 00:00:00,051|INFO |GWNotify-2/50 |ClientAbort >>> 2016-01-26 00:00:00,051|TRACE|GWNotify-2/50 >>> |ClientAbortException:java.io.IOException: APR error: -32 >>> org.apache.catalina.connector.ClientAbortException: >>> java.io.IOException: APR error: -32 >> [snip ...] >> >> Hi. >> I am unqualified to check your code, but a first question would be : >> where is the Client, and where is the Server and what is the >> connection between them like ? >> >> And the reason for the question is : it is not at all unusual, that >> any kind of network connection would be interrupted at some point over >> a 24-hour period, specially if nothing is sent over that connection >> for a long time. >> When a connection "disappears", TCP sends no signal to either the >> client or the server, and an error will only be caught, if one of the >> parties tries to write to the (now gone) connection (which seems to be >> what happens above, when the server tries to write to the Client, and >> gets a "client is no longer there" exception). >> >> If you want to avoid this, you will have to handle this in your code. >> You cannot just expect the connection to be alive no matter what. >> >> >> >> >> - >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> > > >- >To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >For additional commands, e-mail: users-h...@tomcat.apache.org >
Re: HTTP communication
Hi Christoph There is such option in my app (heartbeats from server to client) but I’m curious why connection becomes stale if I don’t send any heartbeats. On 29/01/16 16:04, "Christoph Nenning" wrote: >> I have a problem with my java application related to HTTP communication. >> >> Application description: >> >> 1. Client – server. Server is running in servlet container. We >> use Tomcat. >> >> Client use java HTTP library to communicate with the server. >> >> 2. When client establish connection to the server it sends GET >> request (keepalive) and server creates AsyncContext for the client >> with 10 days timeout. >> >> 3. After connection established server periodically sends data >> to the client using AsyncContext and on the client side there is >> special thread which reads and processes the data. >> >> Client reading thread example: >> >> private class GetRunnable implements Runnable { >> >>private final HttpURLConnection httpConn; >>private final MyConnection myConn; >> >>public GetRunnable(final MyConnection myConn) throws >> IOException, AppException { >> this.myConn = myConn; >> final String jSession = myConn.getJSession(); >> httpConn = openHttpUrlConnection(false, >> httpPostTimeout, jSession); >> httpConn.connect(); >> final int respCode = httpConn.getResponseCode(); >> if (respCode != HttpURLConnection.HTTP_OK) { >> String info = "Incorrect response from server, >> responseCode:" + respCode + ", message:" + >httpConn.getResponseMessage(); >> log.error(info); >> throw new AppException(info); >> } else { >> log.trace("GET connected"); >> } >>} >> >>@Override >>public void run() { >> try (final BufferedReader reader = new BufferedReader >> (new InputStreamReader(httpConn.getInputStream(), "UTF-8")) ) { >> log.info("doGet STARTED"); >> final Thread t = Thread.currentThread(); >> while (!t.isInterrupted()) { >>log.trace("before readline"); >>final String line = reader.readLine(); >>log.trace("after readline: [" + line + "]"); >> >>//INFO: DATA PROCESSING HERE >> } >> } catch (IOException e) { >> log.error("Error while doGet"); >> } catch (Throwable e) { >> log.debug("doGet STOPED..."); >> log.error("Error while read input msg, do logoff", >e); >> logoff(); >> throw e; >> } >> log.debug("doGet STOPED..."); >> myCallback.onGetClose(myConn, connInfo); >>} >> } >> >> Server side code to push data looks like this: >> >> protected int sendMsgs(final MyConnection myConn, final String info) { >> >>int res; >>try { >> final AsyncContext lAc = _ac; >> final ServletRequest req = lAc.getRequest(); >> if (!req.isAsyncStarted()) { >> log.debug("AsyncStarted=false on begin, uid:" + >> uid + ", sid:" + sid); >> return -1; >> } >> >> final ServletResponse res = lAc.getResponse(); >> ServletOutputStream os = null; >> final String text = getData(myConn); >> >> if (os == null) >> os = res.getOutputStream(); >> >> write2stream(os, text, 0x4000); >> os.println(); >> >> if (os != null) >> os.flush(); >> } catch(IOException ex) { >> log.error("Notify failed"); >> } catch (InterruptedException e) { >> log.error("Notify failed"); >> Thread.currentThread().interrupt(); >> } catch (Throwable e) { >> log.error("Notify failed"); >> logoff(true); &g
Re: HTTP communication
Hi André, First about network. Client and server are in the same segment and there are no firewalls in between them. Second I expect connection to be alive because during initiation I specify 'keep-alive' property of HTTP request. Also Tomcat and WLS are configured to support keepalive connections. From what I noticed, I monitored TCP connection using TCPView and Wireshark (on the client side), TCP connection is still alive even Tomcat or WLS cannot write to the socket. Just to mention, I have in my code an option to send periodical heartbeats to the client if there is no data. It is configurable and when I use this option everything works fine. But if I don’t send heartbeats as I wrote before connection becomes stale and I want to understand why it happens so that I can initiate, for example, automatic client logoff and cleanup resources. On 29/01/16 11:33, "André Warnier (tomcat)" wrote: >On 28.01.2016 18:38, Maxim Neshcheret wrote: >> I have a problem with my java application related to HTTP communication. >> >> Application description: >> >> 1. Client – server. Server is running in servlet container. We use >>Tomcat. >> >> Client use java HTTP library to communicate with the server. >> >> 2. When client establish connection to the server it sends GET >>request (keepalive) and server creates AsyncContext for the client with >>10 days timeout. >> >> 3. After connection established server periodically sends data to >>the client using AsyncContext and on the client side there is special >>thread which reads and processes the data. >> >[ snip ...] >> >> Usually this code works fine but it there is no data from server to >>client for 1 day and after 24 hours (can be 16 or 12) data appears, >>server cannot send data to the client. >> In the log there is no error. It is written that everything flushed but >>client still waiting for data in “final String line = reader.readLine();” >> When 2nd portion of data is sent by the server, then during flush I see >>the following error >> >> 2016-01-26 00:00:00,051|INFO |GWNotify-2/50 |ClientAbort >> 2016-01-26 00:00:00,051|TRACE|GWNotify-2/50 >>|ClientAbortException:java.io.IOException: APR error: -32 >> org.apache.catalina.connector.ClientAbortException: >>java.io.IOException: APR error: -32 >[snip ...] > >Hi. >I am unqualified to check your code, but a first question would be : >where is the Client, >and where is the Server and what is the connection between them like ? > >And the reason for the question is : it is not at all unusual, that any >kind of network >connection would be interrupted at some point over a 24-hour period, >specially if nothing >is sent over that connection for a long time. >When a connection "disappears", TCP sends no signal to either the client >or the server, >and an error will only be caught, if one of the parties tries to write to >the (now gone) >connection (which seems to be what happens above, when the server tries >to write to the >Client, and gets a "client is no longer there" exception). > >If you want to avoid this, you will have to handle this in your code. >You cannot just >expect the connection to be alive no matter what. > > > > >- >To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >For additional commands, e-mail: users-h...@tomcat.apache.org >
Re: HTTP communication
I'll second Andre's answer: Just because you declare a 10d timeout, you can't rely on the connection to stay up for that long. You can't even rely on a connection to stay up during the download of a simple gif (although that's so quick that the odds for connection termination are a lot lower). This is without looking at the code - just relying on a connection to stay up is not a proper assumption. You'll have to work around anything in between to fail and reconnect if necessary. Declaring a 10d timeout - if it works - will cause the appropriately configured endpoint to assume it's open. It has no effect on any of the components in between the endpoints. Olaf Am 29.01.2016 um 09:33 schrieb André Warnier (tomcat): > On 28.01.2016 18:38, Maxim Neshcheret wrote: >> I have a problem with my java application related to HTTP communication. >> >> Application description: >> >> 1. Client – server. Server is running in servlet container. We >> use Tomcat. >> >> Client use java HTTP library to communicate with the server. >> >> 2. When client establish connection to the server it sends GET >> request (keepalive) and server creates AsyncContext for the client >> with 10 days timeout. >> >> 3. After connection established server periodically sends data >> to the client using AsyncContext and on the client side there is >> special thread which reads and processes the data. >> > [ snip ...] >> >> Usually this code works fine but it there is no data from server to >> client for 1 day and after 24 hours (can be 16 or 12) data appears, >> server cannot send data to the client. >> In the log there is no error. It is written that everything flushed >> but client still waiting for data in “final String line = >> reader.readLine();” >> When 2nd portion of data is sent by the server, then during flush I >> see the following error >> >> 2016-01-26 00:00:00,051|INFO |GWNotify-2/50 |ClientAbort >> 2016-01-26 00:00:00,051|TRACE|GWNotify-2/50 >> |ClientAbortException:java.io.IOException: APR error: -32 >> org.apache.catalina.connector.ClientAbortException: >> java.io.IOException: APR error: -32 > [snip ...] > > Hi. > I am unqualified to check your code, but a first question would be : > where is the Client, and where is the Server and what is the > connection between them like ? > > And the reason for the question is : it is not at all unusual, that > any kind of network connection would be interrupted at some point over > a 24-hour period, specially if nothing is sent over that connection > for a long time. > When a connection "disappears", TCP sends no signal to either the > client or the server, and an error will only be caught, if one of the > parties tries to write to the (now gone) connection (which seems to be > what happens above, when the server tries to write to the Client, and > gets a "client is no longer there" exception). > > If you want to avoid this, you will have to handle this in your code. > You cannot just expect the connection to be alive no matter what. > > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: HTTP communication
> I have a problem with my java application related to HTTP communication. > > Application description: > > 1. Client – server. Server is running in servlet container. We > use Tomcat. > > Client use java HTTP library to communicate with the server. > > 2. When client establish connection to the server it sends GET > request (keepalive) and server creates AsyncContext for the client > with 10 days timeout. > > 3. After connection established server periodically sends data > to the client using AsyncContext and on the client side there is > special thread which reads and processes the data. > > Client reading thread example: > > private class GetRunnable implements Runnable { > >private final HttpURLConnection httpConn; >private final MyConnection myConn; > >public GetRunnable(final MyConnection myConn) throws > IOException, AppException { > this.myConn = myConn; > final String jSession = myConn.getJSession(); > httpConn = openHttpUrlConnection(false, > httpPostTimeout, jSession); > httpConn.connect(); > final int respCode = httpConn.getResponseCode(); > if (respCode != HttpURLConnection.HTTP_OK) { > String info = "Incorrect response from server, > responseCode:" + respCode + ", message:" + httpConn.getResponseMessage(); > log.error(info); > throw new AppException(info); > } else { > log.trace("GET connected"); > } >} > >@Override >public void run() { > try (final BufferedReader reader = new BufferedReader > (new InputStreamReader(httpConn.getInputStream(), "UTF-8")) ) { > log.info("doGet STARTED"); > final Thread t = Thread.currentThread(); > while (!t.isInterrupted()) { >log.trace("before readline"); >final String line = reader.readLine(); >log.trace("after readline: [" + line + "]"); > >//INFO: DATA PROCESSING HERE > } > } catch (IOException e) { > log.error("Error while doGet"); > } catch (Throwable e) { > log.debug("doGet STOPED..."); > log.error("Error while read input msg, do logoff", e); > logoff(); > throw e; > } > log.debug("doGet STOPED..."); > myCallback.onGetClose(myConn, connInfo); >} > } > > Server side code to push data looks like this: > > protected int sendMsgs(final MyConnection myConn, final String info) { > >int res; >try { > final AsyncContext lAc = _ac; > final ServletRequest req = lAc.getRequest(); > if (!req.isAsyncStarted()) { > log.debug("AsyncStarted=false on begin, uid:" + > uid + ", sid:" + sid); > return -1; > } > > final ServletResponse res = lAc.getResponse(); > ServletOutputStream os = null; > final String text = getData(myConn); > > if (os == null) > os = res.getOutputStream(); > > write2stream(os, text, 0x4000); > os.println(); > > if (os != null) > os.flush(); > } catch(IOException ex) { > log.error("Notify failed"); > } catch (InterruptedException e) { > log.error("Notify failed"); > Thread.currentThread().interrupt(); > } catch (Throwable e) { > log.error("Notify failed"); > logoff(true); > } > > return res; > } > > public static void write2stream(final OutputStream outputStream, > final String text, int bufferSize) throws IOException { > final CharsetEncoder encoder = getHttpEncodingCharset > ().newEncoder(); > > final int fullLen = text.length(); > final int byteBufferSize = Math.min(bufferSize, fullLen); > > final int en = (int)(byteBufferSize * (double) > encoder.maxBytesPerChar()); > final byte[] byteArray = new byte[en]; > > final ByteBuffer
Re: HTTP communication
On 28.01.2016 18:38, Maxim Neshcheret wrote: I have a problem with my java application related to HTTP communication. Application description: 1. Client – server. Server is running in servlet container. We use Tomcat. Client use java HTTP library to communicate with the server. 2. When client establish connection to the server it sends GET request (keepalive) and server creates AsyncContext for the client with 10 days timeout. 3. After connection established server periodically sends data to the client using AsyncContext and on the client side there is special thread which reads and processes the data. [ snip ...] Usually this code works fine but it there is no data from server to client for 1 day and after 24 hours (can be 16 or 12) data appears, server cannot send data to the client. In the log there is no error. It is written that everything flushed but client still waiting for data in “final String line = reader.readLine();” When 2nd portion of data is sent by the server, then during flush I see the following error 2016-01-26 00:00:00,051|INFO |GWNotify-2/50 |ClientAbort 2016-01-26 00:00:00,051|TRACE|GWNotify-2/50 |ClientAbortException:java.io.IOException: APR error: -32 org.apache.catalina.connector.ClientAbortException: java.io.IOException: APR error: -32 [snip ...] Hi. I am unqualified to check your code, but a first question would be : where is the Client, and where is the Server and what is the connection between them like ? And the reason for the question is : it is not at all unusual, that any kind of network connection would be interrupted at some point over a 24-hour period, specially if nothing is sent over that connection for a long time. When a connection "disappears", TCP sends no signal to either the client or the server, and an error will only be caught, if one of the parties tries to write to the (now gone) connection (which seems to be what happens above, when the server tries to write to the Client, and gets a "client is no longer there" exception). If you want to avoid this, you will have to handle this in your code. You cannot just expect the connection to be alive no matter what. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
HTTP communication
I have a problem with my java application related to HTTP communication. Application description: 1. Client – server. Server is running in servlet container. We use Tomcat. Client use java HTTP library to communicate with the server. 2. When client establish connection to the server it sends GET request (keepalive) and server creates AsyncContext for the client with 10 days timeout. 3. After connection established server periodically sends data to the client using AsyncContext and on the client side there is special thread which reads and processes the data. Client reading thread example: private class GetRunnable implements Runnable { private final HttpURLConnection httpConn; private final MyConnection myConn; public GetRunnable(final MyConnection myConn) throws IOException, AppException { this.myConn = myConn; final String jSession = myConn.getJSession(); httpConn = openHttpUrlConnection(false, httpPostTimeout, jSession); httpConn.connect(); final int respCode = httpConn.getResponseCode(); if (respCode != HttpURLConnection.HTTP_OK) { String info = "Incorrect response from server, responseCode:" + respCode + ", message:" + httpConn.getResponseMessage(); log.error(info); throw new AppException(info); } else { log.trace("GET connected"); } } @Override public void run() { try (final BufferedReader reader = new BufferedReader(new InputStreamReader(httpConn.getInputStream(), "UTF-8")) ) { log.info("doGet STARTED"); final Thread t = Thread.currentThread(); while (!t.isInterrupted()) { log.trace("before readline"); final String line = reader.readLine(); log.trace("after readline: [" + line + "]"); //INFO: DATA PROCESSING HERE } } catch (IOException e) { log.error("Error while doGet"); } catch (Throwable e) { log.debug("doGet STOPED..."); log.error("Error while read input msg, do logoff", e); logoff(); throw e; } log.debug("doGet STOPED..."); myCallback.onGetClose(myConn, connInfo); } } Server side code to push data looks like this: protected int sendMsgs(final MyConnection myConn, final String info) { int res; try { final AsyncContext lAc = _ac; final ServletRequest req = lAc.getRequest(); if (!req.isAsyncStarted()) { log.debug("AsyncStarted=false on begin, uid:" + uid + ", sid:" + sid); return -1; } final ServletResponse res = lAc.getResponse(); ServletOutputStream os = null; final String text = getData(myConn); if (os == null) os = res.getOutputStream(); write2stream(os, text, 0x4000); os.println(); if (os != null) os.flush(); } catch(IOException ex) { log.error("Notify failed"); } catch (InterruptedException e) { log.error("Notify failed"); Thread.currentThread().interrupt(); } catch (Throwable e) { log.error("Notify failed"); logoff(true); } return res; } public static void write2stream(final OutputStream outputStream, final String text, int bufferSize) throws IOException { final CharsetEncoder encoder = getHttpEncodingCharset().newEncoder(); final int fullLen = text.length(); final int byteBufferSize = Math.min(bufferSize, fullLen); final int en = (int)(byteBufferSize * (double)encoder.maxBytesPerChar()); final byte[] byteArray = new byte[en]; final ByteBuffer byteBuffer = ByteBuffer.wrap(byteArray); final CharBuffer charBuffer = CharBuffer.wrap(text); for(int pos=0, len=byteBufferSize; pos < fullLen; pos=len, len=Math.min(pos+byteBufferSize, fullLen)) { try { final CharBuffer window = charBuffer.subSequence(pos, len); CoderResult res = encoder.encode(window, byteBuffer, true); if (!res.isUnderflow()) res.throwException(); res = encoder.flush(byteBuffer);