On Tue, 2013-07-23 at 20:24 -0700, Hiranya Jayathilaka wrote: > It looks like the socket goes into the CLOSE_WAIT state after shutting > down the IO reactor and remains there until the JVM it self is killed. > Since all automated test cases are executed within a single JVM > instance, this issue continues to occur. Looks like something is not > getting cleaned up properly with the latest HTTP Core. > > > Thanks, > Hiranya >
Hiranya I need a more specific evidence of HttpCore doing something wrong. Could you please reproduce the issue with a test app, preferably not dependent on Synapse? Oleg > On Jul 23, 2013, at 3:54 PM, Hiranya Jayathilaka > <[email protected]> wrote: > > > Hi Oleg, > > > > > > I've done the necessary code changes locally and it seems to work > > fine. The HTTPS sender now works as expected. But when running > > automated tests, I frequently get the following exception: > > > > > > 34 [HttpCoreNIOListener] WARN > > org.apache.synapse.transport.nhttp.HttpCoreNIOListener - System may > > be unstable: IOReactor encountered a checked exception : Address > > already in use > > java.net.BindException: Address already in use > > at sun.nio.ch.Net.bind(Native Method) > > at > > sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:124) > > at > > sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) > > at > > sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52) > > at > > org.apache.http.impl.nio.reactor.DefaultListeningIOReactor.processSessionRequests(DefaultListeningIOReactor.java:238) > > at > > org.apache.http.impl.nio.reactor.DefaultListeningIOReactor.processEvents(DefaultListeningIOReactor.java:144) > > at > > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:374) > > at org.apache.synapse.transport.nhttp.HttpCoreNIOListener > > $2.run(HttpCoreNIOListener.java:254) > > at java.lang.Thread.run(Thread.java:680) > > > > > > This is seen in both NHTTP unit tests and Synapse integration tests. > > Any idea what is going on here? Synapse cleans up the IO reactor by > > calling ioReactor.shutdown(). Is there anything else we need to do > > to clean up the reactor properly? > > > > > > Thanks, > > Hiranya > > > > > > On Jul 23, 2013, at 10:54 AM, Hiranya Jayathilaka > > <[email protected]> wrote: > > > > > HI Oleg, > > > > > > On Jul 23, 2013, at 12:32 AM, Oleg Kalnichevski <[email protected]> > > > wrote: > > > > > > > On Mon, 2013-07-22 at 20:51 -0700, Hiranya Jayathilaka wrote: > > > > > On Jul 22, 2013, at 7:53 PM, Rajika Kumarasiri > > > > > <[email protected]> wrote: > > > > > > > > > > > May be we should enable the SSl debug logs and see what's > > > > > > going on ? > > > > > > > > > > I ran some more tests and it looks like the HTTPS sender > > > > > doesn't work with HTTP Core 4.2. It just sends out the > > > > > messages without any SSL security. I can even monitor the > > > > > message in plain text using TCPMon. Looks like some SSL > > > > > related code in the Synapse HTTPS sender doesn't work properly > > > > > with the new HTTP Core. > > > > > > > > > > Thanks, > > > > > Hiranya > > > > > > > > > > > > > Hiranya > > > > > > > > Have you migrated off the deprecated APIs to new 4.2 APIs? > > > > > > > > > No not yet. I just tried changing the HTTP Core dependency version > > > to begin with. I guess migrating off the deprecated APIs is the > > > next logical step. > > > > > > > > > Thanks, > > > Hiranya > > > > > > > > > > > Oleg > > > > > > > > PS: By the way, you should probably consider migrating straight > > > > to 4.3 > > > > at this point. > > > > > > > > > > > > > > > > > > > > Rajika > > > > > > > > > > > > > > > > > > On Mon, Jul 22, 2013 at 8:23 PM, Hiranya Jayathilaka > > > > > > <[email protected]> wrote: > > > > > > Hi Folks, > > > > > > > > > > > > Currently Synapse is based on HTTP Core 4.1.4. I just tried > > > > > > using 4.2 instead (just changed the version of the maven > > > > > > dependency). Almost everything worked fine, except for a > > > > > > couple of integration test failures. In both these tests > > > > > > Synapse uses HTTPS to contact the backend services, so I > > > > > > suppose that's where things went wrong. The backend server > > > > > > threw the following exception in the process: > > > > > > > > > > > > 2013-07-22 17:06:36,970 [-] [https-Listener I/O > > > > > > dispatcher-1] ERROR ServerHandler I/O error: Unrecognized > > > > > > SSL message, plaintext connection? > > > > > > javax.net.ssl.SSLException: Unrecognized SSL message, > > > > > > plaintext connection? > > > > > > at > > > > > > com.sun.net.ssl.internal.ssl.EngineInputRecord.bytesInCompletePacket(EngineInputRecord.java:152) > > > > > > at > > > > > > com.sun.net.ssl.internal.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:806) > > > > > > at > > > > > > com.sun.net.ssl.internal.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:721) > > > > > > at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:607) > > > > > > at > > > > > > org.apache.http.nio.reactor.ssl.SSLIOSession.doUnwrap(SSLIOSession.java:228) > > > > > > at > > > > > > org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:263) > > > > > > at > > > > > > org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:370) > > > > > > at > > > > > > org.apache.http.impl.nio.SSLServerIOEventDispatch.inputReady(SSLServerIOEventDispatch.java:229) > > > > > > at > > > > > > org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:160) > > > > > > at > > > > > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:342) > > > > > > at > > > > > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:320) > > > > > > at > > > > > > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) > > > > > > at > > > > > > org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106) > > > > > > at > > > > > > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:604) > > > > > > at java.lang.Thread.run(Thread.java:680) > > > > > > > > > > > > I also tried upgrading to the latest HTTP Core version > > > > > > (4.2.4). With this version NHTTP unit tests failed and most > > > > > > of the integration tests failed too. It turns out only the > > > > > > very first integration test ran successfully. Everything > > > > > > that followed threw a bind exception. Looks like something > > > > > > isn't getting cleaned up properly (although the logs > > > > > > indicate that Synapse transport listeners are shutting down > > > > > > cleanly after each test case). > > > > > > > > > > > > Does anybody got an idea what's going on? Do we need to do > > > > > > any code changes to migrate to the latest HTTP Core? > > > > > > > > > > > > Thanks, > > > > > > Hiranya > > > > > > -- > > > > > > Hiranya Jayathilaka > > > > > > Mayhem Lab/RACE Lab; > > > > > > Dept. of Computer Science, UCSB; http://cs.ucsb.edu > > > > > > E-mail: [email protected]; Mobile: +1 (805) 895-7443 > > > > > > Blog: http://techfeast-hiranya.blogspot.com > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Hiranya Jayathilaka > > > > > Mayhem Lab/RACE Lab; > > > > > Dept. of Computer Science, UCSB; http://cs.ucsb.edu > > > > > E-mail: [email protected]; Mobile: +1 (805) 895-7443 > > > > > Blog: http://techfeast-hiranya.blogspot.com > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > -- > > > Hiranya Jayathilaka > > > Mayhem Lab/RACE Lab; > > > Dept. of Computer Science, UCSB; http://cs.ucsb.edu > > > E-mail: [email protected]; Mobile: +1 (805) 895-7443 > > > Blog: http://techfeast-hiranya.blogspot.com > > > > > > > > > > -- > > Hiranya Jayathilaka > > Mayhem Lab/RACE Lab; > > Dept. of Computer Science, UCSB; http://cs.ucsb.edu > > E-mail: [email protected]; Mobile: +1 (805) 895-7443 > > Blog: http://techfeast-hiranya.blogspot.com > > > > > > -- > Hiranya Jayathilaka > Mayhem Lab/RACE Lab; > Dept. of Computer Science, UCSB; http://cs.ucsb.edu > E-mail: [email protected]; Mobile: +1 (805) 895-7443 > Blog: http://techfeast-hiranya.blogspot.com > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
