On Tue, 2013-07-23 at 20:24 -0700, Hiranya Jayathilaka wrote:
It looks like the socket goes into the CLOSE_WAIT state after shutting
down the IO reactor and remains there until the JVM it self is killed.
Since all automated test cases are executed within a single JVM
instance, this issue continues to occur. Looks like something is not
getting cleaned up properly with the latest HTTP Core.
Thanks,
Hiranya
Hiranya
I need a more specific evidence of HttpCore doing something wrong. Could
you please reproduce the issue with a test app, preferably not dependent
on Synapse?
Oleg
On Jul 23, 2013, at 3:54 PM, Hiranya Jayathilaka
<[email protected]> wrote:
Hi Oleg,
I've done the necessary code changes locally and it seems to work
fine. The HTTPS sender now works as expected. But when running
automated tests, I frequently get the following exception:
34 [HttpCoreNIOListener] WARN
org.apache.synapse.transport.nhttp.HttpCoreNIOListener - System may
be unstable: IOReactor encountered a checked exception : Address
already in use
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind(Native Method)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:124)
at
sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
at
sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52)
at
org.apache.http.impl.nio.reactor.DefaultListeningIOReactor.processSessionRequests(DefaultListeningIOReactor.java:238)
at
org.apache.http.impl.nio.reactor.DefaultListeningIOReactor.processEvents(DefaultListeningIOReactor.java:144)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:374)
at org.apache.synapse.transport.nhttp.HttpCoreNIOListener
$2.run(HttpCoreNIOListener.java:254)
at java.lang.Thread.run(Thread.java:680)
This is seen in both NHTTP unit tests and Synapse integration tests.
Any idea what is going on here? Synapse cleans up the IO reactor by
calling ioReactor.shutdown(). Is there anything else we need to do
to clean up the reactor properly?
Thanks,
Hiranya
On Jul 23, 2013, at 10:54 AM, Hiranya Jayathilaka
<[email protected]> wrote:
HI Oleg,
On Jul 23, 2013, at 12:32 AM, Oleg Kalnichevski <[email protected]>
wrote:
On Mon, 2013-07-22 at 20:51 -0700, Hiranya Jayathilaka wrote:
On Jul 22, 2013, at 7:53 PM, Rajika Kumarasiri
<[email protected]> wrote:
May be we should enable the SSl debug logs and see what's
going on ?
I ran some more tests and it looks like the HTTPS sender
doesn't work with HTTP Core 4.2. It just sends out the
messages without any SSL security. I can even monitor the
message in plain text using TCPMon. Looks like some SSL
related code in the Synapse HTTPS sender doesn't work properly
with the new HTTP Core.
Thanks,
Hiranya
Hiranya
Have you migrated off the deprecated APIs to new 4.2 APIs?
No not yet. I just tried changing the HTTP Core dependency version
to begin with. I guess migrating off the deprecated APIs is the
next logical step.
Thanks,
Hiranya
Oleg
PS: By the way, you should probably consider migrating straight
to 4.3
at this point.
Rajika
On Mon, Jul 22, 2013 at 8:23 PM, Hiranya Jayathilaka
<[email protected]> wrote:
Hi Folks,
Currently Synapse is based on HTTP Core 4.1.4. I just tried
using 4.2 instead (just changed the version of the maven
dependency). Almost everything worked fine, except for a
couple of integration test failures. In both these tests
Synapse uses HTTPS to contact the backend services, so I
suppose that's where things went wrong. The backend server
threw the following exception in the process:
2013-07-22 17:06:36,970 [-] [https-Listener I/O
dispatcher-1] ERROR ServerHandler I/O error: Unrecognized
SSL message, plaintext connection?
javax.net.ssl.SSLException: Unrecognized SSL message,
plaintext connection?
at
com.sun.net.ssl.internal.ssl.EngineInputRecord.bytesInCompletePacket(EngineInputRecord.java:152)
at
com.sun.net.ssl.internal.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:806)
at
com.sun.net.ssl.internal.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:721)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:607)
at
org.apache.http.nio.reactor.ssl.SSLIOSession.doUnwrap(SSLIOSession.java:228)
at
org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:263)
at
org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:370)
at
org.apache.http.impl.nio.SSLServerIOEventDispatch.inputReady(SSLServerIOEventDispatch.java:229)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:160)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:342)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:320)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:604)
at java.lang.Thread.run(Thread.java:680)
I also tried upgrading to the latest HTTP Core version
(4.2.4). With this version NHTTP unit tests failed and most
of the integration tests failed too. It turns out only the
very first integration test ran successfully. Everything
that followed threw a bind exception. Looks like something
isn't getting cleaned up properly (although the logs
indicate that Synapse transport listeners are shutting down
cleanly after each test case).
Does anybody got an idea what's going on? Do we need to do
any code changes to migrate to the latest HTTP Core?
Thanks,
Hiranya
--
Hiranya Jayathilaka
Mayhem Lab/RACE Lab;
Dept. of Computer Science, UCSB; http://cs.ucsb.edu
E-mail: [email protected]; Mobile: +1 (805) 895-7443
Blog: http://techfeast-hiranya.blogspot.com
--
Hiranya Jayathilaka
Mayhem Lab/RACE Lab;
Dept. of Computer Science, UCSB; http://cs.ucsb.edu
E-mail: [email protected]; Mobile: +1 (805) 895-7443
Blog: http://techfeast-hiranya.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
--
Hiranya Jayathilaka
Mayhem Lab/RACE Lab;
Dept. of Computer Science, UCSB; http://cs.ucsb.edu
E-mail: [email protected]; Mobile: +1 (805) 895-7443
Blog: http://techfeast-hiranya.blogspot.com
--
Hiranya Jayathilaka
Mayhem Lab/RACE Lab;
Dept. of Computer Science, UCSB; http://cs.ucsb.edu
E-mail: [email protected]; Mobile: +1 (805) 895-7443
Blog: http://techfeast-hiranya.blogspot.com
--
Hiranya Jayathilaka
Mayhem Lab/RACE Lab;
Dept. of Computer Science, UCSB; http://cs.ucsb.edu
E-mail: [email protected]; Mobile: +1 (805) 895-7443
Blog: http://techfeast-hiranya.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
|
HttpCoreTest.java