[
https://issues.apache.org/jira/browse/DIRMINA-301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Emmanuel Lecharny updated DIRMINA-301:
--------------------------------------
Status for this issue ?
> New Multi threaded SocketIOProcessor to improve fairness of socket
> reads/writes
> -------------------------------------------------------------------------------
>
> Key: DIRMINA-301
> URL: https://issues.apache.org/jira/browse/DIRMINA-301
> Project: MINA
> Issue Type: Improvement
> Components: Transport
> Affects Versions: 1.0.0
> Environment: Problem on all platforms, example test ran on a 8 Way
> Opteron
> Reporter: Martin Ritchie
> Attachments: JIRA-301-Release-4.zip, Mina Multi Thread
> SocketIOProcessor(MinaHead).zip, Mina Multi Thread SocketIOProcessor.zip,
> Mina Multi Thread SocketIOProcessor.zip,
> MultiThreadSocketIOProcessor-java-1.0-proposal-R4.1-src.zip
>
>
> The current SocketIOProcessor uses a single thread to do both reads and
> socket flushes. During some testing of Apache Qpid (A messaging broker) which
> uses mina as a transport we ran in to two problems.
> The first was the client that produces lots of data failed with an
> OutOfMemoryException, as mentioned in DIRMINA-206.
> The second is that under constant load the broker cannot read and write from
> the same socket at the same time. I have created a MultiThreaded
> SocketIOProcessor that allows reads and writes to occur simultaneously. On
> low core boxes the performance is similar to the existing SocketIOProcessor.
> However, on higher core boxes the Multi threaded box can be more than twice
> the speed.
> The attached zip file is an attempt to resolve the second.
> To run the test:
> run "ant " to build everything.
> The memory requirements for each process are shown in parenthesis, these were
> just the largest numbers shown in top as the processes ran.
> Then
> "ant acceptor_mina"(1.2G) or "ant acceptor_multi"(250M) to run the listener
> process.
> This process simply sends the received message back down the same socket.
> Then run the corresponding writer
> "ant writer_mina"(560M) or "ant writer_multi"(540M)
> The results from each of the writers I ran on an 8way Opteron are shown below.
> The out of memory issue on the writer can be demonstrated buy running:
> "ant writer_mina_mem" or "ant writer_multi_mem"
> The increased throughput on the Multi Threaded SocketIOProcessor should allow
> it survive the low memory setting.
> ~/dev/TempProjects/mina-2006-11-02-1056/Mina Multi Thread SocketIOProcessor$
> ant writer_mina
> Buildfile: build.xml
> writer_mina:
> [java] main 2006-11-02 10:57:36,932 INFO
> [apache.mina.SocketIOTest.WriterTest] Starting 2k test
> [java] main 2006-11-02 10:57:36,933 WARN
> [apache.mina.SocketIOTest.WriterTest] Using MINA NIO
> [java] main 2006-11-02 10:57:36,999 INFO
> [apache.mina.SocketIOTest.WriterTest] Attempting connection to
> localhost/127.0.0.1:9999
> [java] main 2006-11-02 10:57:37,035 INFO
> [apache.mina.SocketIOTest.WriterTest] Connection completed
> [java] Thread-2 2006-11-02 10:57:37,038 INFO
> [apache.mina.SocketIOTest.WriterTest] Starting to send 200000 buffers of 2048B
> [java] Thread-2 2006-11-02 10:57:43,409 INFO
> [apache.mina.SocketIOTest.WriterTest] All buffers sent; waiting for receipt
> from server
> [java] Thread-2 2006-11-02 10:57:58,652 INFO
> [apache.mina.SocketIOTest.WriterTest] Completed
> [java] Thread-2 2006-11-02 10:57:58,652 INFO
> [apache.mina.SocketIOTest.WriterTest] Total time waiting for server after
> last write: 15243
> [java] Thread-2 2006-11-02 10:57:58,652 INFO
> [apache.mina.SocketIOTest.WriterTest] Total time: 21613
> [java] Thread-2 2006-11-02 10:57:58,652 INFO
> [apache.mina.SocketIOTest.WriterTest] MB per second: 18951
> [java] Thread-2 2006-11-02 10:57:58,654 INFO
> [apache.mina.SocketIOTest.WriterTest] Average chunk time: 1.0000000000011369ms
> [java] Thread-2 2006-11-02 10:57:58,654 INFO
> [apache.mina.SocketIOTest.WriterTest] Maximum WriteRequestQueue size: 171836
> [java] Thread-2 2006-11-02 10:57:58,654 INFO
> [apache.mina.SocketIOTest.WriterTest] Closing session
> ~/dev/TempProjects/mina-2006-11-02-1056/Mina Multi Thread SocketIOProcessor$
> ant writer_multi
> Buildfile: build.xml
> writer_multi:
> [java] main 2006-11-02 10:58:10,544 INFO
> [apache.mina.SocketIOTest.WriterTest] Starting 2k test
> [java] main 2006-11-02 10:58:10,546 WARN
> [apache.mina.SocketIOTest.WriterTest] Using MultiThread NIO
> [java] main 2006-11-02 10:58:10,620 INFO
> [apache.mina.SocketIOTest.WriterTest] Attempting connection to
> localhost/127.0.0.1:9999
> [java] main 2006-11-02 10:58:10,675 INFO
> [apache.mina.SocketIOTest.WriterTest] Connection completed
> [java] Thread-3 2006-11-02 10:58:10,678 INFO
> [apache.mina.SocketIOTest.WriterTest] Starting to send 200000 buffers of 2048B
> [java] Thread-3 2006-11-02 10:58:15,464 INFO
> [apache.mina.SocketIOTest.WriterTest] All buffers sent; waiting for receipt
> from server
> [java] Thread-3 2006-11-02 10:58:20,214 INFO
> [apache.mina.SocketIOTest.WriterTest] Completed
> [java] Thread-3 2006-11-02 10:58:20,215 INFO
> [apache.mina.SocketIOTest.WriterTest] Total time waiting for server after
> last write: 4750
> [java] Thread-3 2006-11-02 10:58:20,215 INFO
> [apache.mina.SocketIOTest.WriterTest] Total time: 9537
> [java] Thread-3 2006-11-02 10:58:20,215 INFO
> [apache.mina.SocketIOTest.WriterTest] MB per second: 42948
> [java] Thread-3 2006-11-02 10:58:20,216 INFO
> [apache.mina.SocketIOTest.WriterTest] Average chunk time: 1.0ms
> [java] Thread-3 2006-11-02 10:58:20,216 INFO
> [apache.mina.SocketIOTest.WriterTest] Maximum WriteRequestQueue size: 159054
> [java] Thread-3 2006-11-02 10:58:20,216 INFO
> [apache.mina.SocketIOTest.WriterTest] Closing session
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
