[
https://issues.apache.org/jira/browse/HADOOP-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12528323
]
Enis Soztutar commented on HADOOP-1841:
---------------------------------------
I started to work on this issue, but before any code, i would like to discuss a
few things.
Current ipc/rpc handling is done as a 1 dispatcher N worker scenario. The
Listener thread asynchronously listens to the socket for incoming connections,
accepts them(creating Connection instances), or reads the data and puts in the
queue for the handler threads for later processing.
The handler threads on the other hand, listens to the call queue, removes the
first element and process the Call. It first makes the actual call to the
method(through {{Server#call}} ), gets the return value and writes back the
results to output stream(blocking).
As Doug suggested the performance might be improved if the results would have
been written asynchronously. But i am not sure that it will affect the
performance, if any, a lot. The costly operation in handler threads is expected
to be the method calls. How do others think about this? We should start to
think about developing benchmark tests. It would be really cool if we could
improved the performance of ipc calls say %10. I am thinking about using jmeter
for this, however maybe we need to develop our own benchmarking framework to
test all the components of Hadoop. Any thoughts ?
> IPC server should write repsonses asynchronously
> ------------------------------------------------
>
> Key: HADOOP-1841
> URL: https://issues.apache.org/jira/browse/HADOOP-1841
> Project: Hadoop
> Issue Type: Improvement
> Components: ipc
> Reporter: Doug Cutting
> Assignee: Enis Soztutar
>
> Hadoop's IPC Server currently writes responses from request handler threads
> using blocking writes. Performance and scalability might be improved if
> responses were written asynchronously.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
