Reduce CPU usage of hadoop ipc package
--------------------------------------
Key: HADOOP-970
URL: https://issues.apache.org/jira/browse/HADOOP-970
Project: Hadoop
Issue Type: Bug
Components: ipc
Reporter: dhruba borthakur
There are a couple of optimizations that could be done to reduce CPU
consumption.
1. The method Server.cleanupConnections() could be invoked less often.
2. The method Server.cleanupConnections() uses a List to manage all active
connections and uses connectionList.get(i) to iterate. Locating the ith
element essentially translates to traversing the list from the beginning to the
ith position.
3. The current DFS heartbeattime is 3 seconds whereas
ipc.client.connection.maxidletime is set to 1 second. The proposal is to change
the default value of ipc.client.connection.maxidletime to something larger than
the heartbeat interval. This also has to suit the heartbeat periodicity of
map-reduce software.
4. Evaluate epoll() added in JDK 1.5.10 (this is a java cmd line option)
http://java.sun.com/j2se/1.5.0/ReleaseNotes.html#150_10
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
