[ http://issues.apache.org/jira/browse/HADOOP-312?page=comments#action_12430160 ] Devaraj Das commented on HADOOP-312: ------------------------------------
>Threads should be cheap to create, but keeping too many >around can be expensive. Again, I think the life of the thread >should be the same as the life of the connection. There are not many threads in the system. It is one per server (in the current version of hadoop, it is a single digit number, maybe, 2 or 3). If connection life is 1 sec and if the thread's life is same as the life of the connection, then a thread is destroyed/created ~once/second per server. I really wanted to avoid creating/destroying threads that often. > Connections should not be cached > -------------------------------- > > Key: HADOOP-312 > URL: http://issues.apache.org/jira/browse/HADOOP-312 > Project: Hadoop > Issue Type: Improvement > Components: ipc > Reporter: Devaraj Das > Assigned To: Devaraj Das > Attachments: no_conn_caching.patch, no_conn_caching.patch, > no_conn_caching.patch, no_connection_caching.patch, > no_connection_caching.patch > > > Servers and clients (client include datanodes, tasktrackers, DFSClients & > tasks) should not cache connections or maybe cache them for very short > periods of time. Clients should set up & tear down connections to the servers > everytime they need to contact the servers (including the heartbeats). If > connection is cached, then reuse the existing connection for a few subsequent > transactions until the connection expires. The heartbeat interval should be > more so that many more clients (order of tens of thousands) can be > accomodated within 1 heartbeat interval. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
