[ https://issues.apache.org/jira/browse/HADOOP-11552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddharth Seth updated HADOOP-11552: ------------------------------------ Status: Patch Available (was: Open) > Allow handoff on the server side for RPC requests > ------------------------------------------------- > > Key: HADOOP-11552 > URL: https://issues.apache.org/jira/browse/HADOOP-11552 > Project: Hadoop Common > Issue Type: Improvement > Components: ipc > Reporter: Siddharth Seth > Assignee: Siddharth Seth > Attachments: HADOOP-11552.1.wip.txt, HADOOP-11552.2.txt, > HADOOP-11552.3.txt > > > An RPC server handler thread is tied up for each incoming RPC request. This > isn't ideal, since this essentially implies that RPC operations should be > short lived, and most operations which could take time end up falling back to > a polling mechanism. > Some use cases where this is useful. > - YARN submitApplication - which currently submits, followed by a poll to > check if the application is accepted while the submit operation is written > out to storage. This can be collapsed into a single call. > - YARN allocate - requests and allocations use the same protocol. New > allocations are received via polling. > The allocate protocol could be split into a request/heartbeat along with a > 'awaitResponse'. The request/heartbeat is sent only when there's a request or > on a much longer heartbeat interval. awaitResponse is always left active with > the RM - and returns the moment something is available. > MapReduce/Tez task to AM communication is another example of this pattern. > The same pattern of splitting calls can be used for other protocols as well. > This should serve to improve latency, as well as reduce network traffic since > the keep-alive heartbeat can be sent less frequently. > I believe there's some cases in HDFS as well, where the DN gets told to > perform some operations when they heartbeat into the NN. -- This message was sent by Atlassian JIRA (v6.3.4#6332)