How to unsubscribe from these mails?? - Sachin
On Sat, Dec 13, 2008 at 4:44 PM, Raghu Angadi (JIRA) <[email protected]>wrote: > > [ > https://issues.apache.org/jira/browse/HADOOP-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel] > > Raghu Angadi updated HADOOP-4797: > --------------------------------- > > Attachment: TestRpcCpu.patch > > > Ok, benchmark with much saner results. Only difference is that this one > returns (a Writable) ByteArray instead of naked a 'byte []' to avoid > ObjectWritable from handling the array. CPU for 100 calls : > > * With out the patch ~ 7000 > * With the patch ~ 1050 > * RPC server takes 6-7 times less CPU to serve 10MB buffer. > > I hope 6-7 times less is pretty good for a side benefit. > > In the he previous version of the benchmark, client reads much slower, so > 10MB mostly requires more write() calls. The extra CPU penalty in trunk is > directly proportional to number write() calls required to write the full > buffer. > > > RPC Server can leave a lot of direct buffers > > --------------------------------------------- > > > > Key: HADOOP-4797 > > URL: https://issues.apache.org/jira/browse/HADOOP-4797 > > Project: Hadoop Core > > Issue Type: Bug > > Components: ipc > > Affects Versions: 0.17.0 > > Reporter: Raghu Angadi > > Assignee: Raghu Angadi > > Priority: Blocker > > Fix For: 0.18.3, 0.19.1, 0.20.0 > > > > Attachments: HADOOP-4797-branch-18.patch, > HADOOP-4797-branch-18.patch, HADOOP-4797-branch-18.patch, HADOOP-4797.patch, > HADOOP-4797.patch, TestRpcCpu.patch, TestRpcCpu.patch > > > > > > RPC server unwittingly can soft-leak direct buffers. One observed case is > that one of the namenodes at Yahoo took 40GB of virtual memory though it was > configured for 24GB memory. Most of the memory outside Java heap expected to > be direct buffers. This shown to be because of how RPC server reads and > writes serialized data. The cause and proposed fix are in following comment. > > > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > >
