[ https://issues.apache.org/jira/browse/HBASE-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185909#comment-13185909 ]
Jean-Daniel Cryans commented on HBASE-5190: ------------------------------------------- About #3. I like the elegance of that solution since we don't have to keep track of the calls in flight but I see 2 big issues: - if you set a max call size you need to keep both clients and servers in sink and also decide who's going to do the check. - if you plan for big calls by default, you may end up with a tiny size for the queue. For example, let's say you cap calls at 10% of the heap and set their max individual size at 10MB, it means that you can only allow 10 items in the queue (and you don't account listeners). > Limit the IPC queue size based on calls' payload size > ----------------------------------------------------- > > Key: HBASE-5190 > URL: https://issues.apache.org/jira/browse/HBASE-5190 > Project: HBase > Issue Type: Improvement > Affects Versions: 0.90.5 > Reporter: Jean-Daniel Cryans > Fix For: 0.94.0 > > > Currently we limit the number of calls in the IPC queue only on their count. > It used to be really high and was dropped down recently to num_handlers * 10 > (so 100 by default) because it was easy to OOME yourself when huge calls were > being queued. It's still possible to hit this problem if you use really big > values and/or a lot of handlers, so the idea is that we should take into > account the payload size. I can see 3 solutions: > - Do the accounting outside of the queue itself for all calls coming in and > out and when a call doesn't fit, throw a retryable exception. > - Same accounting but instead block the call when it comes in until space is > made available. > - Add a new parameter for the maximum size (in bytes) of a Call and then set > the size the IPC queue (in terms of the number of items) so that it could > only contain as many items as some predefined maximum size (in bytes) for the > whole queue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira