[ https://issues.apache.org/jira/browse/HDFS-15409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149176#comment-17149176 ]
Danil Lipovoy commented on HDFS-15409: -------------------------------------- I was surprised, but old (current) version shows better distribution: {code:java} public ShortCircuitCache getShortCircuitCache(Long idx) { LOG.info("shortCircuitCache: " + (int) (idx % clientShortCircuitNum)); return shortCircuitCache[(int) (idx % clientShortCircuitNum)]; } {code} cat /var/log/hbase/hbase-cmf-hbase-REGIONSERVER-home.com.log.out |grep "shortCircuitCache:"|awk '\{print $6}'| sort | uniq -c | sort -nr | awk '\{printf "%-8s%s\n", $2, $1}'|sort 0 557207 1 545778 2 556965 It was the same test scenario. I agree, It looks strange, but I can't beat the facts) > Optimization Strategy for choosing ShortCircuitCache > ----------------------------------------------------- > > Key: HDFS-15409 > URL: https://issues.apache.org/jira/browse/HDFS-15409 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Lisheng Sun > Assignee: Lisheng Sun > Priority: Major > > When clientShortCircuitNum is 10, the probability of falling into each > ShortCircuitCache is the same, while the probability of other > clientShortCircuitNum is different. > For example if clientShortCircuitNum is 3, when a lot of blockids of SSR are > ***1, ***4, ***7, this situation will fall into a ShortCircuitCache. > Since the real environment blockid is completely unpredictable, i think it is > need to design a strategy which is allocated to a specific ShortCircuitCache. > This should improve performance even more. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org