[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16848259#comment-16848259
 ] 

Enrico Olivelli commented on ZOOKEEPER-3399:
--------------------------------------------

{quote}
Recently we have moved some of our production clusters to the top of the trunk
{quote}

This is encouraging !
I think it is time to think to cut a release for 3.6.0

> Remove logging in getGlobalOutstandingLimit for optimal performance.
> --------------------------------------------------------------------
>
>                 Key: ZOOKEEPER-3399
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3399
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: server
>    Affects Versions: 3.6.0
>            Reporter: Michael Han
>            Assignee: Michael Han
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.6.0
>
>          Time Spent: 1h
>  Remaining Estimate: 0h
>
> Recently we have moved some of our production clusters to the top of the 
> trunk. One issue we found is a performance regression on read and write 
> latency on the clusters where the quorum is also serving traffic. The average 
> read latency increased by 50x, p99 read latency increased by 300x. 
> The root cause is a log statement introduced in ZOOKEEPER-3177 (PR711), where 
> we added a LOG.info statement in getGlobalOutstandingLimit. 
> getGlobalOutstandingLimit is on the critical code path for request processing 
> and for each request, it will be called twice (one at processing the packet, 
> one at finalizing the request response). This not only degrades performance 
> of the server, but also bloated the log file, when the QPS of a server is 
> high.
> This only impacts clusters when the quorum (leader + follower) is serving 
> traffic. For clusters where only observers are serving traffic no impact is 
> observed.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to