[jira] [Created] (ZOOKEEPER-3484) Improve the throughput by optimizing the synchronization around outstandingChanges
Yisong Yue created ZOOKEEPER-3484:
-
Summary: Improve the throughput by optimizing the synchronization
around outstandingChanges
Key: ZOOKEEPER-3484
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3484
Project: ZooKeeper
Issue Type: Improvement
Reporter: Yisong Yue
Assignee: Yisong Yue
Fix For: 3.6.0
The "processTxn(Request request)" function in ZooKeeperServer.java ("entry
point for FinalRequestProcessor.java") is synchronized around
`outstandingChanges` for all requests now. However, this synchronization is
unnecessary for read requests, and skipping such synchronization for reads can
improve the overall throughput of the request processor pipeline.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (ZOOKEEPER-3179) Add snapshot compression to reduce the disk IO
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yisong Yue reassigned ZOOKEEPER-3179: - Assignee: Yisong Yue > Add snapshot compression to reduce the disk IO > -- > > Key: ZOOKEEPER-3179 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3179 > Project: ZooKeeper > Issue Type: Improvement >Reporter: Fangmin Lv >Assignee: Yisong Yue >Priority: Major > Fix For: 3.6.0 > > > When the snapshot becomes larger, the periodically snapshot after certain > number of txns will be more expensive. Which will in turn affect the maximum > throughput we can support within SLA, because of the disk contention between > snapshot and txn when they're on the same drive. > > With compression like zstd/snappy/gzip, the actual snapshot size could be > much smaller, the compress ratio depends on the actual data. It might make > the recovery time (loading from disk) faster in some cases, but will take > longer sometimes because of the extra time used to compress/decompress. > > Based on the production traffic, the performance various with different > compress method as well, that's why we provided different implementations, we > can select different compress method for different use cases. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2847) Cannot bind to client port when reconfig based on old static config
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633616#comment-16633616
]
Yisong Yue commented on ZOOKEEPER-2847:
---
^ seems like a flakey test case. I cannot reproduce the failure locally.
> Cannot bind to client port when reconfig based on old static config
> ---
>
> Key: ZOOKEEPER-2847
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2847
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.5.3, 3.6.0
>Reporter: Fangmin Lv
>Assignee: Yisong Yue
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.6.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> When started the ensemble with old static config that the server string
> doesn't have client port, dynamically remove and add the same server from the
> ensemble will cause that server cannot bind to client port, and the ZooKeeper
> server cannot serve client requests anymore.
> From the code, we'll set the clientAddr to null when start up with old static
> config, and dynamic config forces to have part, which will
> trigger the following rebind code in QuorumPeer#processReconfig, and cause
> the address already in used issue.
> public boolean processReconfig(QuorumVerifier qv, Long suggestedLeaderId,
> Long zxid, boolean restartLE) {
> ...
> if (myNewQS != null && myNewQS.clientAddr != null
> && !myNewQS.clientAddr.equals(oldClientAddr)) {
> cnxnFactory.reconfigure(myNewQS.clientAddr);
> updateThreadName();
> }
> ...
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2847) Cannot bind to client port when reconfig based on old static config
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632638#comment-16632638
]
Yisong Yue commented on ZOOKEEPER-2847:
---
[~andorm] [~hanm] Sorry for the delay! It has been a busy week. We'll submit a
patch for it today.
(Just found this comment didn't get sent out... :()
> Cannot bind to client port when reconfig based on old static config
> ---
>
> Key: ZOOKEEPER-2847
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2847
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.5.3, 3.6.0
>Reporter: Fangmin Lv
>Assignee: Yisong Yue
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.6.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When started the ensemble with old static config that the server string
> doesn't have client port, dynamically remove and add the same server from the
> ensemble will cause that server cannot bind to client port, and the ZooKeeper
> server cannot serve client requests anymore.
> From the code, we'll set the clientAddr to null when start up with old static
> config, and dynamic config forces to have part, which will
> trigger the following rebind code in QuorumPeer#processReconfig, and cause
> the address already in used issue.
> public boolean processReconfig(QuorumVerifier qv, Long suggestedLeaderId,
> Long zxid, boolean restartLE) {
> ...
> if (myNewQS != null && myNewQS.clientAddr != null
> && !myNewQS.clientAddr.equals(oldClientAddr)) {
> cnxnFactory.reconfigure(myNewQS.clientAddr);
> updateThreadName();
> }
> ...
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2847) Cannot bind to client port when reconfig based on old static config
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626522#comment-16626522
]
Yisong Yue commented on ZOOKEEPER-2847:
---
[~hanm] Yes, I'll put up a patch (by that you mean open another issue, right?)
to fix it. Thanks for investigating!
> Cannot bind to client port when reconfig based on old static config
> ---
>
> Key: ZOOKEEPER-2847
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2847
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.5.3, 3.6.0
>Reporter: Fangmin Lv
>Assignee: Yisong Yue
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.6.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When started the ensemble with old static config that the server string
> doesn't have client port, dynamically remove and add the same server from the
> ensemble will cause that server cannot bind to client port, and the ZooKeeper
> server cannot serve client requests anymore.
> From the code, we'll set the clientAddr to null when start up with old static
> config, and dynamic config forces to have part, which will
> trigger the following rebind code in QuorumPeer#processReconfig, and cause
> the address already in used issue.
> public boolean processReconfig(QuorumVerifier qv, Long suggestedLeaderId,
> Long zxid, boolean restartLE) {
> ...
> if (myNewQS != null && myNewQS.clientAddr != null
> && !myNewQS.clientAddr.equals(oldClientAddr)) {
> cnxnFactory.reconfigure(myNewQS.clientAddr);
> updateThreadName();
> }
> ...
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (ZOOKEEPER-2847) Cannot bind to client port when reconfig based on old static config
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yisong Yue reassigned ZOOKEEPER-2847:
-
Assignee: Yisong Yue
> Cannot bind to client port when reconfig based on old static config
> ---
>
> Key: ZOOKEEPER-2847
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2847
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Affects Versions: 3.5.3, 3.6.0
>Reporter: Fangmin Lv
>Assignee: Yisong Yue
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When started the ensemble with old static config that the server string
> doesn't have client port, dynamically remove and add the same server from the
> ensemble will cause that server cannot bind to client port, and the ZooKeeper
> server cannot serve client requests anymore.
> From the code, we'll set the clientAddr to null when start up with old static
> config, and dynamic config forces to have part, which will
> trigger the following rebind code in QuorumPeer#processReconfig, and cause
> the address already in used issue.
> public boolean processReconfig(QuorumVerifier qv, Long suggestedLeaderId,
> Long zxid, boolean restartLE) {
> ...
> if (myNewQS != null && myNewQS.clientAddr != null
> && !myNewQS.clientAddr.equals(oldClientAddr)) {
> cnxnFactory.reconfigure(myNewQS.clientAddr);
> updateThreadName();
> }
> ...
> }
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ZOOKEEPER-3063) Track outstanding changes with ArrayDeque
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yisong Yue updated ZOOKEEPER-3063: -- Summary: Track outstanding changes with ArrayDeque (was: track outstanding changes with ArrayDeque) > Track outstanding changes with ArrayDeque > - > > Key: ZOOKEEPER-3063 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3063 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Yisong Yue >Assignee: Yisong Yue >Priority: Trivial > Fix For: 3.6.0 > > > Outstanding changes are tracked with an ArrayList, which has O(N) remove from > head (and possibly add) performance. This means that as we get further > behind, we will slow down the processing of outstanding changes, which would > make us get further behind. > We should switch to using ArrayDeque which achieves O(1) add and remove on > both ends, which should result in much happiness. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ZOOKEEPER-3063) track outstanding changes with ArrayDeque
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yisong Yue updated ZOOKEEPER-3063: -- Description: Outstanding changes are tracked with an ArrayList, which has O(N) remove from head (and possibly add) performance. This means that as we get further behind, we will slow down the processing of outstanding changes, which would make us get further behind. We should switch to using ArrayDeque which achieves O(1) add and remove on both ends, which should result in much happiness. was: Outstanding changes are tracked with an ArrayList, which has O(n) remove from head (and possibly add) performance. This means that as we get further behind, we will slow down the processing of outstanding changes, which would make us get further behind. We should switch to using ArrayDeque which achieves O(1) add and remove on both ends, which should result in much happiness. > track outstanding changes with ArrayDeque > - > > Key: ZOOKEEPER-3063 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3063 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Reporter: Yisong Yue >Assignee: Yisong Yue >Priority: Trivial > Fix For: 3.6.0 > > > Outstanding changes are tracked with an ArrayList, which has O(N) remove from > head (and possibly add) performance. This means that as we get further > behind, we will slow down the processing of outstanding changes, which would > make us get further behind. > We should switch to using ArrayDeque which achieves O(1) add and remove on > both ends, which should result in much happiness. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZOOKEEPER-3063) track outstanding changes with ArrayDeque
Yisong Yue created ZOOKEEPER-3063: - Summary: track outstanding changes with ArrayDeque Key: ZOOKEEPER-3063 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3063 Project: ZooKeeper Issue Type: Improvement Components: server Reporter: Yisong Yue Assignee: Yisong Yue Fix For: 3.6.0 Outstanding changes are tracked with an ArrayList, which has O(n) remove from head (and possibly add) performance. This means that as we get further behind, we will slow down the processing of outstanding changes, which would make us get further behind. We should switch to using ArrayDeque which achieves O(1) add and remove on both ends, which should result in much happiness. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
