[GitHub] zookeeper issue #611: ZOOKEEPER-3131
Github user wangchaod commented on the issue: https://github.com/apache/zookeeper/pull/611 @lvfangmin OK thanks ---
[jira] [Commented] (ZOOKEEPER-1177) Enabling a large number of watches for a large number of clients
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599408#comment-16599408 ] Hadoop QA commented on ZOOKEEPER-1177: -- -1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 31 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 4 new Findbugs (version 3.0.1) warnings. -1 release audit. The applied patch generated 2 release audit warnings (more than the trunk's current 0 warnings). +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2109//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2109//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2109//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2109//console This message is automatically generated. > Enabling a large number of watches for a large number of clients > > > Key: ZOOKEEPER-1177 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1177 > Project: ZooKeeper > Issue Type: Improvement > Components: server >Affects Versions: 3.3.3 >Reporter: Vishal Kathuria >Assignee: Fangmin Lv >Priority: Major > Labels: pull-request-available > Fix For: 3.6.0, 3.5.5 > > Attachments: ZOOKEEPER-1177.patch, ZOOKEEPER-1177.patch, > ZooKeeper-with-fix-for-findbugs-warning.patch, ZooKeeper.patch, > Zookeeper-after-resolving-merge-conflicts.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > In my ZooKeeper, I see watch manager consuming several GB of memory and I dug > a bit deeper. > In the scenario I am testing, I have 10K clients connected to an observer. > There are about 20K znodes in ZooKeeper, each is about 1K - so about 20M data > in total. > Each client fetches and puts watches on all the znodes. That is 200 million > watches. > It seems a single watch takes about 100 bytes. I am currently at 14528037 > watches and according to the yourkit profiler, WatchManager has 1.2 G > already. This is not going to work as it might end up needing 20G of RAM just > for the watches. > So we need a more compact way of storing watches. Here are the possible > solutions. > 1. Use a bitmap instead of the current hashmap. In this approach, each znode > would get a unique id when its gets created. For every session, we can keep > track of a bitmap that indicates the set of znodes this session is watching. > A bitmap, assuming a 100K znodes, would be 12K. For 10K sessions, we can keep > track of watches using 120M instead of 20G. > 2. This second idea is based on the observation that clients watch znodes in > sets (for example all znodes under a folder). Multiple clients watch the same > set and the total number of sets is a couple of orders of magnitude smaller > than the total number of znodes. In my scenario, there are about 100 sets. So > instead of keeping track of watches at the znode level, keep track of it at > the set level. It may mean that get may also need to be implemented at the > set level. With this, we can save the watches in 100M. > Are there any other suggestions of solutions? > Thanks > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper issue #606: [ZOOKEEPER-3127] Fixing potential data inconsistency d...
Github user lvfangmin commented on the issue: https://github.com/apache/zookeeper/pull/606 The ReconfigRecoveryTest.testCurrentServersAreObserversInNextConfig is failing, I don't think it's related with this diff. @breed I think this is ready to get in now. ---
[GitHub] zookeeper issue #590: [ZOOKEEPER-1177] Add the memory optimized watch manage...
Github user lvfangmin commented on the issue: https://github.com/apache/zookeeper/pull/590 Added JMH micro benchmark for the watch manager: * It shows **big win** for the watch heavy cases, with the current implementation, it uses more than 50MB memory to store 1M watches, with WatchManagerOptimized it only uses around 0.2MB. * It also makes add and trigger watches more efficient, since WatchManagerOptimized doesn't maintain the reverse map. * In sparse watches use case, the WatchManagerOptimized is expected to use a bit more memory because it needs extra effort to maintain those bit set. In the test it shows around 10% more memory usage. Here are more result about the throughput/latency related with WatchManager: ``` Benchmark (pathCount)(watchManagerClass) (watcherCount) Mode Cnt ScoreError Units WatchBench.testAddConcentrateWatch1 WatchManager N/A avgt9 5.382 ± 0.968 ms/op WatchBench.testAddConcentrateWatch1 WatchManagerOptimized N/A avgt9 0.696 ± 0.133 ms/op WatchBench.testAddSparseWatch 1 WatchManager 1 avgt9 4.889 ± 1.585 ms/op WatchBench.testAddSparseWatch 1 WatchManagerOptimized 1 avgt9 4.794 ± 1.068 ms/op WatchBench.testTriggerConcentrateWatch1 WatchManager 1 avgt9 â 10â»â´ ms/op WatchBench.testTriggerConcentrateWatch1 WatchManager 1000 avgt9 0.037 ± 0.002 ms/op WatchBench.testTriggerConcentrateWatch1 WatchManagerOptimized 1 avgt9 â 10â»â´ ms/op WatchBench.testTriggerConcentrateWatch1 WatchManagerOptimized 1000 avgt9 0.025 ± 0.001 ms/op WatchBench.testTriggerConcentrateWatch 1000 WatchManager 1 avgt9 0.048 ± 0.003 ms/op WatchBench.testTriggerConcentrateWatch 1000 WatchManager 1000 avgt9 71.838 ± 4.043 ms/op WatchBench.testTriggerConcentrateWatch 1000 WatchManagerOptimized 1 avgt9 0.079 ± 0.002 ms/op WatchBench.testTriggerConcentrateWatch 1000 WatchManagerOptimized 1000 avgt9 26.135 ± 0.223 ms/op WatchBench.testTriggerSparseWatch 1 WatchManager 1 avgt9 1.207 ± 0.035 ms/op WatchBench.testTriggerSparseWatch 1 WatchManagerOptimized 1 avgt9 1.321 ± 0.019 ms/op ``` You can try the following command to run the micro benchmark: ``` $ ant clean package $ ant clean package -buildfile zookeeper-contrib/zookeeper-contrib-fatjar/build.xml $ java -jar build/contrib/fatjar/zookeeper-dev-fatjar.jar jmh ``` @maoling @anmolnar hope this gives you a more vivid comparison between the old and new watch manager implementation. ---
[jira] [Commented] (ZOOKEEPER-3137) add a utility to truncate logs to a zxid
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599372#comment-16599372 ] Hadoop QA commented on ZOOKEEPER-3137: -- -1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2108//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2108//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2108//console This message is automatically generated. > add a utility to truncate logs to a zxid > > > Key: ZOOKEEPER-3137 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3137 > Project: ZooKeeper > Issue Type: New Feature >Affects Versions: 3.6.0 >Reporter: Brian Nixon >Priority: Trivial > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Add a utility that allows an admin to truncate a given transaction log to a > specified zxid. This can be similar to the existent LogFormatter. > Among the benefits, this allows an admin to put together a point-in-time view > of a data tree by manually mutating files from a saved backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-3123) MetricsProvider Lifecycle in ZooKeeper Server
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599370#comment-16599370 ] Hadoop QA commented on ZOOKEEPER-3123: -- -1 overall. GitHub Pull Request Build +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 8 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2107//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2107//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2107//console This message is automatically generated. > MetricsProvider Lifecycle in ZooKeeper Server > - > > Key: ZOOKEEPER-3123 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3123 > Project: ZooKeeper > Issue Type: Sub-task > Components: metric system >Affects Versions: 3.6.0 >Reporter: Enrico Olivelli >Assignee: Enrico Olivelli >Priority: Major > Labels: pull-request-available > Fix For: 3.6.0 > > Time Spent: 4h 20m > Remaining Estimate: 0h > > This subtask is for the licefycle code of the configured MetricsProvider > inside ZooKeeper server, both standalone mode and quorum peer mode. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-3136) Reduce log in ClientBase in case of ConnectException
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16599369#comment-16599369
]
Hadoop QA commented on ZOOKEEPER-3136:
--
-1 overall. GitHub Pull Request Build
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2105//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2105//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2105//console
This message is automatically generated.
> Reduce log in ClientBase in case of ConnectException
>
>
> Key: ZOOKEEPER-3136
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3136
> Project: ZooKeeper
> Issue Type: Task
> Components: tests
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.6.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While running tests you will always see spammy log lines like the ones below.
> As we are expecting the server to be up, it is not useful to log such
> stacktraces.
> The patch simply reduce the log in this specific case, because it adds no
> value and it is very annoying.
>
> {code:java}
> [junit] 2018-08-31 23:31:49,173 [myid:] - INFO [main:ClientBase@292] -
> server 127.0.0.1:11222 not up
> [junit] java.net.ConnectException: Connection refused (Connection refused)
> [junit] at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit] at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> [junit] at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> [junit] at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> [junit] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> [junit] at java.net.Socket.connect(Socket.java:589)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:101)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:71)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:285)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:276)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] zookeeper issue #615: ZOOKEEPER-3137: add a utility to truncate logs to a zx...
Github user enixon commented on the issue: https://github.com/apache/zookeeper/pull/615 Sample usage is something like: `java -cp build/zookeeper-3.6.0-SNAPSHOT.jar:build/lib/* org.apache.zookeeper.server.util.LogChopper 0x3350003 log.333000b shorterlog.333000b` ---
[GitHub] zookeeper issue #601: ZOOKEEPER-3123 MetricsProvider Lifecycle in ZooKeeper ...
Github user eolivelli commented on the issue: https://github.com/apache/zookeeper/pull/601 @lvfangmin I have pushed a bunch of tests for the QuorumPeerMain case. As already said for the ZooKeeperServerMain there is much "copy and paste" for test cases, but this is current "style" of QuorumPeerMainTest. In the future we should clean up. The last commit is only about new testcases, I did not touch the core change you had already reviewed. ---
[jira] [Updated] (ZOOKEEPER-3137) add a utility to truncate logs to a zxid
[ https://issues.apache.org/jira/browse/ZOOKEEPER-3137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ZOOKEEPER-3137: -- Labels: pull-request-available (was: ) > add a utility to truncate logs to a zxid > > > Key: ZOOKEEPER-3137 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3137 > Project: ZooKeeper > Issue Type: New Feature >Affects Versions: 3.6.0 >Reporter: Brian Nixon >Priority: Trivial > Labels: pull-request-available > > Add a utility that allows an admin to truncate a given transaction log to a > specified zxid. This can be similar to the existent LogFormatter. > Among the benefits, this allows an admin to put together a point-in-time view > of a data tree by manually mutating files from a saved backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper pull request #615: ZOOKEEPER-3137: add a utility to truncate logs ...
GitHub user enixon opened a pull request: https://github.com/apache/zookeeper/pull/615 ZOOKEEPER-3137: add a utility to truncate logs to a zxid You can merge this pull request into a Git repository by running: $ git pull https://github.com/enixon/zookeeper add-chop Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/615.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #615 commit 77540aadc862397b32e3d11fb40c1427ed5729cb Author: Brian Nixon Date: 2018-08-31T22:44:34Z ZOOKEEPER-3137: add a utility to truncate logs to a zxid ---
[jira] [Updated] (ZOOKEEPER-3136) Reduce log in ClientBase in case of ConnectException
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enrico Olivelli updated ZOOKEEPER-3136:
---
Fix Version/s: 3.6.0
> Reduce log in ClientBase in case of ConnectException
>
>
> Key: ZOOKEEPER-3136
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3136
> Project: ZooKeeper
> Issue Type: Task
> Components: tests
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.6.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> While running tests you will always see spammy log lines like the ones below.
> As we are expecting the server to be up, it is not useful to log such
> stacktraces.
> The patch simply reduce the log in this specific case, because it adds no
> value and it is very annoying.
>
> {code:java}
> [junit] 2018-08-31 23:31:49,173 [myid:] - INFO [main:ClientBase@292] -
> server 127.0.0.1:11222 not up
> [junit] java.net.ConnectException: Connection refused (Connection refused)
> [junit] at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit] at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> [junit] at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> [junit] at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> [junit] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> [junit] at java.net.Socket.connect(Socket.java:589)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:101)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:71)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:285)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:276)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ZOOKEEPER-3136) Reduce log in ClientBase in case of ConnectException
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ZOOKEEPER-3136:
--
Labels: pull-request-available (was: )
> Reduce log in ClientBase in case of ConnectException
>
>
> Key: ZOOKEEPER-3136
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3136
> Project: ZooKeeper
> Issue Type: Task
> Components: tests
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.6.0
>
>
> While running tests you will always see spammy log lines like the ones below.
> As we are expecting the server to be up, it is not useful to log such
> stacktraces.
> The patch simply reduce the log in this specific case, because it adds no
> value and it is very annoying.
>
> {code:java}
> [junit] 2018-08-31 23:31:49,173 [myid:] - INFO [main:ClientBase@292] -
> server 127.0.0.1:11222 not up
> [junit] java.net.ConnectException: Connection refused (Connection refused)
> [junit] at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit] at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> [junit] at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> [junit] at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> [junit] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> [junit] at java.net.Socket.connect(Socket.java:589)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:101)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:71)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:285)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:276)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ZOOKEEPER-3137) add a utility to truncate logs to a zxid
Brian Nixon created ZOOKEEPER-3137: -- Summary: add a utility to truncate logs to a zxid Key: ZOOKEEPER-3137 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3137 Project: ZooKeeper Issue Type: New Feature Affects Versions: 3.6.0 Reporter: Brian Nixon Add a utility that allows an admin to truncate a given transaction log to a specified zxid. This can be similar to the existent LogFormatter. Among the benefits, this allows an admin to put together a point-in-time view of a data tree by manually mutating files from a saved backup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper pull request #614: ZOOKEEPER-3136 Reduce log in ClientBase in case...
GitHub user eolivelli opened a pull request: https://github.com/apache/zookeeper/pull/614 ZOOKEEPER-3136 Reduce log in ClientBase in case of ConnectException Do not log stacktraces of a very common error during the execution of tests, that is while waiting for a server to start. You can merge this pull request into a Git repository by running: $ git pull https://github.com/eolivelli/zookeeper fix/ZOOKEEPER-3136-noisy-log Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zookeeper/pull/614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #614 commit 3e1fe86de944edb97bc560c116be2e32bbadee1b Author: Enrico Olivelli Date: 2018-08-31T22:42:05Z ZOOKEEPER-3136 Reduce log in ClientBase in case of ConnectException ---
[jira] [Updated] (ZOOKEEPER-3136) Reduce log in ClientBase in case of ConnectException
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enrico Olivelli updated ZOOKEEPER-3136:
---
Summary: Reduce log in ClientBase in case of ConnectException (was: Reduce
log in ClientBase in case of "ConnectException")
> Reduce log in ClientBase in case of ConnectException
>
>
> Key: ZOOKEEPER-3136
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3136
> Project: ZooKeeper
> Issue Type: Task
> Components: tests
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Minor
>
> While running tests you will always see spammy log lines like the ones below.
> As we are expecting the server to be up, it is not useful to log such
> stacktraces.
> The patch simply reduce the log in this specific case, because it adds no
> value and it is very annoying.
>
> {code:java}
> [junit] 2018-08-31 23:31:49,173 [myid:] - INFO [main:ClientBase@292] -
> server 127.0.0.1:11222 not up
> [junit] java.net.ConnectException: Connection refused (Connection refused)
> [junit] at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit] at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> [junit] at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> [junit] at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> [junit] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> [junit] at java.net.Socket.connect(Socket.java:589)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:101)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:71)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:285)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:276)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ZOOKEEPER-3136) Reduce log in ClientBase in case of "ConnectException"
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enrico Olivelli updated ZOOKEEPER-3136:
---
Summary: Reduce log in ClientBase in case of "ConnectException" (was:
Reduce log in FourLetterWordMain in case of "ConnectException")
> Reduce log in ClientBase in case of "ConnectException"
> --
>
> Key: ZOOKEEPER-3136
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3136
> Project: ZooKeeper
> Issue Type: Task
> Components: tests
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Minor
>
> While running tests you will always see spammy log lines like the ones below.
> As we are expecting the server to be up, it is not useful to log such
> stacktraces.
> The patch simply reduce the log in this specific case, because it adds no
> value and it is very annoying.
>
> {code:java}
> [junit] 2018-08-31 23:31:49,173 [myid:] - INFO [main:ClientBase@292] -
> server 127.0.0.1:11222 not up
> [junit] java.net.ConnectException: Connection refused (Connection refused)
> [junit] at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit] at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> [junit] at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> [junit] at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> [junit] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> [junit] at java.net.Socket.connect(Socket.java:589)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:101)
> [junit] at
> org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:71)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:285)
> [junit] at
> org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:276)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ZOOKEEPER-3136) Reduce log in FourLetterWordMain in case of "ConnectException"
Enrico Olivelli created ZOOKEEPER-3136:
--
Summary: Reduce log in FourLetterWordMain in case of
"ConnectException"
Key: ZOOKEEPER-3136
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3136
Project: ZooKeeper
Issue Type: Task
Components: tests
Reporter: Enrico Olivelli
Assignee: Enrico Olivelli
While running tests you will always see spammy log lines like the ones below.
As we are expecting the server to be up, it is not useful to log such
stacktraces.
The patch simply reduce the log in this specific case, because it adds no value
and it is very annoying.
{code:java}
[junit] 2018-08-31 23:31:49,173 [myid:] - INFO [main:ClientBase@292] -
server 127.0.0.1:11222 not up
[junit] java.net.ConnectException: Connection refused (Connection refused)
[junit] at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit] at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
[junit] at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
[junit] at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
[junit] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[junit] at java.net.Socket.connect(Socket.java:589)
[junit] at
org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:101)
[junit] at
org.apache.zookeeper.client.FourLetterWordMain.send4LetterWord(FourLetterWordMain.java:71)
[junit] at
org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:285)
[junit] at
org.apache.zookeeper.test.ClientBase.waitForServerUp(ClientBase.java:276)
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2284) LogFormatter and SnapshotFormatter does not handle FileNotFoundException gracefully
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598688#comment-16598688
]
Hadoop QA commented on ZOOKEEPER-2284:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12764303/ZOOKEEPER-2284-04.patch
against trunk revision 4d7b9e8f34775261fed517d232094f0247a4d3cf.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/3702//console
This message is automatically generated.
> LogFormatter and SnapshotFormatter does not handle FileNotFoundException
> gracefully
> ---
>
> Key: ZOOKEEPER-2284
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2284
> Project: ZooKeeper
> Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Mohammad Arshad
>Assignee: maoling
>Priority: Minor
> Fix For: 3.6.0, 3.5.5
>
> Attachments: ZOOKEEPER-2284-01.patch, ZOOKEEPER-2284-02.patch,
> ZOOKEEPER-2284-03.patch, ZOOKEEPER-2284-04.patch
>
>
> {{LogFormatter}} and {{SnapshotFormatter}} does not handle
> FileNotFoundException gracefully. If file no exist then these classes
> propagate the exception to console.
> {code}
> Exception in thread "main" java.io.FileNotFoundException: log.1 (The system
> cannot find the file specified)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.(FileInputStream.java:146)
> at java.io.FileInputStream.(FileInputStream.java:101)
> at org.apache.zookeeper.server.LogFormatter.main(LogFormatter.java:49)
> {code}
> File existence should be validated and appropriate message should be
> displayed on console if file does not exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-2284) LogFormatter and SnapshotFormatter does not handle FileNotFoundException gracefully
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598684#comment-16598684
]
Mohammad Arshad commented on ZOOKEEPER-2284:
Please take it. Assigned to you.
> LogFormatter and SnapshotFormatter does not handle FileNotFoundException
> gracefully
> ---
>
> Key: ZOOKEEPER-2284
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2284
> Project: ZooKeeper
> Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Mohammad Arshad
>Assignee: maoling
>Priority: Minor
> Fix For: 3.6.0, 3.5.5
>
> Attachments: ZOOKEEPER-2284-01.patch, ZOOKEEPER-2284-02.patch,
> ZOOKEEPER-2284-03.patch, ZOOKEEPER-2284-04.patch
>
>
> {{LogFormatter}} and {{SnapshotFormatter}} does not handle
> FileNotFoundException gracefully. If file no exist then these classes
> propagate the exception to console.
> {code}
> Exception in thread "main" java.io.FileNotFoundException: log.1 (The system
> cannot find the file specified)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.(FileInputStream.java:146)
> at java.io.FileInputStream.(FileInputStream.java:101)
> at org.apache.zookeeper.server.LogFormatter.main(LogFormatter.java:49)
> {code}
> File existence should be validated and appropriate message should be
> displayed on console if file does not exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (ZOOKEEPER-2284) LogFormatter and SnapshotFormatter does not handle FileNotFoundException gracefully
[
https://issues.apache.org/jira/browse/ZOOKEEPER-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Arshad reassigned ZOOKEEPER-2284:
--
Assignee: maoling (was: Mohammad Arshad)
> LogFormatter and SnapshotFormatter does not handle FileNotFoundException
> gracefully
> ---
>
> Key: ZOOKEEPER-2284
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2284
> Project: ZooKeeper
> Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Mohammad Arshad
>Assignee: maoling
>Priority: Minor
> Fix For: 3.6.0, 3.5.5
>
> Attachments: ZOOKEEPER-2284-01.patch, ZOOKEEPER-2284-02.patch,
> ZOOKEEPER-2284-03.patch, ZOOKEEPER-2284-04.patch
>
>
> {{LogFormatter}} and {{SnapshotFormatter}} does not handle
> FileNotFoundException gracefully. If file no exist then these classes
> propagate the exception to console.
> {code}
> Exception in thread "main" java.io.FileNotFoundException: log.1 (The system
> cannot find the file specified)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.(FileInputStream.java:146)
> at java.io.FileInputStream.(FileInputStream.java:101)
> at org.apache.zookeeper.server.LogFormatter.main(LogFormatter.java:49)
> {code}
> File existence should be validated and appropriate message should be
> displayed on console if file does not exist
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (ZOOKEEPER-1823) LogFormatter should support printing transaction data as a string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling reassigned ZOOKEEPER-1823: -- Assignee: maoling (was: Raul Gutierrez Segales) > LogFormatter should support printing transaction data as a string > - > > Key: ZOOKEEPER-1823 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1823 > Project: ZooKeeper > Issue Type: Bug > Components: server >Reporter: Raul Gutierrez Segales >Assignee: maoling >Priority: Trivial > Labels: pull-request-available > Fix For: 3.6.0, 3.5.5 > > Attachments: ZOOKEEPER-1823.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Some times it's handy to have LogFormatter show you the content of the > transactions (i.e.: if you are storing text). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] zookeeper pull request #613: ZOOKEEPER-1823:LogFormatter should support prin...
GitHub user maoling opened a pull request:
https://github.com/apache/zookeeper/pull/613
ZOOKEEPER-1823:LogFormatter should support printing transaction data as a
string
- It would be a useful addition for debuggling to show transaction data as
strings with a option `-s `
- update the usage string to `USAGE: LogFormatter [-s] log_file` and add a
short description for the `-s `
option
- for the issue
[michim](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=michim)
mentioned has been resolved,the test evidence has been included in the
jira(Notice:we cannot `split(",")[1]` to replace the node data,because the node
path can be like this `"/mao,ling"`)
- more details in
[ZOOKEEPER-1823](https://issues.apache.org/jira/browse/ZOOKEEPER-1823).Thanks
[rgs](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=rgs) for the
origin work
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/maoling/zookeeper ZOOKEEPER-1823
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/613.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #613
commit cf1f55bc023cde7921b6258597a832f6d097ea28
Author: maoling
Date: 2018-08-31T11:12:05Z
ZOOKEEPER-1823:LogFormatter should support printing transaction data as a
string
---
[jira] [Updated] (ZOOKEEPER-1823) LogFormatter should support printing transaction data as a string
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ZOOKEEPER-1823: -- Labels: pull-request-available (was: ) > LogFormatter should support printing transaction data as a string > - > > Key: ZOOKEEPER-1823 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1823 > Project: ZooKeeper > Issue Type: Bug > Components: server >Reporter: Raul Gutierrez Segales >Assignee: Raul Gutierrez Segales >Priority: Trivial > Labels: pull-request-available > Fix For: 3.6.0, 3.5.5 > > Attachments: ZOOKEEPER-1823.patch > > > Some times it's handy to have LogFormatter show you the content of the > transactions (i.e.: if you are storing text). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-1823) LogFormatter should support printing transaction data as a string
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598609#comment-16598609
]
maoling commented on ZOOKEEPER-1823:
---
*with -s option:*
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
18-8-31 下午06时27分22秒 session 0x1658f85c391 cxid 0x0 zxid 0x90002242c
createSession 3
18-8-31 下午06时29分05秒 session 0x1658f85c391 cxid 0x1 zxid 0x90002242d create
/president,Obama,[31,s\{'world,'anyone}
],false,88
18-8-31 下午06时41分12秒 session 0x1658f85c391 cxid 0x2 zxid 0x90002242e setData
/president,Trump,1
EOF reached after 3 txns.
---
*without -s option:*
ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
18-8-31 下午06时27分22秒 session 0x1658f85c391 cxid 0x0 zxid 0x90002242c
createSession 3
18-8-31 下午06时29分05秒 session 0x1658f85c391 cxid 0x1 zxid 0x90002242d create
'/president,#4f62616d61,v\{s{31,s{'world,'anyone}}},F,88
18-8-31 下午06时41分12秒 session 0x1658f85c391 cxid 0x2 zxid 0x90002242e setData
'/president,#5472756d70,1
EOF reached after 3 txns.
> LogFormatter should support printing transaction data as a string
> -
>
> Key: ZOOKEEPER-1823
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1823
> Project: ZooKeeper
> Issue Type: Bug
> Components: server
>Reporter: Raul Gutierrez Segales
>Assignee: Raul Gutierrez Segales
>Priority: Trivial
> Fix For: 3.6.0, 3.5.5
>
> Attachments: ZOOKEEPER-1823.patch
>
>
> Some times it's handy to have LogFormatter show you the content of the
> transactions (i.e.: if you are storing text).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ZOOKEEPER-3080) Step 1.5 - Separate jute structure
[
https://issues.apache.org/jira/browse/ZOOKEEPER-3080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16598563#comment-16598563
]
Hadoop QA commented on ZOOKEEPER-3080:
--
+1 overall. GitHub Pull Request Build
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 3.0.1)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2102//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2102//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2102//console
This message is automatically generated.
> Step 1.5 - Separate jute structure
> --
>
> Key: ZOOKEEPER-3080
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3080
> Project: ZooKeeper
> Issue Type: Sub-task
> Components: build, scripts
>Affects Versions: 3.6.0
>Reporter: Norbert Kalmar
>Assignee: Norbert Kalmar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Create a project structure that separates the different parts of ZooKeeper
> into a more meaningful packages for the future maven build.
> This should be done in iterations to limit the impact.
> * First iteration - safe changes including moving src/docs to zk-docs,
> creating zk-it empty directory. Build and conf directory remains unchanged.
> These changes also have minimum impact on PR’s.
> * Second iteration - move src/recipes to zk-recipes.
> * Third iteration - move src/contrib to zk-contrib.
> * Fourth iteration - move src/c to zk-client (java will be moved in Phase 2)
> * *Fifth iteration* - move jute under src directory
> * Sixth iteration - move src/java/main to zk-server, which will be further
> separated in Step 2.
> {noformat}
> zookeeper
> | -bin
> | -conf
> | -jute
> | -zookeeper-client
> || -zookeeper-client-c
> | -zookeeper-contrib
> || -zookeeper-contrib-fatjar
> || -zookeeper-contrib-huebrowser
> || -zookeeper-contrib-loggraph
> || -zookeeper-contrib-monitoring
> || -zookeeper-contrib-rest
> || -zookeeper-contrib-zkfuse
> || -zookeeper-contrib-zkperl
> || -zookeeper-contrib-zkpython
> || -zookeeper-contrib-zktreeutil
> |\ -zookeeper-contrib-zooinspector
> | -zookeeper-docs
> | -zookeeper-it (integration tests)
> | -zookeeper-server
> | -zookeeper-recipes
> || -zookeeper-recipes-election
> || -zookeeper-recipes-lock
> \\ -zookeeper-recipes-queue
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] zookeeper issue #610: [ZOOKEEPER-3124] Remove special logic to handle cversi...
Github user nkalmar commented on the issue: https://github.com/apache/zookeeper/pull/610 Looks like the C test failed, can you please re-run if Jenkins throws the same error? (just add an empty --amend commit). ---
[GitHub] zookeeper issue #610: [ZOOKEEPER-3124] Remove special logic to handle cversi...
Github user nkalmar commented on the issue: https://github.com/apache/zookeeper/pull/610 Oh, nice, thanks @lvfangmin ! I will take a look on both then. ---
