[jira] [Commented] (ZOOKEEPER-1621) ZooKeeper does not recover from crash when disk was full

[Abhishek Rai (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616237#comment-15616237
 ] 

Abhishek Rai commented on ZOOKEEPER-1621:
-

Thanks [~mkizner].  Your suggestion of doing this only for the most recent txn 
log file is sound.  Are you also suggesting that we delete this truncated txn 
log file?

Cause, if we skip it and don't delete, then in the future, newer txn log files 
will get created.  So, the truncated txn log file will no longer be the latest 
txn log when we do a purge afterwards.

Deletion seems consistent with this approach as well as consistent with 
PurgeTxnLog's behavior.

> ZooKeeper does not recover from crash when disk was full
> 
>
> Key: ZOOKEEPER-1621
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1621
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.3
> Environment: Ubuntu 12.04, Amazon EC2 instance
>Reporter: David Arthur
>Assignee: Michi Mutsuzaki
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-1621.patch, zookeeper.log.gz
>
>
> The disk that ZooKeeper was using filled up. During a snapshot write, I got 
> the following exception
> 2013-01-16 03:11:14,098 - ERROR [SyncThread:0:SyncRequestProcessor@151] - 
> Severe unrecoverable error, exiting
> java.io.IOException: No space left on device
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:282)
> at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog.commit(FileTxnLog.java:309)
> at 
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.commit(FileTxnSnapLog.java:306)
> at org.apache.zookeeper.server.ZKDatabase.commit(ZKDatabase.java:484)
> at 
> org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:162)
> at 
> org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:101)
> Then many subsequent exceptions like:
> 2013-01-16 15:02:23,984 - ERROR [main:Util@239] - Last transaction was 
> partial.
> 2013-01-16 15:02:23,985 - ERROR [main:ZooKeeperServerMain@63] - Unexpected 
> exception, exiting abnormally
> java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at 
> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
> at 
> org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:558)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:577)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:543)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:625)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.init(FileTxnLog.java:529)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.(FileTxnLog.java:504)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog.read(FileTxnLog.java:341)
> at 
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:130)
> at 
> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
> at 
> org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:259)
> at 
> org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:386)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:138)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:112)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:86)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:52)
> at 
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
> at 
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
> It seems to me that writing the transaction log should be fully atomic to 
> avoid such situations. Is this not the case?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper-trunk-jdk8 - Build # 799 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-jdk8/799/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 447322 lines...]
[junit] 2016-10-28 18:13:11,714 [myid:] - INFO  
[main:PrepRequestProcessor@996] - Shutting down
[junit] 2016-10-28 18:13:11,715 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2016-10-28 18:13:11,715 [myid:] - INFO  [ProcessThread(sid:0 
cport:11466)::PrepRequestProcessor@156] - PrepRequestProcessor exited loop!
[junit] 2016-10-28 18:13:11,715 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2016-10-28 18:13:11,715 [myid:] - INFO  
[main:FinalRequestProcessor@481] - shutdown of request processor complete
[junit] 2016-10-28 18:13:11,715 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port11466,name1=InMemoryDataTree]
[junit] 2016-10-28 18:13:11,716 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port11466]
[junit] 2016-10-28 18:13:11,716 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 11466
[junit] 2016-10-28 18:13:11,716 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2016-10-28 18:13:11,720 [myid:] - INFO  [main:ClientBase@568] - 
fdcount after test is: 4821 at start it was 4823
[junit] 2016-10-28 18:13:11,720 [myid:] - INFO  [main:ZKTestCase$1@65] - 
SUCCEEDED testWatcherAutoResetWithLocal
[junit] 2016-10-28 18:13:11,721 [myid:] - INFO  [main:ZKTestCase$1@60] - 
FINISHED testWatcherAutoResetWithLocal
[junit] Tests run: 101, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
541.537 sec, Thread: 1, Class: org.apache.zookeeper.test.NioNettySuiteTest
[junit] 2016-10-28 18:14:03,558 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 131492
[junit] 2016-10-28 18:14:03,558 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 55
[junit] 2016-10-28 18:14:03,559 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testManyChildWatchersAutoReset
[junit] 2016-10-28 18:14:03,559 [myid:] - INFO  [main:ClientBase@543] - 
tearDown starting
[junit] 2016-10-28 18:14:03,560 [myid:] - INFO  [ProcessThread(sid:0 
cport:21997)::PrepRequestProcessor@605] - Processed session termination for 
sessionid: 0x10311c21a68
[junit] 2016-10-28 18:14:03,575 [myid:] - INFO  [main:ZooKeeper@1315] - 
Session: 0x10311c21a68 closed
[junit] 2016-10-28 18:14:03,575 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x10311c21a68
[junit] 2016-10-28 18:14:03,585 [myid:] - INFO  [ProcessThread(sid:0 
cport:21997)::PrepRequestProcessor@605] - Processed session termination for 
sessionid: 0x10311c21a680001
[junit] 2016-10-28 18:14:03,585 [myid:] - WARN  
[NIOWorkerThread-21:NIOServerCnxn@365] - Unable to read additional data from 
client sessionid 0x10311c21a68, likely client has closed socket
[junit] 2016-10-28 18:14:03,586 [myid:] - INFO  
[NIOWorkerThread-21:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port21997,name1=Connections,name2=127.0.0.1,name3=0x10311c21a68]
[junit] 2016-10-28 18:14:03,587 [myid:] - INFO  
[NIOWorkerThread-21:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:46911 which had sessionid 0x10311c21a68
[junit] 2016-10-28 18:14:03,616 [myid:] - INFO  [main:ZooKeeper@1315] - 
Session: 0x10311c21a680001 closed
[junit] 2016-10-28 18:14:03,617 [myid:] - INFO  [main:ClientBase@513] - 
STOPPING server
[junit] 2016-10-28 18:14:03,616 [myid:] - INFO  
[NIOWorkerThread-1:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port21997,name1=Connections,name2=127.0.0.1,name3=0x10311c21a680001]
[junit] 2016-10-28 18:14:03,617 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x10311c21a680001
[junit] 2016-10-28 18:14:03,617 [myid:] - INFO  
[NIOWorkerThread-1:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:46920 which had sessionid 0x10311c21a680001
[junit] 2016-10-28 18:14:03,619 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:21997:NIOServerCnxnFactory$AcceptThread@219]
 - accept thread exitted run method
[junit] 2016-10-28 18:14:03,623 [myid:] - INFO  
[ConnnectionExpirer:NIOServerCnxnFactory$ConnectionExpirerThread@583] - 
ConnnectionExpirerThread interrupted
[junit] 2016-10-28 18:14:03,629 [myid:] - INFO  
[NIOServerCxnFactory.SelectorThread-0:NIOServerCnxnFactory$SelectorThread@420] 
- selector thread exitted run method
[junit] 2016-10-28 18:14:03,630 

[jira] [Commented] (ZOOKEEPER-1621) ZooKeeper does not recover from crash when disk was full

[Meyer Kizner (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616169#comment-15616169
 ] 

Meyer Kizner commented on ZOOKEEPER-1621:
-

Agreed. Forcing users to manually clean up the partial/empty header in this 
scenario seems undesirable, and if we only catch EOFException instead of 
IOException, we shouldn't run into any problems with correctness. Additionally, 
since this issue should only occur "legitimately" in the most recent txn log 
file, we can be even more conservative and only continue in that case.

> ZooKeeper does not recover from crash when disk was full
> 
>
> Key: ZOOKEEPER-1621
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1621
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.3
> Environment: Ubuntu 12.04, Amazon EC2 instance
>Reporter: David Arthur
>Assignee: Michi Mutsuzaki
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-1621.patch, zookeeper.log.gz
>
>
> The disk that ZooKeeper was using filled up. During a snapshot write, I got 
> the following exception
> 2013-01-16 03:11:14,098 - ERROR [SyncThread:0:SyncRequestProcessor@151] - 
> Severe unrecoverable error, exiting
> java.io.IOException: No space left on device
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:282)
> at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog.commit(FileTxnLog.java:309)
> at 
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.commit(FileTxnSnapLog.java:306)
> at org.apache.zookeeper.server.ZKDatabase.commit(ZKDatabase.java:484)
> at 
> org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:162)
> at 
> org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:101)
> Then many subsequent exceptions like:
> 2013-01-16 15:02:23,984 - ERROR [main:Util@239] - Last transaction was 
> partial.
> 2013-01-16 15:02:23,985 - ERROR [main:ZooKeeperServerMain@63] - Unexpected 
> exception, exiting abnormally
> java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at 
> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
> at 
> org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:558)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:577)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:543)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:625)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.init(FileTxnLog.java:529)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.(FileTxnLog.java:504)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog.read(FileTxnLog.java:341)
> at 
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:130)
> at 
> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
> at 
> org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:259)
> at 
> org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:386)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:138)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:112)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:86)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:52)
> at 
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
> at 
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
> It seems to me that writing the transaction log should be fully atomic to 
> avoid such situations. Is this not the case?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper_branch35_jdk8 - Build # 282 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch35_jdk8/282/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 426046 lines...]
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-10-28 18:53:11,553 [myid:127.0.0.1:11346] - INFO  
[main-SendThread(127.0.0.1:11346):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:11346. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 18:53:11,554 [myid:127.0.0.1:11346] - WARN  
[main-SendThread(127.0.0.1:11346):ClientCnxn$SendThread@1235] - Session 
0x2005bc1dc15 for server 127.0.0.1/127.0.0.1:11346, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-10-28 18:53:11,865 [myid:] - INFO  [ProcessThread(sid:0 
cport:11466)::PrepRequestProcessor@647] - Processed session termination for 
sessionid: 0x1005bc53011
[junit] 2016-10-28 18:53:11,868 [myid:] - INFO  [main:ZooKeeper@1313] - 
Session: 0x1005bc53011 closed
[junit] 2016-10-28 18:53:11,868 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 102695
[junit] 2016-10-28 18:53:11,868 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 468
[junit] 2016-10-28 18:53:11,868 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testWatcherAutoResetWithLocal
[junit] 2016-10-28 18:53:11,869 [myid:] - INFO  [main:ClientBase@543] - 
tearDown starting
[junit] 2016-10-28 18:53:11,869 [myid:] - INFO  [main:ClientBase@513] - 
STOPPING server
[junit] 2016-10-28 18:53:11,869 [myid:] - INFO  
[main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:11466
[junit] 2016-10-28 18:53:11,868 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x1005bc53011
[junit] 2016-10-28 18:53:11,868 [myid:] - INFO  
[SyncThread:0:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port11466,name1=Connections,name2=127.0.0.1,name3=0x1005bc53011]
[junit] 2016-10-28 18:53:11,871 [myid:] - INFO  [main:ZooKeeperServer@529] 
- shutting down
[junit] 2016-10-28 18:53:11,871 [myid:] - ERROR [main:ZooKeeperServer@501] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-10-28 18:53:11,872 [myid:] - INFO  
[main:SessionTrackerImpl@232] - Shutting down
[junit] 2016-10-28 18:53:11,872 [myid:] - INFO  
[main:PrepRequestProcessor@965] - Shutting down
[junit] 2016-10-28 18:53:11,872 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2016-10-28 18:53:11,872 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2016-10-28 18:53:11,873 [myid:] - INFO  
[main:FinalRequestProcessor@479] - shutdown of request processor complete
[junit] 2016-10-28 18:53:11,873 [myid:] - INFO  [ProcessThread(sid:0 
cport:11466)::PrepRequestProcessor@154] - PrepRequestProcessor exited loop!
[junit] 2016-10-28 18:53:11,873 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port11466,name1=InMemoryDataTree]
[junit] 2016-10-28 18:53:11,874 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port11466]
[junit] 2016-10-28 18:53:11,874 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 11466
[junit] 2016-10-28 18:53:11,875 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2016-10-28 18:53:11,882 [myid:] - INFO  [main:ClientBase@568] - 
fdcount after test is: 1377 at start it was 1377
[junit] 2016-10-28 18:53:11,883 [myid:] - INFO  [main:ZKTestCase$1@65] - 
SUCCEEDED testWatcherAutoResetWithLocal
[junit] 2016-10-28 18:53:11,883 [myid:] - INFO  [main:ZKTestCase$1@60] - 
FINISHED testWatcherAutoResetWithLocal
[junit] Tests run: 101, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
467.717 sec, Thread: 1, Class: org.apache.zookeeper.test.NioNettySuiteTest
[junit] 2016-10-28 18:53:11,938 

[jira] [Commented] (ZOOKEEPER-2346) SASL Auth failure manifested to client as connection refusal

[Meyer Kizner (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616074#comment-15616074
 ] 

Meyer Kizner commented on ZOOKEEPER-2346:
-

It looks like this is a race in the code that handles SASL authentication 
failures. While testing out SASL on our installation, I've observed both the 
behavior described in this issue and a more correct version, in which the 
server sends a null SASL token back to the client before closing the connection.

I have a short patch for this, but it doesn't look like I can upload it unless 
I'm assigned this issue. Can someone fix that for me?

> SASL Auth failure manifested to client as connection refusal
> 
>
> Key: ZOOKEEPER-2346
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2346
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.6
>Reporter: Steve Loughran
>
> If a client can't authenticate via sasl then (a) the stack trace is lost on 
> the server logs, and (b) it is exposed to the client as a connection refusal. 
> This results in curator retrying many times before giving up —and with the 
> cause being misinterpreted as a server-down problem, rather than a 
> client-not-trusted problem



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ZOOKEEPER-1621) ZooKeeper does not recover from crash when disk was full

[Meyer Kizner (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616298#comment-15616298
 ] 

Meyer Kizner commented on ZOOKEEPER-1621:
-

Yes, we would have to delete such a log file upon encountering it. I don't 
believe this would cause any problems, and it seems desirable to have the extra 
check this enables.

> ZooKeeper does not recover from crash when disk was full
> 
>
> Key: ZOOKEEPER-1621
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1621
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.3
> Environment: Ubuntu 12.04, Amazon EC2 instance
>Reporter: David Arthur
>Assignee: Michi Mutsuzaki
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-1621.patch, zookeeper.log.gz
>
>
> The disk that ZooKeeper was using filled up. During a snapshot write, I got 
> the following exception
> 2013-01-16 03:11:14,098 - ERROR [SyncThread:0:SyncRequestProcessor@151] - 
> Severe unrecoverable error, exiting
> java.io.IOException: No space left on device
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:282)
> at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog.commit(FileTxnLog.java:309)
> at 
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.commit(FileTxnSnapLog.java:306)
> at org.apache.zookeeper.server.ZKDatabase.commit(ZKDatabase.java:484)
> at 
> org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:162)
> at 
> org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:101)
> Then many subsequent exceptions like:
> 2013-01-16 15:02:23,984 - ERROR [main:Util@239] - Last transaction was 
> partial.
> 2013-01-16 15:02:23,985 - ERROR [main:ZooKeeperServerMain@63] - Unexpected 
> exception, exiting abnormally
> java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at 
> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
> at 
> org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:558)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:577)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:543)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:625)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.init(FileTxnLog.java:529)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.(FileTxnLog.java:504)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog.read(FileTxnLog.java:341)
> at 
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:130)
> at 
> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
> at 
> org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:259)
> at 
> org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:386)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:138)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:112)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:86)
> at 
> org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:52)
> at 
> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
> at 
> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
> It seems to me that writing the transaction log should be fully atomic to 
> avoid such situations. Is this not the case?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper_branch35_solaris - Build # 299 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch35_solaris/299/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 436231 lines...]
[junit] 2016-10-28 17:16:31,063 [myid:] - INFO  [main:ClientBase@386] - 
CREATING server instance 127.0.0.1:11222
[junit] 2016-10-28 17:16:31,063 [myid:] - INFO  
[main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s 
sessionless connection timeout, 2 selector thread(s), 16 worker threads, and 64 
kB direct buffers.
[junit] 2016-10-28 17:16:31,064 [myid:] - INFO  
[main:NIOServerCnxnFactory@686] - binding to port 0.0.0.0/0.0.0.0:11222
[junit] 2016-10-28 17:16:31,065 [myid:] - INFO  [main:ClientBase@361] - 
STARTING server instance 127.0.0.1:11222
[junit] 2016-10-28 17:16:31,065 [myid:] - INFO  [main:ZooKeeperServer@889] 
- minSessionTimeout set to 6000
[junit] 2016-10-28 17:16:31,065 [myid:] - INFO  [main:ZooKeeperServer@898] 
- maxSessionTimeout set to 6
[junit] 2016-10-28 17:16:31,065 [myid:] - INFO  [main:ZooKeeperServer@159] 
- Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 
6 datadir 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch35_solaris/build/test/tmp/test2820682500032776974.junit.dir/version-2
 snapdir 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch35_solaris/build/test/tmp/test2820682500032776974.junit.dir/version-2
[junit] 2016-10-28 17:16:31,066 [myid:] - INFO  [main:FileSnap@83] - 
Reading snapshot 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch35_solaris/build/test/tmp/test2820682500032776974.junit.dir/version-2/snapshot.b
[junit] 2016-10-28 17:16:31,068 [myid:] - INFO  [main:FileTxnSnapLog@306] - 
Snapshotting: 0xb to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper_branch35_solaris/build/test/tmp/test2820682500032776974.junit.dir/version-2/snapshot.b
[junit] 2016-10-28 17:16:31,069 [myid:] - ERROR [main:ZooKeeperServer@501] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-10-28 17:16:31,069 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 11222
[junit] 2016-10-28 17:16:31,070 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11222:NIOServerCnxnFactory$AcceptThread@296]
 - Accepted socket connection from /127.0.0.1:40922
[junit] 2016-10-28 17:16:31,070 [myid:] - INFO  
[NIOWorkerThread-1:NIOServerCnxn@485] - Processing stat command from 
/127.0.0.1:40922
[junit] 2016-10-28 17:16:31,071 [myid:] - INFO  
[NIOWorkerThread-1:StatCommand@49] - Stat command output
[junit] 2016-10-28 17:16:31,071 [myid:] - INFO  
[NIOWorkerThread-1:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:40922 (no session established for client)
[junit] 2016-10-28 17:16:31,071 [myid:] - INFO  [main:JMXEnv@228] - 
ensureParent:[InMemoryDataTree, StandaloneServer_port]
[junit] 2016-10-28 17:16:31,073 [myid:] - INFO  [main:JMXEnv@245] - 
expect:InMemoryDataTree
[junit] 2016-10-28 17:16:31,073 [myid:] - INFO  [main:JMXEnv@249] - 
found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port11222,name1=InMemoryDataTree
[junit] 2016-10-28 17:16:31,073 [myid:] - INFO  [main:JMXEnv@245] - 
expect:StandaloneServer_port
[junit] 2016-10-28 17:16:31,073 [myid:] - INFO  [main:JMXEnv@249] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11222
[junit] 2016-10-28 17:16:31,073 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 17683
[junit] 2016-10-28 17:16:31,073 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 24
[junit] 2016-10-28 17:16:31,074 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testQuota
[junit] 2016-10-28 17:16:31,074 [myid:] - INFO  [main:ClientBase@543] - 
tearDown starting
[junit] 2016-10-28 17:16:31,153 [myid:] - INFO  [main:ZooKeeper@1313] - 
Session: 0x1248945a8bb closed
[junit] 2016-10-28 17:16:31,153 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x1248945a8bb
[junit] 2016-10-28 17:16:31,153 [myid:] - INFO  [main:ClientBase@513] - 
STOPPING server
[junit] 2016-10-28 17:16:31,153 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11222:NIOServerCnxnFactory$AcceptThread@219]
 - accept thread exitted run method
[junit] 2016-10-28 17:16:31,154 [myid:] - INFO  
[ConnnectionExpirer:NIOServerCnxnFactory$ConnectionExpirerThread@583] - 
ConnnectionExpirerThread interrupted
[junit] 2016-10-28 17:16:31,154 [myid:] - INFO  

[jira] [Commented] (ZOOKEEPER-2623) CheckVersion outside of Multi causes NullPointerException

[Flavio Junqueira (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616683#comment-15616683
 ] 

Flavio Junqueira commented on ZOOKEEPER-2623:
-

[~ongardie] hey, this is not a call in the {{ZooKeeper}} class, right? At least 
for the Java client, this is a call in the {{Op}} which is used with {{Multi}}. 
What client is it that you're using? Is it an issue with the C client that 
allows a check call outside multi?

It is not a bad idea to perform a check in the server, though.

> CheckVersion outside of Multi causes NullPointerException
> -
>
> Key: ZOOKEEPER-2623
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2623
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Diego Ongaro
>Priority: Minor
>
> I wasn't sure if check version (opcode 13) was permitted outside of a multi 
> op, so I tried it. My server crashed with a NullPointerException and became 
> unusable until restarted. I guess it's not allowed, but perhaps the server 
> should handle this more gracefully?
> Here are the server logs:
> {noformat}
> Accepted socket connection from /0:0:0:0:0:0:0:1:51737
> Session establishment request from client /0:0:0:0:0:0:0:1:51737 client's 
> lastZxid is 0x0
> Connection request from old client /0:0:0:0:0:0:0:1:51737; will be dropped if 
> server is in r-o mode
> Client attempting to establish new session at /0:0:0:0:0:0:0:1:51737
> :Fsessionid:0x10025651faa type:createSession cxid:0x0 
> zxid:0xfffe txntype:unknown reqpath:n/a
> Processing request:: sessionid:0x10025651faa type:createSession cxid:0x0 
> zxid:0xfffe txntype:unknown reqpath:n/a
> Got zxid 0x6065e expected 0x1
> Creating new log file: log.6065e
> Committing request:: sessionid:0x10025651faa type:createSession cxid:0x0 
> zxid:0x6065e txntype:-10 reqpath:n/a
> Processing request:: sessionid:0x10025651faa type:createSession cxid:0x0 
> zxid:0x6065e txntype:-10 reqpath:n/a
> :Esessionid:0x10025651faa type:createSession cxid:0x0 zxid:0x6065e 
> txntype:-10 reqpath:n/a
> sessionid:0x10025651faa type:createSession cxid:0x0 zxid:0x6065e 
> txntype:-10 reqpath:n/a
> Add a buffer to outgoingBuffers, sk sun.nio.ch.SelectionKeyImpl@28e9f397 is 
> valid: true
> Established session 0x10025651faa with negotiated timeout 2 for 
> client /0:0:0:0:0:0:0:1:51737
> :Fsessionid:0x10025651faa type:check cxid:0x1 zxid:0xfffe 
> txntype:unknown reqpath:/
> Processing request:: sessionid:0x10025651faa type:check cxid:0x1 
> zxid:0xfffe txntype:unknown reqpath:/
> Processing request:: sessionid:0x10025651faa type:check cxid:0x1 
> zxid:0xfffe txntype:unknown reqpath:/
> Exception causing close of session 0x10025651faa: Connection reset by peer
> :Esessionid:0x10025651faa type:check cxid:0x1 zxid:0xfffe 
> txntype:unknown reqpath:/
> IOException stack trace
> java.io.IOException: Connection reset by peer
>   at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>   at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>   at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
>   at sun.nio.ch.IOUtil.read(IOUtil.java:197)
>   at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
>   at 
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:320)
>   at 
> org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:530)
>   at 
> org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:162)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Unexpected exception
> java.lang.NullPointerException
>   at 
> org.apache.zookeeper.server.ZKDatabase.addCommittedProposal(ZKDatabase.java:252)
>   at 
> org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:127)
>   at 
> org.apache.zookeeper.server.quorum.CommitProcessor$CommitWorkRequest.doWork(CommitProcessor.java:362)
>   at 
> org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:162)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Committing request:: sessionid:0x10025651faa type:error cxid:0x1 
> zxid:0x6065f txntype:-1 reqpath:n/a
> Unregister MBean 
> 

ZooKeeper_branch34_openjdk7 - Build # 1258 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch34_openjdk7/1258/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 192769 lines...]
[junit] 2016-10-28 19:52:20,063 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@219] - 
NIOServerCnxn factory exited run method
[junit] 2016-10-28 19:52:20,063 [myid:] - INFO  [main:ZooKeeperServer@497] 
- shutting down
[junit] 2016-10-28 19:52:20,064 [myid:] - ERROR [main:ZooKeeperServer@472] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-10-28 19:52:20,064 [myid:] - INFO  
[main:SessionTrackerImpl@225] - Shutting down
[junit] 2016-10-28 19:52:20,064 [myid:] - INFO  
[main:PrepRequestProcessor@765] - Shutting down
[junit] 2016-10-28 19:52:20,064 [myid:] - INFO  
[main:SyncRequestProcessor@208] - Shutting down
[junit] 2016-10-28 19:52:20,064 [myid:] - INFO  [ProcessThread(sid:0 
cport:11221)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop!
[junit] 2016-10-28 19:52:20,065 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@186] - SyncRequestProcessor exited!
[junit] 2016-10-28 19:52:20,065 [myid:] - INFO  
[main:FinalRequestProcessor@402] - shutdown of request processor complete
[junit] 2016-10-28 19:52:20,066 [myid:] - INFO  
[main:FourLetterWordMain@62] - connecting to 127.0.0.1 11221
[junit] 2016-10-28 19:52:20,066 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2016-10-28 19:52:20,068 [myid:] - INFO  [main:ClientBase@445] - 
STARTING server
[junit] 2016-10-28 19:52:20,068 [myid:] - INFO  [main:ClientBase@366] - 
CREATING server instance 127.0.0.1:11221
[junit] 2016-10-28 19:52:20,069 [myid:] - INFO  
[main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11221
[junit] 2016-10-28 19:52:20,069 [myid:] - INFO  [main:ClientBase@341] - 
STARTING server instance 127.0.0.1:11221
[junit] 2016-10-28 19:52:20,069 [myid:] - INFO  [main:ZooKeeperServer@173] 
- Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 
6 datadir 
/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/build/test/tmp/test2679897701477206706.junit.dir/version-2
 snapdir 
/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_openjdk7/build/test/tmp/test2679897701477206706.junit.dir/version-2
[junit] 2016-10-28 19:52:20,074 [myid:] - ERROR [main:ZooKeeperServer@472] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-10-28 19:52:20,074 [myid:] - INFO  
[main:FourLetterWordMain@62] - connecting to 127.0.0.1 11221
[junit] 2016-10-28 19:52:20,075 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@192] - 
Accepted socket connection from /127.0.0.1:41484
[junit] 2016-10-28 19:52:20,075 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@827] - Processing 
stat command from /127.0.0.1:41484
[junit] 2016-10-28 19:52:20,075 [myid:] - INFO  
[Thread-4:NIOServerCnxn$StatCommand@663] - Stat command output
[junit] 2016-10-28 19:52:20,076 [myid:] - INFO  
[Thread-4:NIOServerCnxn@1008] - Closed socket connection for client 
/127.0.0.1:41484 (no session established for client)
[junit] 2016-10-28 19:52:20,076 [myid:] - INFO  [main:JMXEnv@229] - 
ensureParent:[InMemoryDataTree, StandaloneServer_port]
[junit] 2016-10-28 19:52:20,078 [myid:] - INFO  [main:JMXEnv@246] - 
expect:InMemoryDataTree
[junit] 2016-10-28 19:52:20,078 [myid:] - INFO  [main:JMXEnv@250] - 
found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221,name1=InMemoryDataTree
[junit] 2016-10-28 19:52:20,078 [myid:] - INFO  [main:JMXEnv@246] - 
expect:StandaloneServer_port
[junit] 2016-10-28 19:52:20,078 [myid:] - INFO  [main:JMXEnv@250] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221
[junit] 2016-10-28 19:52:20,079 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 32634
[junit] 2016-10-28 19:52:20,079 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 20
[junit] 2016-10-28 19:52:20,079 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testQuota
[junit] 2016-10-28 19:52:20,079 [myid:] - INFO  [main:ClientBase@522] - 
tearDown starting
[junit] 2016-10-28 19:52:20,131 [myid:] - INFO  [main:ZooKeeper@684] - 
Session: 0x1580cd9a3d3 closed
[junit] 2016-10-28 19:52:20,131 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@519] - EventThread shut down for 
session: 0x1580cd9a3d3
[junit] 2016-10-28 19:52:20,131 [myid:] - INFO  [main:ClientBase@492] - 
STOPPING server
[junit] 2016-10-28 

ZooKeeper-trunk-openjdk7 - Build # 1219 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1219/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 429613 lines...]
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-10-28 21:12:13,664 [myid:127.0.0.1:19301] - INFO  
[main-SendThread(127.0.0.1:19301):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19301. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 21:12:13,664 [myid:127.0.0.1:19301] - WARN  
[main-SendThread(127.0.0.1:19301):ClientCnxn$SendThread@1235] - Session 
0x101a097a20f for server 127.0.0.1/127.0.0.1:19301, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-10-28 21:12:13,704 [myid:127.0.0.1:19446] - INFO  
[main-SendThread(127.0.0.1:19446):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19446. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 21:12:13,704 [myid:127.0.0.1:19446] - INFO  
[main-SendThread(127.0.0.1:19446):ClientCnxn$SendThread@948] - Socket 
connection established, initiating session, client: /127.0.0.1:34990, server: 
127.0.0.1/127.0.0.1:19446
[junit] 2016-10-28 21:12:13,705 [myid:] - WARN  [New I/O worker 
#4043:NettyServerCnxn@399] - Closing connection to /127.0.0.1:34990
[junit] java.io.IOException: ZK down
[junit] at 
org.apache.zookeeper.server.NettyServerCnxn.receiveMessage(NettyServerCnxn.java:336)
[junit] at 
org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.processMessage(NettyServerCnxnFactory.java:244)
[junit] at 
org.apache.zookeeper.server.NettyServerCnxnFactory$CnxnChannelHandler.messageReceived(NettyServerCnxnFactory.java:166)
[junit] at 
org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
[junit] at 
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
[junit] at 
org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
[junit] at 
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
[junit] at 
org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
[junit] at 
org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
[junit] at 
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2016-10-28 21:12:13,705 [myid:127.0.0.1:19446] - INFO  
[main-SendThread(127.0.0.1:19446):ClientCnxn$SendThread@1231] - Unable to read 
additional data from server sessionid 0x0, likely server has closed socket, 
closing socket connection and attempting reconnect
[junit] 2016-10-28 21:12:13,869 [myid:127.0.0.1:19425] - INFO  
[main-SendThread(127.0.0.1:19425):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19425. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 21:12:13,869 [myid:127.0.0.1:19425] - WARN  
[main-SendThread(127.0.0.1:19425):ClientCnxn$SendThread@1235] - Session 
0x201a09ce8eb for server 127.0.0.1/127.0.0.1:19425, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 

Success: ZOOKEEPER- PreCommit Build #5

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/5/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 434555 lines...]
 [exec] ]
 [exec]  : elapsed 1001 : OK
 [exec] Zookeeper_simpleSystem::testLogCallbackClearLog Message Received: 
[2016-10-28 12:58:57,126:13193(0x2b2e47d2f540):ZOO_INFO@log_env@1027: Client 
environment:zookeeper.version=zookeeper C client 3.6.0]
 [exec] Log Message Received: [2016-10-28 
12:58:57,126:13193(0x2b2e47d2f540):ZOO_INFO@log_env@1031: Client 
environment:host.name=asf906.gq1.ygridcore.net]
 [exec] Log Message Received: [2016-10-28 
12:58:57,126:13193(0x2b2e47d2f540):ZOO_INFO@log_env@1038: Client 
environment:os.name=Linux]
 [exec] Log Message Received: [2016-10-28 
12:58:57,126:13193(0x2b2e47d2f540):ZOO_INFO@log_env@1039: Client 
environment:os.arch=3.13.0-36-lowlatency]
 [exec] Log Message Received: [2016-10-28 
12:58:57,126:13193(0x2b2e47d2f540):ZOO_INFO@log_env@1040: Client 
environment:os.version=#63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014]
 [exec] Log Message Received: [2016-10-28 
12:58:57,126:13193(0x2b2e47d2f540):ZOO_INFO@log_env@1048: Client 
environment:user.name=jenkins]
 [exec] Log Message Received: [2016-10-28 
12:58:57,126:13193(0x2b2e47d2f540):ZOO_INFO@log_env@1056: Client 
environment:user.home=/home/jenkins]
 [exec] Log Message Received: [2016-10-28 
12:58:57,126:13193(0x2b2e47d2f540):ZOO_INFO@log_env@1068: Client 
environment:user.dir=/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build/test/test-cppunit]
 [exec] Log Message Received: [2016-10-28 
12:58:57,126:13193(0x2b2e47d2f540):ZOO_INFO@zookeeper_init_internal@: 
Initiating client connection, host=127.0.0.1:22181 sessionTimeout=1 
watcher=0x45d200 sessionId=0 sessionPasswd= context=0x7fff77a39a80 
flags=0]
 [exec] Log Message Received: [2016-10-28 
12:58:57,127:13193(0x2b2e49d8d700):ZOO_INFO@check_events@2360: initiated 
connection to server [127.0.0.1:22181]]
 [exec] Log Message Received: [2016-10-28 
12:58:57,130:13193(0x2b2e49d8d700):ZOO_INFO@check_events@2412: session 
establishment complete on server [127.0.0.1:22181], 
sessionId=0x101e54901bb000f, negotiated timeout=1 ]
 [exec]  : elapsed 1001 : OK
 [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset ZooKeeper server 
started : elapsed 10240 : OK
 [exec] Zookeeper_simpleSystem::testDeserializeString : elapsed 0 : OK
 [exec] Zookeeper_simpleSystem::testFirstServerDown : elapsed 1000 : OK
 [exec] Zookeeper_simpleSystem::testNullData : elapsed 1021 : OK
 [exec] Zookeeper_simpleSystem::testIPV6 : elapsed 1003 : OK
 [exec] Zookeeper_simpleSystem::testCreate : elapsed 1007 : OK
 [exec] Zookeeper_simpleSystem::testPath : elapsed 1013 : OK
 [exec] Zookeeper_simpleSystem::testPathValidation : elapsed 1038 : OK
 [exec] Zookeeper_simpleSystem::testPing : elapsed 17172 : OK
 [exec] Zookeeper_simpleSystem::testAcl : elapsed 1012 : OK
 [exec] Zookeeper_simpleSystem::testChroot : elapsed 3035 : OK
 [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started ZooKeeper 
server started : elapsed 30492 : OK
 [exec] Zookeeper_simpleSystem::testHangingClient : elapsed 1021 : OK
 [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithGlobal ZooKeeper 
server started ZooKeeper server started ZooKeeper server started : elapsed 
14783 : OK
 [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithLocal ZooKeeper 
server started ZooKeeper server started ZooKeeper server started : elapsed 
14794 : OK
 [exec] Zookeeper_simpleSystem::testGetChildren2 : elapsed 1036 : OK
 [exec] Zookeeper_simpleSystem::testLastZxid : elapsed 4512 : OK
 [exec] Zookeeper_simpleSystem::testRemoveWatchers ZooKeeper server started 
: elapsed 4318 : OK
 [exec] Zookeeper_readOnly::testReadOnly : elapsed 4130 : OK
 [exec] OK (72)
 [exec] PASS: zktest-mt
 [exec] ==
 [exec] All 2 tests passed
 [exec] ==
 [exec] make[1]: Leaving directory 
`/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build/test/test-cppunit'

test-core-cppunit:

test-core:

BUILD SUCCESSFUL
Total time: 15 minutes 34 seconds
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[description-setter] Could not determine description.
Putting comment on the pull request
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Success
Sending email for trigger: Success
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting 

[jira] [Assigned] (ZOOKEEPER-2622) ZooTrace.logQuorumPacket does nothing

[Flavio Junqueira (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Flavio Junqueira reassigned ZOOKEEPER-2622:
---

Assignee: Flavio Junqueira

> ZooTrace.logQuorumPacket does nothing
> -
>
> Key: ZOOKEEPER-2622
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2622
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Flavio Junqueira
>Assignee: Flavio Junqueira
>Priority: Trivial
>  Labels: newbie
> Fix For: 3.5.3, 3.6.0
>
>
> The method simply returns and there is some code commented out:
> {code}
> // if (isTraceEnabled(log, mask)) {
> // logTraceMessage(LOG, mask, direction + " "
> // + FollowerHandler.packetToString(qp));
> // }
> {code}
> There are calls to this trace method, so I think we should fix it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper-trunk-solaris - Build # 1364 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-solaris/1364/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 463583 lines...]
[junit] 2016-10-28 08:47:01,500 [myid:] - INFO  [main:ClientBase@386] - 
CREATING server instance 127.0.0.1:11222
[junit] 2016-10-28 08:47:01,500 [myid:] - INFO  
[main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s 
sessionless connection timeout, 2 selector thread(s), 16 worker threads, and 64 
kB direct buffers.
[junit] 2016-10-28 08:47:01,501 [myid:] - INFO  
[main:NIOServerCnxnFactory@686] - binding to port 0.0.0.0/0.0.0.0:11222
[junit] 2016-10-28 08:47:01,502 [myid:] - INFO  [main:ClientBase@361] - 
STARTING server instance 127.0.0.1:11222
[junit] 2016-10-28 08:47:01,503 [myid:] - INFO  [main:ZooKeeperServer@889] 
- minSessionTimeout set to 6000
[junit] 2016-10-28 08:47:01,503 [myid:] - INFO  [main:ZooKeeperServer@898] 
- maxSessionTimeout set to 6
[junit] 2016-10-28 08:47:01,503 [myid:] - INFO  [main:ZooKeeperServer@159] 
- Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 
6 datadir 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/build/test/tmp/test5678960446654410383.junit.dir/version-2
 snapdir 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/build/test/tmp/test5678960446654410383.junit.dir/version-2
[junit] 2016-10-28 08:47:01,504 [myid:] - INFO  [main:FileSnap@83] - 
Reading snapshot 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/build/test/tmp/test5678960446654410383.junit.dir/version-2/snapshot.b
[junit] 2016-10-28 08:47:01,506 [myid:] - INFO  [main:FileTxnSnapLog@306] - 
Snapshotting: 0xb to 
/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/ZooKeeper-trunk-solaris/build/test/tmp/test5678960446654410383.junit.dir/version-2/snapshot.b
[junit] 2016-10-28 08:47:01,507 [myid:] - ERROR [main:ZooKeeperServer@501] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-10-28 08:47:01,508 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 11222
[junit] 2016-10-28 08:47:01,508 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11222:NIOServerCnxnFactory$AcceptThread@296]
 - Accepted socket connection from /127.0.0.1:48864
[junit] 2016-10-28 08:47:01,509 [myid:] - INFO  
[NIOWorkerThread-1:NIOServerCnxn@485] - Processing stat command from 
/127.0.0.1:48864
[junit] 2016-10-28 08:47:01,509 [myid:] - INFO  
[NIOWorkerThread-1:StatCommand@49] - Stat command output
[junit] 2016-10-28 08:47:01,509 [myid:] - INFO  
[NIOWorkerThread-1:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:48864 (no session established for client)
[junit] 2016-10-28 08:47:01,509 [myid:] - INFO  [main:JMXEnv@228] - 
ensureParent:[InMemoryDataTree, StandaloneServer_port]
[junit] 2016-10-28 08:47:01,511 [myid:] - INFO  [main:JMXEnv@245] - 
expect:InMemoryDataTree
[junit] 2016-10-28 08:47:01,511 [myid:] - INFO  [main:JMXEnv@249] - 
found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port11222,name1=InMemoryDataTree
[junit] 2016-10-28 08:47:01,511 [myid:] - INFO  [main:JMXEnv@245] - 
expect:StandaloneServer_port
[junit] 2016-10-28 08:47:01,511 [myid:] - INFO  [main:JMXEnv@249] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11222
[junit] 2016-10-28 08:47:01,511 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 17664
[junit] 2016-10-28 08:47:01,512 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 24
[junit] 2016-10-28 08:47:01,512 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testQuota
[junit] 2016-10-28 08:47:01,512 [myid:] - INFO  [main:ClientBase@543] - 
tearDown starting
[junit] 2016-10-28 08:47:01,543 [myid:] - INFO  [main:ZooKeeper@1315] - 
Session: 0x1248773294b closed
[junit] 2016-10-28 08:47:01,544 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x1248773294b
[junit] 2016-10-28 08:47:01,544 [myid:] - INFO  [main:ClientBase@513] - 
STOPPING server
[junit] 2016-10-28 08:47:01,544 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:0.0.0.0/0.0.0.0:11222:NIOServerCnxnFactory$AcceptThread@219]
 - accept thread exitted run method
[junit] 2016-10-28 08:47:01,544 [myid:] - INFO  
[NIOServerCxnFactory.SelectorThread-1:NIOServerCnxnFactory$SelectorThread@420] 
- selector thread exitted run method
[junit] 2016-10-28 08:47:01,544 [myid:] - INFO  

[jira] [Commented] (ZOOKEEPER-2622) ZooTrace.logQuorumPacket does nothing

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15614872#comment-15614872
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2622:
---

GitHub user fpj opened a pull request:

https://github.com/apache/zookeeper/pull/95

ZOOKEEPER-2622: ZooTrace.logQuorumPacket does nothing

DO NOT MERGE

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/fpj/zookeeper ZK-2622

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/95.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #95


commit 76681a075e045107f149e5ab67756991faf1d8ba
Author: fpj 
Date:   2016-10-28T09:14:51Z

ZOOKEEPER-2622: ZooTrace.logQuorumPacket does nothing




> ZooTrace.logQuorumPacket does nothing
> -
>
> Key: ZOOKEEPER-2622
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2622
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Flavio Junqueira
>Priority: Trivial
>  Labels: newbie
> Fix For: 3.5.3, 3.6.0
>
>
> The method simply returns and there is some code commented out:
> {code}
> // if (isTraceEnabled(log, mask)) {
> // logTraceMessage(LOG, mask, direction + " "
> // + FollowerHandler.packetToString(qp));
> // }
> {code}
> There are calls to this trace method, so I think we should fix it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zookeeper pull request #95: ZOOKEEPER-2622: ZooTrace.logQuorumPacket does no...

[fpj]

GitHub user fpj opened a pull request:

https://github.com/apache/zookeeper/pull/95

ZOOKEEPER-2622: ZooTrace.logQuorumPacket does nothing

DO NOT MERGE

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/fpj/zookeeper ZK-2622

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/95.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #95


commit 76681a075e045107f149e5ab67756991faf1d8ba
Author: fpj 
Date:   2016-10-28T09:14:51Z

ZOOKEEPER-2622: ZooTrace.logQuorumPacket does nothing




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Failed: ZOOKEEPER- PreCommit Build #2

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 54 lines...]
open files  (-n) 6
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority  (-r) 0
stack size  (kbytes, -s) 8192
cpu time   (seconds, -t) unlimited
max user processes  (-u) 10240
virtual memory  (kbytes, -v) unlimited
file locks  (-x) unlimited
Buildfile: 
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml

check-for-findbugs:

findbugs.check:

forrest.check:

hudson-test-patch:
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Testing patch for ZOOKEEPER-.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] 
 [exec] ZOOKEEPER- is not "Patch Available".  Exiting.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Finished build.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] mv: 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 and 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 are the same file

BUILD SUCCESSFUL
Total time: 0 seconds
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
ERROR: No artifacts found that match the file pattern 
"build/test/findbugs/newPatchFindbugsWarnings.html,patchprocess/*.txt,patchprocess/*Warnings.xml,build/test/test-cppunit/*.txt,build/tmp/zk.log".
 Configuration error?
ERROR: ?build/test/findbugs/newPatchFindbugsWarnings.html? doesn?t match 
anything: even ?build? doesn?t exist
Build step 'Archive the artifacts' changed build result to FAILURE
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
ERROR: Step ?Publish JUnit test result report? failed: No test report files 
were found. Configuration error?
[description-setter] Could not determine description.
Putting comment on the pull request
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Commented] (ZOOKEEPER-2615) Zookeeper server holds onto dead/expired session ids in the watch data structures

[Camille Fournier (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15615399#comment-15615399
 ] 

Camille Fournier commented on ZOOKEEPER-2615:
-

Because watches are one-time triggers, you could try triggering the watch by 
removing, adding, or changing the data in the watched nodes. I believe that 
should remove all set watches. But there's no easy fix that I can see.

> Zookeeper server holds onto dead/expired session ids in the watch data 
> structures
> -
>
> Key: ZOOKEEPER-2615
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2615
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.6
>Reporter: guoping.gp
>Assignee: Camille Fournier
> Fix For: 3.4.10, 3.5.3, 3.6.0
>
>
> The same issue (https://issues.apache.org/jira/browse/ZOOKEEPER-1382) still 
> can be found even with zookeeper 3.4.6.
> this issue cause our production zookeeper cluster leak about 1 million 
> watchs, after restart the server one by one, the watch count decrease to only 
> about 4.
> I can reproduce the issue on my mac,here it is:
> 
> pguodeMacBook-Air:bin pguo$ echo srvr | nc localhost 6181
> Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT
> Latency min/avg/max: 0/1156/128513
> Received: 539
> Sent: 531
> Connections: 1
> Outstanding: 0
> Zxid: 0x10037
> Mode: follower
> Node count: 5
> 
> pguodeMacBook-Air:bin pguo$ echo cons | nc localhost 6181
>  
> /127.0.0.1:55759[1](queued=0,recved=5,sent=5,sid=0x157be2732de,lop=PING,est=1476372631116,to=15000,lcxid=0x1,lzxid=0x,lresp=1476372646260,llat=8,minlat=0,avglat=6,maxlat=17)
>  /0:0:0:0:0:0:0:1:55767[0](queued=0,recved=1,sent=0)
> 
> pguodeMacBook-Air:bin pguo$ echo wchp | nc localhost 6181
> /curator_exists_watch
>   0x357be48e4d90007
>   0x357be48e4d90009
>   0x157be2732de
> as above 4-letter's report shows, 0x357be48e4d90007 and 0x357be48e4d90009 
> are leaked after the two sessions expired 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Success: ZOOKEEPER- PreCommit Build #6

[Apache Jenkins Server]

${ghprbPullTitle}
Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/6/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 434212 lines...]
 [exec] Log Message Received: [2016-10-28 
13:52:09,369:22231(0x2ba6058d1540):ZOO_INFO@testLogCallbackInit@993: 
testLogCallbackInit #9]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@zookeeper_close@3261: Closing 
zookeeper sessionId=0x101e579b722000e to [127.0.0.1:22181]
 [exec] ]
 [exec]  : elapsed 1001 : OK
 [exec] Zookeeper_simpleSystem::testLogCallbackClearLog Message Received: 
[2016-10-28 13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@log_env@1027: Client 
environment:zookeeper.version=zookeeper C client 3.6.0]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@log_env@1031: Client 
environment:host.name=asf906.gq1.ygridcore.net]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@log_env@1038: Client 
environment:os.name=Linux]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@log_env@1039: Client 
environment:os.arch=3.13.0-36-lowlatency]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@log_env@1040: Client 
environment:os.version=#63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@log_env@1048: Client 
environment:user.name=jenkins]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@log_env@1056: Client 
environment:user.home=/home/jenkins]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@log_env@1068: Client 
environment:user.dir=/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build/test/test-cppunit]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba6058d1540):ZOO_INFO@zookeeper_init_internal@: 
Initiating client connection, host=127.0.0.1:22181 sessionTimeout=1 
watcher=0x45d200 sessionId=0 sessionPasswd= context=0x7fff9fc25980 
flags=0]
 [exec] Log Message Received: [2016-10-28 
13:52:09,370:22231(0x2ba60792f700):ZOO_INFO@check_events@2360: initiated 
connection to server [127.0.0.1:22181]]
 [exec] Log Message Received: [2016-10-28 
13:52:09,373:22231(0x2ba60792f700):ZOO_INFO@check_events@2412: session 
establishment complete on server [127.0.0.1:22181], 
sessionId=0x101e579b722000f, negotiated timeout=1 ]
 [exec]  : elapsed 1001 : OK
 [exec] Zookeeper_simpleSystem::testAsyncWatcherAutoReset ZooKeeper server 
started : elapsed 10239 : OK
 [exec] Zookeeper_simpleSystem::testDeserializeString : elapsed 0 : OK
 [exec] Zookeeper_simpleSystem::testFirstServerDown : elapsed 1001 : OK
 [exec] Zookeeper_simpleSystem::testNullData : elapsed 1027 : OK
 [exec] Zookeeper_simpleSystem::testIPV6 : elapsed 1003 : OK
 [exec] Zookeeper_simpleSystem::testCreate : elapsed 1007 : OK
 [exec] Zookeeper_simpleSystem::testPath : elapsed 1014 : OK
 [exec] Zookeeper_simpleSystem::testPathValidation : elapsed 1039 : OK
 [exec] Zookeeper_simpleSystem::testPing : elapsed 17173 : OK
 [exec] Zookeeper_simpleSystem::testAcl : elapsed 1012 : OK
 [exec] Zookeeper_simpleSystem::testChroot : elapsed 3038 : OK
 [exec] Zookeeper_simpleSystem::testAuth ZooKeeper server started ZooKeeper 
server started : elapsed 30547 : OK
 [exec] Zookeeper_simpleSystem::testHangingClient : elapsed 1022 : OK
 [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithGlobal ZooKeeper 
server started ZooKeeper server started ZooKeeper server started : elapsed 
14767 : OK
 [exec] Zookeeper_simpleSystem::testWatcherAutoResetWithLocal ZooKeeper 
server started ZooKeeper server started ZooKeeper server started : elapsed 
14982 : OK
 [exec] Zookeeper_simpleSystem::testGetChildren2 : elapsed 1033 : OK
 [exec] Zookeeper_simpleSystem::testLastZxid : elapsed 4513 : OK
 [exec] Zookeeper_simpleSystem::testRemoveWatchers ZooKeeper server started 
: elapsed 4345 : OK
 [exec] Zookeeper_readOnly::testReadOnly : elapsed 4134 : OK
 [exec] OK (72)
 [exec] PASS: zktest-mt
 [exec] ==
 [exec] All 2 tests passed
 [exec] ==
 [exec] make[1]: Leaving directory 
`/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build/test/test-cppunit'

test-core-cppunit:

test-core:

BUILD SUCCESSFUL
Total time: 15 minutes 11 seconds
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[description-setter] Could not determine description.
Setting 

Failed: ZOOKEEPER- PreCommit Build #3

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/3/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 26 lines...]
Done: 9
  Compressing objects
Done: 8815
  Writing objects
Done: 10
  remote: Updating references
Merging refs/tags/changes/3
 > git rev-parse refs/tags/changes/3^{commit} # timeout=10
 > git merge 0822c07964ea4122d6590125bdbd761ed92e1cc6 # timeout=10
 > git rev-parse master^{commit} # timeout=10
Checking out Revision 0822c07964ea4122d6590125bdbd761ed92e1cc6 (master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 0822c07964ea4122d6590125bdbd761ed92e1cc6
 > git rev-parse origin/master^{commit} # timeout=10
 > git rev-list f6349d16fcd5f04b560095417fd2a1813ac3e855 # timeout=10
No emails were triggered.
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[PreCommit-ZOOKEEPER-github-pr-build] $ /bin/bash 
/tmp/hudson6155886247766939918.sh
/home/jenkins/tools/java/latest1.7/bin/java
java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
core file size  (blocks, -c) 0
data seg size   (kbytes, -d) unlimited
scheduling priority (-e) 0
file size   (blocks, -f) unlimited
pending signals (-i) 386178
max locked memory   (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files  (-n) 6
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority  (-r) 0
stack size  (kbytes, -s) 8192
cpu time   (seconds, -t) unlimited
max user processes  (-u) 10240
virtual memory  (kbytes, -v) unlimited
file locks  (-x) unlimited
Buildfile: 
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml

BUILD FAILED
Target " " does not exist in the project "ZooKeeper". 

Total time: 0 seconds
/tmp/hudson6155886247766939918.sh: line 29: -Dtest.output=yes: command not found
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
ERROR: Step ?Publish JUnit test result report? failed: No test report files 
were found. Configuration error?
[description-setter] Could not determine description.
Putting comment on the pull request
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Commented] (ZOOKEEPER-2619) Client library reconnecting breaks FIFO client order

[Diego Ongaro (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616890#comment-15616890
 ] 

Diego Ongaro commented on ZOOKEEPER-2619:
-

bq. Some of the issues you're describing, we only had the opportunity to 
discuss in more detail in the book. It is really not self-promotion, and it is 
where we had a chance to talk more about some of the subtleties we have. 
--[~fpj]

Indeed, page 116-117 of the ZooKeeper: Distributed Process Coordination book 
mentions this issue. It suggests:

bq. If Op2 depends on Op1, ... wait for a successful of Op1 before submitting 
Op2...this approach is safe, but it adds a performance penalty...

> Client library reconnecting breaks FIFO client order
> 
>
> Key: ZOOKEEPER-2619
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2619
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Diego Ongaro
>
> According to the USENIX ATC 2010 
> [paper|https://www.usenix.org/conference/usenix-atc-10/zookeeper-wait-free-coordination-internet-scale-systems],
>  ZooKeeper provides "FIFO client order: all requests from a given client are 
> executed in the order that they were sent by the client." I believe 
> applications written using the Java client library are unable to rely on this 
> guarantee, and any current application that does so is broken. Other client 
> libraries are also likely to be affected.
> Consider this application, which is simplified from the algorithm described 
> on Page 4 (right column) of the paper:
> {code}
>   zk = new ZooKeeper(...)
>   zk.createAsync("/data-23857", "...", callback)
>   zk.createSync("/pointer", "/data-23857")
> {code}
> Assume an empty ZooKeeper database to begin with and no other writers. 
> Applying the above definition, if the ZooKeeper database contains /pointer, 
> it must also contain /data-23857.
> Now consider this series of unfortunate events:
> {code}
>   zk = new ZooKeeper(...)
>   // The library establishes a TCP connection.
>   zk.createAsync("/data-23857", "...", callback)
>   // The library/kernel closes the TCP connection because it times out, and
>   // the create of /data-23857 is doomed to fail with ConnectionLoss. Suppose
>   // that it never reaches the server.
>   // The library establishes a new TCP connection.
>   zk.createSync("/pointer", "/data-23857")
>   // The create of /pointer succeeds.
> {code}
> That's the problem: subsequent operations get assigned to the new connection 
> and succeed, while earlier operations fail.
> In general, I believe it's impossible to have a system with the following 
> three properties:
>  # FIFO client order for asynchronous operations,
>  # Failing operations when connections are lost, AND
>  # Transparently reconnecting when connections are lost.
> To argue this, consider an application that issues a series of pipelined 
> operations, then upon noticing a connection loss, issues a series of recovery 
> operations, repeating the recovery procedure as necessary. If a pipelined 
> operation fails, all subsequent operations in the pipeline must also fail. 
> Yet the client must also carry on eventually: the recovery operations cannot 
> be trivially failed forever. Unfortunately, the client library does not know 
> where the pipelined operations end and the recovery operations begin. At the 
> time of a connection loss, subsequent pipelined operations may or may not be 
> queued in the library; others might be upcoming in the application thread. If 
> the library re-establishes a connection too early, it will send pipelined 
> operations out of FIFO client order.
> I considered a possible workaround of having the client diligently check its 
> callbacks and watchers for connection loss events, and do its best to stop 
> the subsequent pipelined operations at the first sign of a connection loss. 
> In addition to being a large burden for the application, this does not solve 
> the problem all the time. In particular, if the callback thread is delayed 
> significantly (as can happen due to excessive computation or scheduling 
> hiccups), the application may not learn about the connection loss event until 
> after the connection has been re-established and after dependent pipelined 
> operations have already been transmitted over the new connection.
> I suggest the following API changes to fix the problem:
>  - Add a method ZooKeeper.getConnection() returning a ZKConnection object. 
> ZKConnection would wrap a TCP connection. It would include all synchronous 
> and asynchronous operations currently defined on the ZooKeeper class. Upon a 
> connection loss on a ZKConnection, all subsequent operations on the same 
> ZKConnection would return a Connection Loss error. Upon noticing, the client 
> would need to call ZooKeeper.getConnection() again to get a working 
> 

[jira] [Comment Edited] (ZOOKEEPER-2619) Client library reconnecting breaks FIFO client order

[Diego Ongaro (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616890#comment-15616890
 ] 

Diego Ongaro edited comment on ZOOKEEPER-2619 at 10/28/16 11:11 PM:


bq. Some of the issues you're describing, we only had the opportunity to 
discuss in more detail in the book. It is really not self-promotion, and it is 
where we had a chance to talk more about some of the subtleties we have. 
--[~fpj]

Indeed, page 116-117 of the ZooKeeper: Distributed Process Coordination book 
mentions this issue. It suggests:

bq. If Op2 depends on Op1, ... wait for a successful execution of Op1 before 
submitting Op2...this approach is safe, but it adds a performance penalty...


was (Author: ongardie):
bq. Some of the issues you're describing, we only had the opportunity to 
discuss in more detail in the book. It is really not self-promotion, and it is 
where we had a chance to talk more about some of the subtleties we have. 
--[~fpj]

Indeed, page 116-117 of the ZooKeeper: Distributed Process Coordination book 
mentions this issue. It suggests:

bq. If Op2 depends on Op1, ... wait for a successful of Op1 before submitting 
Op2...this approach is safe, but it adds a performance penalty...

> Client library reconnecting breaks FIFO client order
> 
>
> Key: ZOOKEEPER-2619
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2619
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Diego Ongaro
>
> According to the USENIX ATC 2010 
> [paper|https://www.usenix.org/conference/usenix-atc-10/zookeeper-wait-free-coordination-internet-scale-systems],
>  ZooKeeper provides "FIFO client order: all requests from a given client are 
> executed in the order that they were sent by the client." I believe 
> applications written using the Java client library are unable to rely on this 
> guarantee, and any current application that does so is broken. Other client 
> libraries are also likely to be affected.
> Consider this application, which is simplified from the algorithm described 
> on Page 4 (right column) of the paper:
> {code}
>   zk = new ZooKeeper(...)
>   zk.createAsync("/data-23857", "...", callback)
>   zk.createSync("/pointer", "/data-23857")
> {code}
> Assume an empty ZooKeeper database to begin with and no other writers. 
> Applying the above definition, if the ZooKeeper database contains /pointer, 
> it must also contain /data-23857.
> Now consider this series of unfortunate events:
> {code}
>   zk = new ZooKeeper(...)
>   // The library establishes a TCP connection.
>   zk.createAsync("/data-23857", "...", callback)
>   // The library/kernel closes the TCP connection because it times out, and
>   // the create of /data-23857 is doomed to fail with ConnectionLoss. Suppose
>   // that it never reaches the server.
>   // The library establishes a new TCP connection.
>   zk.createSync("/pointer", "/data-23857")
>   // The create of /pointer succeeds.
> {code}
> That's the problem: subsequent operations get assigned to the new connection 
> and succeed, while earlier operations fail.
> In general, I believe it's impossible to have a system with the following 
> three properties:
>  # FIFO client order for asynchronous operations,
>  # Failing operations when connections are lost, AND
>  # Transparently reconnecting when connections are lost.
> To argue this, consider an application that issues a series of pipelined 
> operations, then upon noticing a connection loss, issues a series of recovery 
> operations, repeating the recovery procedure as necessary. If a pipelined 
> operation fails, all subsequent operations in the pipeline must also fail. 
> Yet the client must also carry on eventually: the recovery operations cannot 
> be trivially failed forever. Unfortunately, the client library does not know 
> where the pipelined operations end and the recovery operations begin. At the 
> time of a connection loss, subsequent pipelined operations may or may not be 
> queued in the library; others might be upcoming in the application thread. If 
> the library re-establishes a connection too early, it will send pipelined 
> operations out of FIFO client order.
> I considered a possible workaround of having the client diligently check its 
> callbacks and watchers for connection loss events, and do its best to stop 
> the subsequent pipelined operations at the first sign of a connection loss. 
> In addition to being a large burden for the application, this does not solve 
> the problem all the time. In particular, if the callback thread is delayed 
> significantly (as can happen due to excessive computation or scheduling 
> hiccups), the application may not learn about the connection loss event until 
> after the connection has been re-established and after dependent pipelined 
> 

[jira] [Commented] (ZOOKEEPER-1621) ZooKeeper does not recover from crash when disk was full

[Michael Han (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616950#comment-15616950
 ] 

Michael Han commented on ZOOKEEPER-1621:


The proposal of the fix makes sense to me.

Is it feasible to make a stronger guarantee for the ZooKeeper serialization 
semantics - that is, under no cases (disk full, power failure, hardware 
failure) would ZooKeeper generates invalid persistent files (for both snapshot 
and tx logs)? This might be possible by serializing things to a swap file first 
and then at one point do an atomic rename of the file. With the guarantee of 
the sanity of the on disk formats the deserializing logic would be simplified, 
as there will not be many corner cases to consider, besides the existing basic 
checksum check logic. 

I can think two potential drawback of this approach:
* Performance: if we write to swap file and then rename for every writes, we 
will be making more sys calls per write. Might impact performance / latency of 
write?
* Potential data loss during recover: to improve performance, we could batch 
writes and only do rename at certain points - (i.e. every 1000 writes). In case 
of a failure, part of the data might loss as those data (possibly corrupted / 
partially serialized) living in swap file will not be parsed by ZK during start 
up (we will only load and parse renamed files.).

My feeling is the best approach might be a mix of efforts on both serialization 
and deserialization side:
* When serializing, we do our best efforts to avoid generate corrupted files 
(i.e. through atomic writes to files.).
* When deserializing, we do best efforts to detect corrupt files and recover 
conservatively - the success of recovery might be case by case - for example 
for this disk full case the proposed fix sounds pretty safe to perform while in 
other cases it might not be straightforward to tell which data is good and 
which is bad.
* As a result - the expectation is when things crash and files corrupted, ZK 
should be able to recover later without manual intervention. This would be good 
for users.


> ZooKeeper does not recover from crash when disk was full
> 
>
> Key: ZOOKEEPER-1621
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1621
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.3
> Environment: Ubuntu 12.04, Amazon EC2 instance
>Reporter: David Arthur
>Assignee: Michi Mutsuzaki
> Fix For: 3.5.3, 3.6.0
>
> Attachments: ZOOKEEPER-1621.patch, zookeeper.log.gz
>
>
> The disk that ZooKeeper was using filled up. During a snapshot write, I got 
> the following exception
> 2013-01-16 03:11:14,098 - ERROR [SyncThread:0:SyncRequestProcessor@151] - 
> Severe unrecoverable error, exiting
> java.io.IOException: No space left on device
> at java.io.FileOutputStream.writeBytes(Native Method)
> at java.io.FileOutputStream.write(FileOutputStream.java:282)
> at 
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog.commit(FileTxnLog.java:309)
> at 
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.commit(FileTxnSnapLog.java:306)
> at org.apache.zookeeper.server.ZKDatabase.commit(ZKDatabase.java:484)
> at 
> org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:162)
> at 
> org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:101)
> Then many subsequent exceptions like:
> 2013-01-16 15:02:23,984 - ERROR [main:Util@239] - Last transaction was 
> partial.
> 2013-01-16 15:02:23,985 - ERROR [main:ZooKeeperServerMain@63] - Unexpected 
> exception, exiting abnormally
> java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at 
> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
> at 
> org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:558)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:577)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:543)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:625)
> at 
> org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.init(FileTxnLog.java:529)
> at 
> 

[jira] [Commented] (ZOOKEEPER-2346) SASL Auth failure manifested to client as connection refusal

[Michael Han (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616964#comment-15616964
 ] 

Michael Han commented on ZOOKEEPER-2346:


[~fpj] should be able to assign the issue to you [~mkizner].
Alternatively, you can send a pull request, please refer to [this 
link|https://cwiki.apache.org/confluence/display/ZOOKEEPER/Merging+Github+Pull+Requests]
 on how to.

> SASL Auth failure manifested to client as connection refusal
> 
>
> Key: ZOOKEEPER-2346
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2346
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.6
>Reporter: Steve Loughran
>
> If a client can't authenticate via sasl then (a) the stack trace is lost on 
> the server logs, and (b) it is exposed to the client as a connection refusal. 
> This results in curator retrying many times before giving up —and with the 
> cause being misinterpreted as a server-down problem, rather than a 
> client-not-trusted problem



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper-trunk - Build # 3136 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk/3136/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 407733 lines...]
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
[junit] at 
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2016-10-29 00:06:30,821 [myid:] - INFO  [New I/O worker 
#14392:ClientCnxnSocketNetty$ZKClientHandler@384] - channel is disconnected: 
[id: 0xfff95a3d, /127.0.0.1:36030 :> 127.0.0.1/127.0.0.1:30317]
[junit] 2016-10-29 00:06:30,821 [myid:] - INFO  [New I/O worker 
#14392:ClientCnxnSocketNetty@208] - channel is told closing
[junit] 2016-10-29 00:06:30,821 [myid:] - INFO  
[SyncThread:0:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port30317,name1=Connections,name2=127.0.0.1,name3=0x101e7acd395]
[junit] 2016-10-29 00:06:30,822 [myid:] - INFO  
[main:ClientCnxnSocketNetty@208] - channel is told closing
[junit] 2016-10-29 00:06:30,822 [myid:] - INFO  [main:ZooKeeper@1315] - 
Session: 0x101e7acd395 closed
[junit] 2016-10-29 00:06:30,822 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 172439
[junit] 2016-10-29 00:06:30,822 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x101e7acd395
[junit] 2016-10-29 00:06:30,822 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 1814
[junit] 2016-10-29 00:06:30,823 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testWatcherAutoResetWithLocal
[junit] 2016-10-29 00:06:30,823 [myid:] - INFO  [main:ClientBase@543] - 
tearDown starting
[junit] 2016-10-29 00:06:30,824 [myid:] - INFO  [main:ClientBase@513] - 
STOPPING server
[junit] 2016-10-29 00:06:30,824 [myid:] - INFO  
[main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:30317
[junit] 2016-10-29 00:06:30,828 [myid:] - INFO  [main:ZooKeeperServer@529] 
- shutting down
[junit] 2016-10-29 00:06:30,828 [myid:] - ERROR [main:ZooKeeperServer@501] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-10-29 00:06:30,829 [myid:] - INFO  
[main:SessionTrackerImpl@232] - Shutting down
[junit] 2016-10-29 00:06:30,829 [myid:] - INFO  
[main:PrepRequestProcessor@996] - Shutting down
[junit] 2016-10-29 00:06:30,829 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2016-10-29 00:06:30,829 [myid:] - INFO  [ProcessThread(sid:0 
cport:30317)::PrepRequestProcessor@156] - PrepRequestProcessor exited loop!
[junit] 2016-10-29 00:06:30,830 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2016-10-29 00:06:30,830 [myid:] - INFO  
[main:FinalRequestProcessor@481] - shutdown of request processor complete
[junit] 2016-10-29 00:06:30,830 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port30317,name1=InMemoryDataTree]
[junit] 2016-10-29 00:06:30,831 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port30317]
[junit] 2016-10-29 00:06:30,831 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 30317
[junit] 2016-10-29 00:06:30,831 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2016-10-29 00:06:30,838 [myid:] - INFO  [main:ClientBase@568] - 
fdcount after test is: 5213 at start it was 5213
[junit] 2016-10-29 00:06:30,838 [myid:] - INFO  [main:ZKTestCase$1@65] - 
SUCCEEDED testWatcherAutoResetWithLocal
[junit] 2016-10-29 00:06:30,838 [myid:] - INFO  [main:ZKTestCase$1@60] - 
FINISHED testWatcherAutoResetWithLocal
[junit] Tests run: 101, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
459.961 sec, Thread: 8, Class: org.apache.zookeeper.test.NettyNettySuiteTest

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/ZooKeeper-trunk/build.xml:1322: The 
following error occurred while 

ZooKeeper_branch34 - Build # 1703 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch34/1703/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 175933 lines...]
[junit] 2016-10-29 00:22:36,675 [myid:] - INFO  [main:ZooKeeperServer@497] 
- shutting down
[junit] 2016-10-29 00:22:36,676 [myid:] - ERROR [main:ZooKeeperServer@472] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-10-29 00:22:36,676 [myid:] - INFO  
[main:SessionTrackerImpl@225] - Shutting down
[junit] 2016-10-29 00:22:36,676 [myid:] - INFO  
[main:PrepRequestProcessor@765] - Shutting down
[junit] 2016-10-29 00:22:36,676 [myid:] - INFO  
[main:SyncRequestProcessor@208] - Shutting down
[junit] 2016-10-29 00:22:36,676 [myid:] - INFO  [ProcessThread(sid:0 
cport:11221)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop!
[junit] 2016-10-29 00:22:36,677 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@186] - SyncRequestProcessor exited!
[junit] 2016-10-29 00:22:36,677 [myid:] - INFO  
[main:FinalRequestProcessor@402] - shutdown of request processor complete
[junit] 2016-10-29 00:22:36,678 [myid:] - INFO  
[main:FourLetterWordMain@62] - connecting to 127.0.0.1 11221
[junit] 2016-10-29 00:22:36,678 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2016-10-29 00:22:36,680 [myid:] - INFO  [main:ClientBase@445] - 
STARTING server
[junit] 2016-10-29 00:22:36,680 [myid:] - INFO  [main:ClientBase@366] - 
CREATING server instance 127.0.0.1:11221
[junit] 2016-10-29 00:22:36,681 [myid:] - INFO  
[main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11221
[junit] 2016-10-29 00:22:36,681 [myid:] - INFO  [main:ClientBase@341] - 
STARTING server instance 127.0.0.1:11221
[junit] 2016-10-29 00:22:36,681 [myid:] - INFO  [main:ZooKeeperServer@173] 
- Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 
6 datadir 
/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34/build/test/tmp/test8494846260718638424.junit.dir/version-2
 snapdir 
/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34/build/test/tmp/test8494846260718638424.junit.dir/version-2
[junit] 2016-10-29 00:22:36,687 [myid:] - ERROR [main:ZooKeeperServer@472] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-10-29 00:22:36,687 [myid:] - INFO  
[main:FourLetterWordMain@62] - connecting to 127.0.0.1 11221
[junit] 2016-10-29 00:22:36,688 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@192] - 
Accepted socket connection from /127.0.0.1:58682
[junit] 2016-10-29 00:22:36,688 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@827] - Processing 
stat command from /127.0.0.1:58682
[junit] 2016-10-29 00:22:36,688 [myid:] - INFO  
[Thread-5:NIOServerCnxn$StatCommand@663] - Stat command output
[junit] 2016-10-29 00:22:36,689 [myid:] - INFO  
[Thread-5:NIOServerCnxn@1008] - Closed socket connection for client 
/127.0.0.1:58682 (no session established for client)
[junit] 2016-10-29 00:22:36,689 [myid:] - INFO  [main:JMXEnv@229] - 
ensureParent:[InMemoryDataTree, StandaloneServer_port]
[junit] 2016-10-29 00:22:36,691 [myid:] - INFO  [main:JMXEnv@246] - 
expect:InMemoryDataTree
[junit] 2016-10-29 00:22:36,692 [myid:] - INFO  [main:JMXEnv@250] - 
found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221,name1=InMemoryDataTree
[junit] 2016-10-29 00:22:36,692 [myid:] - INFO  [main:JMXEnv@246] - 
expect:StandaloneServer_port
[junit] 2016-10-29 00:22:36,692 [myid:] - INFO  [main:JMXEnv@250] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221
[junit] 2016-10-29 00:22:36,693 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 31154
[junit] 2016-10-29 00:22:36,693 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 20
[junit] 2016-10-29 00:22:36,693 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testQuota
[junit] 2016-10-29 00:22:36,693 [myid:] - INFO  [main:ClientBase@522] - 
tearDown starting
[junit] 2016-10-29 00:22:36,757 [myid:] - INFO  [main:ZooKeeper@684] - 
Session: 0x1580dd116d1 closed
[junit] 2016-10-29 00:22:36,757 [myid:] - INFO  [main:ClientBase@492] - 
STOPPING server
[junit] 2016-10-29 00:22:36,757 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@519] - EventThread shut down for 
session: 0x1580dd116d1
[junit] 2016-10-29 00:22:36,757 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@219] - 
NIOServerCnxn factory exited run method
[junit] 2016-10-29 00:22:36,757 [myid:] - 

[jira] [Commented] (ZOOKEEPER-2623) CheckVersion outside of Multi causes NullPointerException

[Diego Ongaro (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616879#comment-15616879
 ] 

Diego Ongaro commented on ZOOKEEPER-2623:
-

[~fpj] ah sorry, I should have mentioned that I did this from a Go client 
library that I'm writing.

> CheckVersion outside of Multi causes NullPointerException
> -
>
> Key: ZOOKEEPER-2623
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2623
> Project: ZooKeeper
>  Issue Type: Bug
>Reporter: Diego Ongaro
>Priority: Minor
>
> I wasn't sure if check version (opcode 13) was permitted outside of a multi 
> op, so I tried it. My server crashed with a NullPointerException and became 
> unusable until restarted. I guess it's not allowed, but perhaps the server 
> should handle this more gracefully?
> Here are the server logs:
> {noformat}
> Accepted socket connection from /0:0:0:0:0:0:0:1:51737
> Session establishment request from client /0:0:0:0:0:0:0:1:51737 client's 
> lastZxid is 0x0
> Connection request from old client /0:0:0:0:0:0:0:1:51737; will be dropped if 
> server is in r-o mode
> Client attempting to establish new session at /0:0:0:0:0:0:0:1:51737
> :Fsessionid:0x10025651faa type:createSession cxid:0x0 
> zxid:0xfffe txntype:unknown reqpath:n/a
> Processing request:: sessionid:0x10025651faa type:createSession cxid:0x0 
> zxid:0xfffe txntype:unknown reqpath:n/a
> Got zxid 0x6065e expected 0x1
> Creating new log file: log.6065e
> Committing request:: sessionid:0x10025651faa type:createSession cxid:0x0 
> zxid:0x6065e txntype:-10 reqpath:n/a
> Processing request:: sessionid:0x10025651faa type:createSession cxid:0x0 
> zxid:0x6065e txntype:-10 reqpath:n/a
> :Esessionid:0x10025651faa type:createSession cxid:0x0 zxid:0x6065e 
> txntype:-10 reqpath:n/a
> sessionid:0x10025651faa type:createSession cxid:0x0 zxid:0x6065e 
> txntype:-10 reqpath:n/a
> Add a buffer to outgoingBuffers, sk sun.nio.ch.SelectionKeyImpl@28e9f397 is 
> valid: true
> Established session 0x10025651faa with negotiated timeout 2 for 
> client /0:0:0:0:0:0:0:1:51737
> :Fsessionid:0x10025651faa type:check cxid:0x1 zxid:0xfffe 
> txntype:unknown reqpath:/
> Processing request:: sessionid:0x10025651faa type:check cxid:0x1 
> zxid:0xfffe txntype:unknown reqpath:/
> Processing request:: sessionid:0x10025651faa type:check cxid:0x1 
> zxid:0xfffe txntype:unknown reqpath:/
> Exception causing close of session 0x10025651faa: Connection reset by peer
> :Esessionid:0x10025651faa type:check cxid:0x1 zxid:0xfffe 
> txntype:unknown reqpath:/
> IOException stack trace
> java.io.IOException: Connection reset by peer
>   at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>   at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>   at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
>   at sun.nio.ch.IOUtil.read(IOUtil.java:197)
>   at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
>   at 
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:320)
>   at 
> org.apache.zookeeper.server.NIOServerCnxnFactory$IOWorkRequest.doWork(NIOServerCnxnFactory.java:530)
>   at 
> org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:162)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Unexpected exception
> java.lang.NullPointerException
>   at 
> org.apache.zookeeper.server.ZKDatabase.addCommittedProposal(ZKDatabase.java:252)
>   at 
> org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:127)
>   at 
> org.apache.zookeeper.server.quorum.CommitProcessor$CommitWorkRequest.doWork(CommitProcessor.java:362)
>   at 
> org.apache.zookeeper.server.WorkerService$ScheduledWorkRequest.run(WorkerService.java:162)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Committing request:: sessionid:0x10025651faa type:error cxid:0x1 
> zxid:0x6065f txntype:-1 reqpath:n/a
> Unregister MBean 
> [org.apache.ZooKeeperService:name0=ReplicatedServer_id1,name1=replica.1,name2=Follower,name3=Connections,name4="0:0:0:0:0:0:0:1",name5=0x10025651faa]
> Exception thrown by downstream processor, unable to continue.
> CommitProcessor exited loop!
> Closed socket connection for client /0:0:0:0:0:0:0:1:51737 which had 
> sessionid 

[jira] [Comment Edited] (ZOOKEEPER-2346) SASL Auth failure manifested to client as connection refusal

[Michael Han (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616964#comment-15616964
 ] 

Michael Han edited comment on ZOOKEEPER-2346 at 10/28/16 11:53 PM:
---

[~fpj] should be able to assign the issue to you [~mkizner].
Alternatively, you can send a pull request to 
[apache/zookeeper|https://github.com/apache/zookeeper].


was (Author: hanm):
[~fpj] should be able to assign the issue to you [~mkizner].
Alternatively, you can send a pull request, please refer to [this 
link|https://cwiki.apache.org/confluence/display/ZOOKEEPER/Merging+Github+Pull+Requests]
 on how to.

> SASL Auth failure manifested to client as connection refusal
> 
>
> Key: ZOOKEEPER-2346
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2346
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.6
>Reporter: Steve Loughran
>
> If a client can't authenticate via sasl then (a) the stack trace is lost on 
> the server logs, and (b) it is exposed to the client as a connection refusal. 
> This results in curator retrying many times before giving up —and with the 
> cause being misinterpreted as a server-down problem, rather than a 
> client-not-trusted problem



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

ZooKeeper_branch35_openjdk7 - Build # 279 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch35_openjdk7/279/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 460051 lines...]
[junit] 2016-10-28 14:40:15,256 [myid:127.0.0.1:19355] - INFO  
[main-SendThread(127.0.0.1:19355):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19355. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 14:40:15,257 [myid:127.0.0.1:19355] - WARN  
[main-SendThread(127.0.0.1:19355):ClientCnxn$SendThread@1235] - Session 
0x30310fce9b5 for server 127.0.0.1/127.0.0.1:19355, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-10-28 14:40:15,939 [myid:127.0.0.1:19352] - INFO  
[main-SendThread(127.0.0.1:19352):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19352. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 14:40:15,940 [myid:127.0.0.1:19352] - WARN  
[main-SendThread(127.0.0.1:19352):ClientCnxn$SendThread@1235] - Session 
0x20310fcec64 for server 127.0.0.1/127.0.0.1:19352, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-10-28 14:40:16,143 [myid:127.0.0.1:19379] - INFO  
[main-SendThread(127.0.0.1:19379):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19379. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 14:40:16,143 [myid:127.0.0.1:19379] - INFO  
[main-SendThread(127.0.0.1:19379):ClientCnxn$SendThread@948] - Socket 
connection established, initiating session, client: /127.0.0.1:38204, server: 
127.0.0.1/127.0.0.1:19379
[junit] 2016-10-28 14:40:16,143 [myid:] - INFO  
[NIOServerCxnFactory.AcceptThread:/127.0.0.1:19379:NIOServerCnxnFactory$AcceptThread@296]
 - Accepted socket connection from /127.0.0.1:38204
[junit] 2016-10-28 14:40:16,144 [myid:] - WARN  
[NIOWorkerThread-22:NIOServerCnxn@369] - Exception causing close of session 
0x0: ZooKeeperServer not running
[junit] 2016-10-28 14:40:16,144 [myid:] - INFO  
[NIOWorkerThread-22:NIOServerCnxn@607] - Closed socket connection for client 
/127.0.0.1:38204 (no session established for client)
[junit] 2016-10-28 14:40:16,144 [myid:127.0.0.1:19379] - INFO  
[main-SendThread(127.0.0.1:19379):ClientCnxn$SendThread@1231] - Unable to read 
additional data from server sessionid 0x0, likely server has closed socket, 
closing socket connection and attempting reconnect
[junit] 2016-10-28 14:40:16,395 [myid:127.0.0.1:19355] - INFO  
[main-SendThread(127.0.0.1:19355):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19355. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 14:40:16,395 [myid:127.0.0.1:19355] - WARN  
[main-SendThread(127.0.0.1:19355):ClientCnxn$SendThread@1235] - Session 
0x30310fce9b5 for server 127.0.0.1/127.0.0.1:19355, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2016-10-28 14:40:16,662 [myid:127.0.0.1:19349] - INFO  
[main-SendThread(127.0.0.1:19349):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19349. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 14:40:16,663 [myid:127.0.0.1:19349] - WARN  
[main-SendThread(127.0.0.1:19349):ClientCnxn$SendThread@1235] - Session 
0x10310fce8d3 for server 127.0.0.1/127.0.0.1:19349, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused

Re: QA github pre-commit queue

[Flavio Junqueira]

Here is my progress so far. I managed to get the build to trigger with a pull 
request and pull request changes:

https://builds.apache.org/view/PreCommit%20Builds/job/PreCommit-ZOOKEEPER-github-pr-build/

I had to hack into it a bit because our original pre-commit build is really 
focused on getting patch files from jira, applying the patch, and building it 
(check src/java/test/bin/test-patch.sh). I hardcoded the build commands into 
the queue configuration, which works but makes it hard for others without 
access to jenkins to contribute, so moving forward we should script it.

There is clearly some polishing to be done, so please report back so that we 
can try to fix it.

-Flavio

> On 27 Oct 2016, at 22:24, Flavio Junqueira  wrote:
> 
> Yeah, I tried to trigger the 761 manually and it didn't work. I need to work 
> on the manual trigger.
> 
> The issue with your PR 94 is likely to be a bug in the config that I think I 
> fixed now. I need to create a test PR to debug it.
> 
> One problem is that builds.apache.org  is super 
> slow, so it is difficult to work on the configuration right now. I'll work 
> some more on my morning because it shouldn't be as busy.
> 
> -Flavio
> 
>> On 27 Oct 2016, at 22:19, Michael Han  wrote:
>> 
>> I saw the pre-commit build from the new bot (
>> https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1/) for
>> ZOOKEEPER-761. There are two issues:
>> 
>> * The test does not run, because "[exec] ZOOKEEPER-761 is not "Patch
>> Available".  Exiting."
>> * The build result was sent to dev list, but the result was not posted on
>> JIRA, as previous build bot did.
>> 
>> Still no sign of pre-commit build triggered by my PR94, btw.
>> 
>> On Thu, Oct 27, 2016 at 12:07 PM, Benjamin Reed  wrote:
>> 
>>> i also pushed a new version for
>>> https://issues.apache.org/jira/browse/ZOOKEEPER-761 although that one
>>> might
>>> be tricky since there are attached patches and a pr. should the pr still be
>>> qaed?
>>> 
>>> 
>>> On Thu, Oct 27, 2016 at 12:04 PM, Michael Han  wrote:
>>> 
 Created PR94 to ZOOKEEPER-2014. It's been 2 hours, and no QA bot
>>> activity.
 
 On Thu, Oct 27, 2016 at 9:05 AM, Flavio Junqueira 
>>> wrote:
 
> Ok, I have created this queue: PreCommit-ZOOKEEPER-github-pr-build. I
> have configured it and would kindly appreciate if anyone could update a
 PR
> to test it.
> 
> -Flavio
> 
> 
>> On 27 Oct 2016, at 16:57, Edward Ribeiro 
> wrote:
>> 
>> Cool! Thanks for the heads up. :)
>> 
>> Cheers
>> 
>> Em 27 de out de 2016 1:56 PM, "Flavio Junqueira" 
> escreveu:
>> 
>>> There is no need to create an INFRA jira, I'm taking care of it,
>>> stay
>>> tuned. In the meanwhile, please submit patches as usual through jira
 to
>>> trigger QA.
>>> 
>>> -Flavio
>>> 
 On 27 Oct 2016, at 16:54, Edward Ribeiro >> wrote:
 
 Dear community,
 
 As part of the github move, we are still lacking the plumbing that
> allows
 to run Jenkins CI tests, etc, on open Pull Requests. Please, take a
> look
>>> at
 Kafka pending PR at Github to see what I am referring to.
 
 Any committer could open an INFRA JIRA to address this?
 
 Best regards,
 Eddie
>>> 
>>> 
> 
> 
 
 
 --
 Cheers
 Michael.
 
>>> 
>> 
>> 
>> 
>> -- 
>> Cheers
>> Michael.
> 

[jira] [Commented] (ZOOKEEPER-1045) Support Quorum Peer mutual authentication via SASL

[Eugene Koontz (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-1045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15617246#comment-15617246
 ] 

Eugene Koontz commented on ZOOKEEPER-1045:
--

I left a few minor comments on the reviewboard here:

https://reviews.apache.org/r/47354/#comment223707

I will try to get up to speed with the discussion thus far in the JIRA and 
hopefully have something else to add.

> Support Quorum Peer mutual authentication via SASL
> --
>
> Key: ZOOKEEPER-1045
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1045
> Project: ZooKeeper
>  Issue Type: New Feature
>  Components: quorum, security
>Reporter: Eugene Koontz
>Assignee: Rakesh R
>Priority: Critical
> Fix For: 3.4.10, 3.5.3
>
> Attachments: 0001-ZOOKEEPER-1045-br-3-4.patch, 
> 1045_failing_phunt.tar.gz, HOST_RESOLVER-ZK-1045.patch, QuorumPeer Mutual 
> Authentication Via Sasl Feature Doc - 2016-Sep-25.pdf, 
> TEST-org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.txt, 
> ZK-1045-test-case-failure-logs.zip, ZOOKEEPER-1045 Test Plan.pdf, 
> ZOOKEEPER-1045-00.patch, ZOOKEEPER-1045-Rolling Upgrade Design Proposal.pdf, 
> ZOOKEEPER-1045-br-3-4.patch, ZOOKEEPER-1045-br-3-4.patch, 
> ZOOKEEPER-1045-br-3-4.patch, ZOOKEEPER-1045-br-3-4.patch, 
> ZOOKEEPER-1045-br-3-4.patch, ZOOKEEPER-1045-br-3-4.patch, 
> ZOOKEEPER-1045-br-3-4.patch, ZOOKEEPER-1045-br-3-4.patch, 
> ZOOKEEPER-1045-br-3-4.patch, ZOOKEEPER-1045-br-3-4.patch, 
> ZOOKEEPER-1045-br-3-4.patch, ZOOKEEPER-1045TestValidationDesign.pdf, 
> org.apache.zookeeper.server.quorum.auth.QuorumAuthUpgradeTest.testRollingUpgrade.log
>
>
> ZOOKEEPER-938 addresses mutual authentication between clients and servers. 
> This bug, on the other hand, is for authentication among quorum peers. 
> Hopefully much of the work done on SASL integration with Zookeeper for 
> ZOOKEEPER-938 can be used as a foundation for this enhancement.
> Review board: https://reviews.apache.org/r/47354/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 47354: ZOOKEEPER-1045 : Quorum mutual authentication using SASL mechanism

[Eugene Koontz]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47354/#review154205
---




src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java (line 
188)


typo: should be initializeAuth(..)



src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java (line 
189)


Seems a little strange to call initalizeAuth() if (quorumSaslAuthEnabled == 
false), but not a big deal.



src/java/test/org/apache/zookeeper/server/quorum/QuorumCnxManagerTest.java 
(line 693)


I wonder if it would be clearer to change the variable name from 
clientLoginContext to learnerLoginContext since the we are always calling 
createAndStartManager() with the 3rd argument being the string "QuorumLearner"



src/java/test/org/apache/zookeeper/server/quorum/QuorumCnxManagerTest.java 
(line 695)


As with last comment, maybe this should be called learnerRequireSasl?


- Eugene Koontz


On Oct. 25, 2016, 5:05 a.m., Rakesh R wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47354/
> ---
> 
> (Updated Oct. 25, 2016, 5:05 a.m.)
> 
> 
> Review request for zookeeper, fpj, Ivan Kelly, Patrick Hunt, and Raul 
> Gutierrez Segales.
> 
> 
> Bugs: ZOOKEEPER-1045
> https://issues.apache.org/jira/browse/ZOOKEEPER-1045
> 
> 
> Repository: zookeeper-git
> 
> 
> Description
> ---
> 
> Quorum mutual authentication using SASL mechanism - Digest/Kerberos
> 
> 
> Diffs
> -
> 
>   build.xml 505597a 
>   ivy.xml 95b0e5a 
>   src/java/main/org/apache/zookeeper/Login.java aaa220c 
>   src/java/main/org/apache/zookeeper/SaslClientCallbackHandler.java 
> PRE-CREATION 
>   src/java/main/org/apache/zookeeper/client/ZooKeeperSaslClient.java 21ef0fa 
>   src/java/main/org/apache/zookeeper/server/ZooKeeperSaslServer.java 71870ce 
>   
> src/java/main/org/apache/zookeeper/server/auth/SaslServerCallbackHandler.java 
> 7fdffde 
>   src/java/main/org/apache/zookeeper/server/quorum/Follower.java 043a522 
>   src/java/main/org/apache/zookeeper/server/quorum/Leader.java c83d352 
>   src/java/main/org/apache/zookeeper/server/quorum/Learner.java 647b8a2 
>   src/java/main/org/apache/zookeeper/server/quorum/LearnerHandler.java 
> 8a748c7 
>   src/java/main/org/apache/zookeeper/server/quorum/Observer.java e53f6f2 
>   src/java/main/org/apache/zookeeper/server/quorum/QuorumCnxManager.java 
> 20e5f16 
>   src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java 2f0f21b 
>   src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerConfig.java 
> 0924ef6 
>   src/java/main/org/apache/zookeeper/server/quorum/QuorumPeerMain.java 
> e9c8007 
>   
> src/java/main/org/apache/zookeeper/server/quorum/auth/NullQuorumAuthLearner.java
>  PRE-CREATION 
>   
> src/java/main/org/apache/zookeeper/server/quorum/auth/NullQuorumAuthServer.java
>  PRE-CREATION 
>   src/java/main/org/apache/zookeeper/server/quorum/auth/QuorumAuth.java 
> PRE-CREATION 
>   
> src/java/main/org/apache/zookeeper/server/quorum/auth/QuorumAuthLearner.java 
> PRE-CREATION 
>   src/java/main/org/apache/zookeeper/server/quorum/auth/QuorumAuthServer.java 
> PRE-CREATION 
>   
> src/java/main/org/apache/zookeeper/server/quorum/auth/SaslQuorumAuthLearner.java
>  PRE-CREATION 
>   
> src/java/main/org/apache/zookeeper/server/quorum/auth/SaslQuorumAuthServer.java
>  PRE-CREATION 
>   
> src/java/main/org/apache/zookeeper/server/quorum/auth/SaslQuorumServerCallbackHandler.java
>  PRE-CREATION 
>   src/java/main/org/apache/zookeeper/util/SecurityUtils.java PRE-CREATION 
>   src/java/test/data/kerberos/minikdc-krb5.conf PRE-CREATION 
>   src/java/test/data/kerberos/minikdc.ldiff PRE-CREATION 
>   src/java/test/org/apache/zookeeper/server/quorum/CnxManagerTest.java 
> 8db7fa8 
>   
> src/java/test/org/apache/zookeeper/server/quorum/FLEBackwardElectionRoundTest.java
>  c1259d1 
>   src/java/test/org/apache/zookeeper/server/quorum/FLECompatibilityTest.java 
> 72e4fc9 
>   src/java/test/org/apache/zookeeper/server/quorum/FLEDontCareTest.java 
> a4c0cb0 
>   src/java/test/org/apache/zookeeper/server/quorum/FLELostMessageTest.java 
> 39a53ca 
>   src/java/test/org/apache/zookeeper/server/quorum/LearnerTest.java 2ae57ce 
>   src/java/test/org/apache/zookeeper/server/quorum/QuorumCnxManagerTest.java 
> PRE-CREATION 
>   src/java/test/org/apache/zookeeper/server/quorum/QuorumPeerTestBase.java 
> 85817b2 
>   src/java/test/org/apache/zookeeper/server/quorum/Zab1_0Test.java ab8ce42 
>   
> 

[jira] [Commented] (ZOOKEEPER-2615) Zookeeper server holds onto dead/expired session ids in the watch data structures

[guoping.gp (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15615182#comment-15615182
 ] 

guoping.gp commented on ZOOKEEPER-2615:
---

Is there any recommended fix option on this issue?

> Zookeeper server holds onto dead/expired session ids in the watch data 
> structures
> -
>
> Key: ZOOKEEPER-2615
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2615
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.6
>Reporter: guoping.gp
>Assignee: Camille Fournier
> Fix For: 3.4.10, 3.5.3, 3.6.0
>
>
> The same issue (https://issues.apache.org/jira/browse/ZOOKEEPER-1382) still 
> can be found even with zookeeper 3.4.6.
> this issue cause our production zookeeper cluster leak about 1 million 
> watchs, after restart the server one by one, the watch count decrease to only 
> about 4.
> I can reproduce the issue on my mac,here it is:
> 
> pguodeMacBook-Air:bin pguo$ echo srvr | nc localhost 6181
> Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT
> Latency min/avg/max: 0/1156/128513
> Received: 539
> Sent: 531
> Connections: 1
> Outstanding: 0
> Zxid: 0x10037
> Mode: follower
> Node count: 5
> 
> pguodeMacBook-Air:bin pguo$ echo cons | nc localhost 6181
>  
> /127.0.0.1:55759[1](queued=0,recved=5,sent=5,sid=0x157be2732de,lop=PING,est=1476372631116,to=15000,lcxid=0x1,lzxid=0x,lresp=1476372646260,llat=8,minlat=0,avglat=6,maxlat=17)
>  /0:0:0:0:0:0:0:1:55767[0](queued=0,recved=1,sent=0)
> 
> pguodeMacBook-Air:bin pguo$ echo wchp | nc localhost 6181
> /curator_exists_watch
>   0x357be48e4d90007
>   0x357be48e4d90009
>   0x157be2732de
> as above 4-letter's report shows, 0x357be48e4d90007 and 0x357be48e4d90009 
> are leaked after the two sessions expired 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Failed: ZOOKEEPER- PreCommit Build #4

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/4/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 424764 lines...]
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:391)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:315)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
[junit] at 
org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2016-10-28 11:56:54,123 [myid:] - INFO  
[SyncThread:0:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port14159,name1=Connections,name2=127.0.0.1,name3=0x1009b916a1e]
[junit] 2016-10-28 11:56:54,223 [myid:] - INFO  [main:ZooKeeper@1315] - 
Session: 0x1009b916a1e closed
[junit] 2016-10-28 11:56:54,223 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 156123
[junit] 2016-10-28 11:56:54,223 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 1643
[junit] 2016-10-28 11:56:54,224 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testWatcherAutoResetWithLocal
[junit] 2016-10-28 11:56:54,224 [myid:] - INFO  [main:ClientBase@543] - 
tearDown starting
[junit] 2016-10-28 11:56:54,224 [myid:] - INFO  [main:ClientBase@513] - 
STOPPING server
[junit] 2016-10-28 11:56:54,224 [myid:] - INFO  
[main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:14159
[junit] 2016-10-28 11:56:54,223 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x1009b916a1e
[junit] 2016-10-28 11:56:54,229 [myid:] - INFO  [main:ZooKeeperServer@529] 
- shutting down
[junit] 2016-10-28 11:56:54,229 [myid:] - ERROR [main:ZooKeeperServer@501] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2016-10-28 11:56:54,230 [myid:] - INFO  
[main:SessionTrackerImpl@232] - Shutting down
[junit] 2016-10-28 11:56:54,230 [myid:] - INFO  
[main:PrepRequestProcessor@996] - Shutting down
[junit] 2016-10-28 11:56:54,230 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2016-10-28 11:56:54,230 [myid:] - INFO  [ProcessThread(sid:0 
cport:14159)::PrepRequestProcessor@156] - PrepRequestProcessor exited loop!
[junit] 2016-10-28 11:56:54,230 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2016-10-28 11:56:54,232 [myid:] - INFO  
[main:FinalRequestProcessor@481] - shutdown of request processor complete
[junit] 2016-10-28 11:56:54,233 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port14159,name1=InMemoryDataTree]
[junit] 2016-10-28 11:56:54,233 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port14159]
[junit] 2016-10-28 11:56:54,253 [myid:] - INFO  
[main:FourLetterWordMain@85] - connecting to 127.0.0.1 14159
[junit] 2016-10-28 11:56:54,253 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2016-10-28 11:56:54,261 [myid:] - INFO  [main:ClientBase@568] - 
fdcount after test is: 4820 at start it was 4820
[junit] 2016-10-28 11:56:54,261 [myid:] - INFO  [main:ZKTestCase$1@65] - 
SUCCEEDED testWatcherAutoResetWithLocal
[junit] 2016-10-28 11:56:54,261 [myid:] - INFO  [main:ZKTestCase$1@60] - 
FINISHED testWatcherAutoResetWithLocal
[junit] Tests run: 101, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
458.236 sec, Thread: 2, Class: org.apache.zookeeper.test.NioNettySuiteTest
[junit] 2016-10-28 11:56:54,451 [myid:127.0.0.1:13915] - INFO  
[main-SendThread(127.0.0.1:13915):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:13915. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2016-10-28 11:56:54,451 [myid:127.0.0.1:13915] - WARN  
[main-SendThread(127.0.0.1:13915):ClientCnxn$SendThread@1235] - Session 
0x1009b8a85c2 for server 127.0.0.1/127.0.0.1:13915, unexpected error, 
closing socket connection and attempting reconnect
[junit]