[GitHub] zookeeper issue #310: [ZOOKEEPER-2845][Test] Test used to reproduce the data...

[lvfangmin]

Github user lvfangmin commented on the issue:

https://github.com/apache/zookeeper/pull/310
  
@revans2 my teammate was working on the fix, and he was planning run it on 
prod for a while before sending out the diff. I'll sync with him today about 
the status. 


---

[jira] [Commented] (ZOOKEEPER-2845) Data inconsistency issue due to retain database in leader election

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154246#comment-16154246
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2845:
---

Github user lvfangmin commented on the issue:

https://github.com/apache/zookeeper/pull/310
  
@revans2 my teammate was working on the fix, and he was planning run it on 
prod for a while before sending out the diff. I'll sync with him today about 
the status. 


> Data inconsistency issue due to retain database in leader election
> --
>
> Key: ZOOKEEPER-2845
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2845
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: quorum
>Affects Versions: 3.4.10, 3.5.3, 3.6.0
>Reporter: Fangmin Lv
>Assignee: Fangmin Lv
>Priority: Critical
>
> In ZOOKEEPER-2678, the ZKDatabase is retained to reduce the unavailable time 
> during leader election. In ZooKeeper ensemble, it's possible that the 
> snapshot is ahead of txn file (due to slow disk on the server, etc), or the 
> txn file is ahead of snapshot due to no commit message being received yet. 
> If snapshot is ahead of txn file, since the SyncRequestProcessor queue will 
> be drained during shutdown, the snapshot and txn file will keep consistent 
> before leader election happening, so this is not an issue.
> But if txn is ahead of snapshot, it's possible that the ensemble will have 
> data inconsistent issue, here is the simplified scenario to show the issue:
> Let's say we have a 3 servers in the ensemble, server A and B are followers, 
> and C is leader, and all the snapshot and txn are up to T0:
> 1. A new request reached to leader C to create Node N, and it's converted to 
> txn T1 
> 2. Txn T1 was synced to disk in C, but just before the proposal reaching out 
> to the followers, A and B restarted, so the T1 didn't exist in A and B
> 3. A and B formed a new quorum after restart, let's say B is the leader
> 4. C changed to looking state due to no enough followers, it will sync with 
> leader B with last Zxid T0, which will have an empty diff sync
> 5. Before C take snapshot it restarted, it replayed the txns on disk which 
> includes T1, now it will have Node N, but A and B doesn't have it.
> Also I included the a test case to reproduce this issue consistently. 
> We have a totally different RetainDB version which will avoid this issue by 
> doing consensus between snapshot and txn files before leader election, will 
> submit for review.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

ZooKeeper-trunk-openjdk7 - Build # 1607 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1607/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 66.00 MB...]
[junit] 2017-09-05 20:01:06,326 [myid:] - WARN  [New I/O boss 
#15288:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 
0xf1a745e7] EXCEPTION: java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24813
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24813
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2017-09-05 20:01:06,326 [myid:] - INFO  [New I/O boss 
#15288:ClientCnxnSocketNetty@208] - channel is told closing
[junit] 2017-09-05 20:01:06,326 [myid:127.0.0.1:24813] - INFO  
[main-SendThread(127.0.0.1:24813):ClientCnxn$SendThread@1231] - channel for 
sessionid 0x2067e3d9f55 is lost, closing socket connection and attempting 
reconnect
[junit] 2017-09-05 20:01:06,358 [myid:127.0.0.1:24736] - INFO  
[main-SendThread(127.0.0.1:24736):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:24736. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-09-05 20:01:06,359 [myid:] - INFO  [New I/O boss 
#7497:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24736
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit] at java.lang.Thread.run(Thread.java:745)
[junit] 2017-09-05 20:01:06,417 [myid:] - WARN  [New I/O boss 
#7497:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 
0x8c48fd3d] EXCEPTION: java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24736
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24736
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 

ZooKeeper_branch34 - Build # 2071 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch34/2071/

###
## LAST 60 LINES OF THE CONSOLE 
###
Started by timer
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
ZooKeeper_branch34 #2071
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Created] (ZOOKEEPER-2892) Improve lazy initialize and close stream for `PrepRequestProcessor`

[Benedict Jin (JIRA)]

Benedict Jin created ZOOKEEPER-2892:
---

 Summary: Improve lazy initialize and close stream for 
`PrepRequestProcessor`
 Key: ZOOKEEPER-2892
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2892
 Project: ZooKeeper
  Issue Type: Improvement
  Components: server
Reporter: Benedict Jin
Assignee: Benedict Jin


Improve lazy initialize and close stream for `PrepRequestProcessor`

* Delay the initialization of `ChangeRecord` and `ReconfigRequest` variables
* Close the `ByteArrayOutputStream` I/O stream



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] zookeeper issue #361: ZOOKEEPER-2892: Improve lazy initialize and close stre...

[asdf2014]

Github user asdf2014 commented on the issue:

https://github.com/apache/zookeeper/pull/361
  
Seems like jenkins cannot work normally? @hanm 

```bash
GitHub pull request #361 to apache/zookeeper
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Archive the artifacts? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #998
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #998
[description-setter] Could not determine description.
Putting comment on the pull request
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending email to: dev@zookeeper.apache.org
Warning: this build has no associated authentication, so build permissions 
may be lacking, and downstream projects which cannot even be seen by an 
anonymous user will be silently skipped
Finished: FAILURE
```


---

[jira] [Commented] (ZOOKEEPER-2892) Improve lazy initialize and close stream for `PrepRequestProcessor`

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154707#comment-16154707
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2892:
---

Github user asdf2014 commented on the issue:

https://github.com/apache/zookeeper/pull/361
  
Seems like jenkins cannot work normally? @hanm 

```bash
GitHub pull request #361 to apache/zookeeper
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Archive the artifacts? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #998
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #998
[description-setter] Could not determine description.
Putting comment on the pull request
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Sending email to: dev@zookeeper.apache.org
Warning: this build has no associated authentication, so build permissions 
may be lacking, and downstream projects which cannot even be seen by an 
anonymous user will be silently skipped
Finished: FAILURE
```


> Improve lazy initialize and close stream for `PrepRequestProcessor`
> ---
>
> Key: ZOOKEEPER-2892
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2892
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: server
>Reporter: Benedict Jin
>Assignee: Benedict Jin
>
> Improve lazy initialize and close stream for `PrepRequestProcessor`
> * Delay the initialization of `ChangeRecord` and `ReconfigRequest` variables
> * Close the `ByteArrayOutputStream` I/O stream



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

ZooKeeper_branch34_jdk7 - Build # 1643 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch34_jdk7/1643/

###
## LAST 60 LINES OF THE CONSOLE 
###
Started by timer
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
ZooKeeper_branch34_jdk7 #1643
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

ZooKeeper-trunk - Build # 3524 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk/3524/

###
## LAST 60 LINES OF THE CONSOLE 
###
Started by timer
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Publish FindBugs analysis results? failed: no workspace for 
ZooKeeper-trunk #3524
ERROR: Step ?Scan for compiler warnings? failed: no workspace for 
ZooKeeper-trunk #3524
ERROR: Step ?Archive the artifacts? failed: no workspace for ZooKeeper-trunk 
#3524
ERROR: Step ?Record fingerprints of files to track usage? failed: no workspace 
for ZooKeeper-trunk #3524
ERROR: Step ?JIRA: Update relevant issues? failed: no workspace for 
ZooKeeper-trunk #3524
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
ZooKeeper-trunk #3524
ERROR: Step ?Publish Javadoc? failed: no workspace for ZooKeeper-trunk #3524
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

Failed: ZOOKEEPER- PreCommit Build #998

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/998/

###
## LAST 60 LINES OF THE CONSOLE 
###
GitHub pull request #361 to apache/zookeeper
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Archive the artifacts? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #998
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #998
[description-setter] Could not determine description.
Putting comment on the pull request
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

Failed: ZOOKEEPER- PreCommit Build #997

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/997/

###
## LAST 60 LINES OF THE CONSOLE 
###
GitHub pull request #361 to apache/zookeeper
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Archive the artifacts? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #997
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #997
[description-setter] Could not determine description.
Putting comment on the pull request
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Commented] (ZOOKEEPER-2892) Improve lazy initialize and close stream for `PrepRequestProcessor`

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154700#comment-16154700
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2892:
---

GitHub user asdf2014 opened a pull request:

https://github.com/apache/zookeeper/pull/361

ZOOKEEPER-2892: Improve lazy initialize and close stream for 
`PrepRequestProcessor`

Improve lazy initialize and close stream for `PrepRequestProcessor`

* Delay the initialization of `ChangeRecord` and `ReconfigRequest` variables
* Close the `ByteArrayOutputStream` I/O stream

@hanm PTAL

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/asdf2014/zookeeper ZOOKEEPER-2892

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/361.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #361


commit 3b053f2fe57b3b4bc52dac29134b6f68e7b48c6d
Author: asdf2014 
Date:   2017-09-06T02:02:28Z

ZOOKEEPER-2892: Improve lazy initialize and close stream for 
`PrepRequestProcessor`




> Improve lazy initialize and close stream for `PrepRequestProcessor`
> ---
>
> Key: ZOOKEEPER-2892
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2892
> Project: ZooKeeper
>  Issue Type: Improvement
>  Components: server
>Reporter: Benedict Jin
>Assignee: Benedict Jin
>
> Improve lazy initialize and close stream for `PrepRequestProcessor`
> * Delay the initialization of `ChangeRecord` and `ReconfigRequest` variables
> * Close the `ByteArrayOutputStream` I/O stream



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] zookeeper pull request #361: ZOOKEEPER-2892: Improve lazy initialize and clo...

[asdf2014]

GitHub user asdf2014 opened a pull request:

https://github.com/apache/zookeeper/pull/361

ZOOKEEPER-2892: Improve lazy initialize and close stream for 
`PrepRequestProcessor`

Improve lazy initialize and close stream for `PrepRequestProcessor`

* Delay the initialization of `ChangeRecord` and `ReconfigRequest` variables
* Close the `ByteArrayOutputStream` I/O stream

@hanm PTAL

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/asdf2014/zookeeper ZOOKEEPER-2892

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/361.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #361


commit 3b053f2fe57b3b4bc52dac29134b6f68e7b48c6d
Author: asdf2014 
Date:   2017-09-06T02:02:28Z

ZOOKEEPER-2892: Improve lazy initialize and close stream for 
`PrepRequestProcessor`




---

ZooKeeper_branch35_jdk7 - Build # 1101 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch35_jdk7/1101/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 69.99 MB...]
[junit] 2017-09-05 08:48:57,639 [myid:127.0.0.1:30202] - INFO  
[main-SendThread(127.0.0.1:30202):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:30202. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-09-05 08:48:57,639 [myid:127.0.0.1:30202] - WARN  
[main-SendThread(127.0.0.1:30202):ClientCnxn$SendThread@1235] - Session 
0x3067c339107 for server 127.0.0.1/127.0.0.1:30202, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-09-05 08:48:58,015 [myid:] - INFO  [ProcessThread(sid:0 
cport:30319)::PrepRequestProcessor@611] - Processed session termination for 
sessionid: 0x1067c36f61e
[junit] 2017-09-05 08:48:58,016 [myid:] - INFO  
[SyncThread:0:MBeanRegistry@128] - Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port30319,name1=Connections,name2=127.0.0.1,name3=0x1067c36f61e]
[junit] 2017-09-05 08:48:58,017 [myid:] - INFO  [main:ZooKeeper@1334] - 
Session: 0x1067c36f61e closed
[junit] 2017-09-05 08:48:58,017 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@82] - Memory used 225415
[junit] 2017-09-05 08:48:58,017 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@513] - EventThread shut down for 
session: 0x1067c36f61e
[junit] 2017-09-05 08:48:58,017 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@87] - Number of threads 2432
[junit] 2017-09-05 08:48:58,018 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@102] - FINISHED TEST METHOD 
testWatcherAutoResetWithLocal
[junit] 2017-09-05 08:48:58,018 [myid:] - INFO  [main:ClientBase@586] - 
tearDown starting
[junit] 2017-09-05 08:48:58,018 [myid:] - INFO  [main:ClientBase@556] - 
STOPPING server
[junit] 2017-09-05 08:48:58,018 [myid:] - INFO  
[main:NettyServerCnxnFactory@464] - shutdown called 0.0.0.0/0.0.0.0:30319
[junit] 2017-09-05 08:48:58,026 [myid:] - INFO  [main:ZooKeeperServer@541] 
- shutting down
[junit] 2017-09-05 08:48:58,027 [myid:] - ERROR [main:ZooKeeperServer@505] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2017-09-05 08:48:58,027 [myid:] - INFO  
[main:SessionTrackerImpl@232] - Shutting down
[junit] 2017-09-05 08:48:58,027 [myid:] - INFO  
[main:PrepRequestProcessor@1005] - Shutting down
[junit] 2017-09-05 08:48:58,027 [myid:] - INFO  
[main:SyncRequestProcessor@191] - Shutting down
[junit] 2017-09-05 08:48:58,027 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2017-09-05 08:48:58,027 [myid:] - INFO  [ProcessThread(sid:0 
cport:30319)::PrepRequestProcessor@155] - PrepRequestProcessor exited loop!
[junit] 2017-09-05 08:48:58,028 [myid:] - INFO  
[main:FinalRequestProcessor@481] - shutdown of request processor complete
[junit] 2017-09-05 08:48:58,028 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port30319,name1=InMemoryDataTree]
[junit] 2017-09-05 08:48:58,028 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port30319]
[junit] 2017-09-05 08:48:58,029 [myid:] - INFO  
[main:FourLetterWordMain@87] - connecting to 127.0.0.1 30319
[junit] 2017-09-05 08:48:58,029 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2017-09-05 08:48:58,039 [myid:] - INFO  [main:ClientBase@611] - 
fdcount after test is: 7170 at start it was 7170
[junit] 2017-09-05 08:48:58,039 [myid:] - INFO  [main:ZKTestCase$1@68] - 
SUCCEEDED testWatcherAutoResetWithLocal
[junit] 2017-09-05 08:48:58,040 [myid:] - INFO  [main:ZKTestCase$1@63] - 
FINISHED testWatcherAutoResetWithLocal
[junit] 2017-09-05 08:48:58,042 [myid:127.0.0.1:30137] - INFO  
[main-SendThread(127.0.0.1:30137):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:30137. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-09-05 08:48:58,042 [myid:127.0.0.1:30137] - WARN  
[main-SendThread(127.0.0.1:30137):ClientCnxn$SendThread@1235] - Session 
0x1067c314715 for server 127.0.0.1/127.0.0.1:30137, unexpected error, 
closing socket 

[jira] [Commented] (ZOOKEEPER-2572) Potential resource leak in FileTxnLog.truncate

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153763#comment-16153763
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2572:
---

Github user maoling commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/356#discussion_r137008765
  
--- Diff: 
src/java/main/org/apache/zookeeper/server/persistence/FileTxnLog.java ---
@@ -399,18 +403,20 @@ public boolean truncate(long zxid) throws IOException 
{
 }
 long pos = input.getPosition();
 // now, truncate at the current position
-RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
+raf = new RandomAccessFile(itr.logFile, "rw");
 raf.setLength(pos);
-raf.close();
 while(itr.goToNextLog()) {
-if (!itr.logFile.delete()) {
-LOG.warn("Unable to truncate {}", itr.logFile);
+try {
+Files.delete(itr.logFile.toPath());
+} catch (NoSuchFileException e) {
 }
 }
--- End diff --

- A option:  remain the origin code in the master branch unchanged.just log 
the `itr.logFile` and `zxid`
- B option:  choose `Files.delete`  because it throws `exception` rather 
than `itr.logFile.delete()` returns `boolean` ,**dont't catch it** and it will 
be caught in `Learner.java#syncWithLeader` and log it with `zxid`
- am I right?


> Potential resource leak in FileTxnLog.truncate
> --
>
> Key: ZOOKEEPER-2572
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2572
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.9, 3.5.2
>Reporter: Michael Han
>Assignee: gaoshu
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In FileTxnLog.truncate, we have:
> {code}
> public boolean truncate(long zxid) throws IOException {
> FileTxnIterator itr = null;
> try {
> itr = new FileTxnIterator(this.logDir, zxid);
> PositionInputStream input = itr.inputStream;
> if(input == null) {
> throw new IOException("No log files found to truncate! This 
> could " +
> "happen if you still have snapshots from an old setup 
> or " +
> "log files were deleted accidentally or dataLogDir 
> was changed in zoo.cfg.");
> }
> long pos = input.getPosition();
> // now, truncate at the current position
> RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
> raf.setLength(pos);
> raf.close();
> while(itr.goToNextLog()) {
> if (!itr.logFile.delete()) {
> LOG.warn("Unable to truncate {}", itr.logFile);
> }
> }
> } finally {
> close(itr);
> }
> return true;
> }
> {code}
> {{raf}} here can be potentially in a state of not closed after leaving the 
> method, if there is an (IO) exception thrown from setLength.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] zookeeper pull request #356: ZOOKEEPER-2572: Fix potential resource leak in ...

[maoling]

Github user maoling commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/356#discussion_r137008765
  
--- Diff: 
src/java/main/org/apache/zookeeper/server/persistence/FileTxnLog.java ---
@@ -399,18 +403,20 @@ public boolean truncate(long zxid) throws IOException 
{
 }
 long pos = input.getPosition();
 // now, truncate at the current position
-RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
+raf = new RandomAccessFile(itr.logFile, "rw");
 raf.setLength(pos);
-raf.close();
 while(itr.goToNextLog()) {
-if (!itr.logFile.delete()) {
-LOG.warn("Unable to truncate {}", itr.logFile);
+try {
+Files.delete(itr.logFile.toPath());
+} catch (NoSuchFileException e) {
 }
 }
--- End diff --

- A option:  remain the origin code in the master branch unchanged.just log 
the `itr.logFile` and `zxid`
- B option:  choose `Files.delete`  because it throws `exception` rather 
than `itr.logFile.delete()` returns `boolean` ,**dont't catch it** and it will 
be caught in `Learner.java#syncWithLeader` and log it with `zxid`
- am I right?


---

Success: ZOOKEEPER- PreCommit Build #996

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/996/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 35.42 MB...]
 [exec] 
 [exec] +1 @author.  The patch does not contain any @author tags.
 [exec] 
 [exec] +0 tests included.  The patch appears to be a documentation 
patch that doesn't require tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
(version 3.0.1) warnings.
 [exec] 
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.
 [exec] 
 [exec] +1 core tests.  The patch passed core unit tests.
 [exec] 
 [exec] +1 contrib tests.  The patch passed contrib unit tests.
 [exec] 
 [exec] Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/996//testReport/
 [exec] Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/996//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
 [exec] Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/996//console
 [exec] 
 [exec] This message is automatically generated.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Adding comment to Jira.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] Comment added.
 [exec] e4817635e33549f844e17168836a1856e0d46616 logged out
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Finished build.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] mv: 
‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess’
 and 
‘/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess’
 are the same file

BUILD SUCCESSFUL
Total time: 35 minutes 33 seconds
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[description-setter] Description set: ZOOKEEPER-2891
Putting comment on the pull request
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Success
Sending email for trigger: Success
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Commented] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153524#comment-16153524
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2890:
---

GitHub user xoiss opened a pull request:

https://github.com/apache/zookeeper/pull/358

branch-3.4 -- bugfix -- ZOOKEEPER-2890

Fixes https://issues.apache.org/jira/browse/ZOOKEEPER-2890

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xoiss/zookeeper 
branch-3.4-bugfix-zookeeper-2890

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/358.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #358


commit a752eee7837d241d79151fa6053571df7a00761e
Author: Patrick D. Hunt 
Date:   2013-10-24T05:11:59Z

ZOOKEEPER-1744. clientPortAddress breaks "zkServer.sh status" (Nick Ohanian 
via phunt)

git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1535279 
13f79535-47bb-0310-9956-ffa450edef68

commit 3dae9c4770f77e28fa6bba07e58133f1530f381a
Author: Thawan Kooburat 
Date:   2013-10-24T18:52:31Z

ZOOKEEPER-1798. Fix race condition in testNormalObserverRun (thawan, fpj 
via thawan)

git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1535497 
13f79535-47bb-0310-9956-ffa450edef68

commit 2bdf4678c70ff7639761c459708922cb2309b078
Author: Patrick D. Hunt 
Date:   2013-11-12T06:30:06Z

ZOOKEEPER-1812. ZooInspector reconnection always fails if first connection 
fails (Benjamin Jaton via phunt)

git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1540960 
13f79535-47bb-0310-9956-ffa450edef68

commit ffd23df5f0f5038bee5c3b05f54f12a80f8098b1
Author: Camille Fournier 
Date:   2013-11-13T02:22:39Z

ZOOKEEPER-1666. Avoid Reverse DNS lookup if the hostname in 
  connection string is literal IP address. (George Cao via camille)


git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1541359 
13f79535-47bb-0310-9956-ffa450edef68

commit bf1814b9943588ac25b46aa158d994f5900b9202
Author: Thawan Kooburat 
Date:   2013-11-14T04:43:46Z

ZOOKEEPER-1798. Fix race condition in testNormalObserverRun (Part 2) 
(thawan, fpj via thawan)

git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1541814 
13f79535-47bb-0310-9956-ffa450edef68

commit 6503f12e144bc6b0652f7adf558bdc28c984e6c0
Author: Flavio Paiva Junqueira 
Date:   2013-11-15T18:12:07Z

ZOOKEEPER-1786. ZooKeeper data model documentation is incorrect
  (Niraj Tolia via fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1542356 
13f79535-47bb-0310-9956-ffa450edef68

commit a452d791cff92d65937142c3475a9894264376f6
Author: Flavio Paiva Junqueira 
Date:   2013-11-16T10:06:19Z

ZOOKEEPER-1808. Add version to FLE notifications for 3.4 branch (fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1542489 
13f79535-47bb-0310-9956-ffa450edef68

commit b770aeb0c8bcb28a9b653b1dc0edac623c75b509
Author: Flavio Paiva Junqueira 
Date:   2013-11-17T11:42:22Z

ZOOKEEPER-1597. Windows build failing (michim via fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1542710 
13f79535-47bb-0310-9956-ffa450edef68

commit 7f069c0e084a74f6de661267b828ff3fde30bd11
Author: Flavio Paiva Junqueira 
Date:   2013-11-23T18:21:38Z

ZOOKEEPER-1817. Fix don't care for b3.4 (fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1544858 
13f79535-47bb-0310-9956-ffa450edef68

commit c62f2ffa7b9fb026004f66d788ebf1e293d2fdf8
Author: Flavio Paiva Junqueira 
Date:   2013-11-26T23:43:29Z

ZOOKEEPER-1653. zookeeper fails to start because of inconsistent epoch 
(michim via fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1545883 
13f79535-47bb-0310-9956-ffa450edef68

commit 605c8d1b0db0baa57582bb39229c80bfec832ae3
Author: Flavio Paiva Junqueira 
Date:   2013-11-27T23:23:11Z

ZOOKEEPER-1821. very ugly warning when compiling load_gen.c (german blanco 
via fpj)


git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1546231 
13f79535-47bb-0310-9956-ffa450edef68

commit a1cdea63de42ed1eaba224570ad0304c224b21c1
Author: Michi Mutsuzaki 
Date:   2013-12-04T04:20:53Z

ZOOKEEPER-1632. fix memory leaks in cli_st (fpj via michim)


git-svn-id: 

[jira] [Commented] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[Alexander A. Strelets (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153522#comment-16153522
 ] 

Alexander A. Strelets commented on ZOOKEEPER-2890:
--

Also I have briefly verified the remaining cases in the 
_deserialize_response()_.

As soon as all of them except _case COMPLETION_VOID_ and the mentioned _case 
COMPLETION_STRING_ are not used for _multi_ response, there is no problem with 
them.

However, if some of them are included into _multi_ response in further 
releases, they also must be patched in the same way.

Here is the brief:

case COMPLETION_DATA:  // get response
  deserialize_Buffer(in, "data", >data) may left v->data uninitialized
  ... then deallocate_Buffer(>data) will free by uninitialized v->data

case COMPLETION_STRINGLIST: // get_children response
  deserialize_String_vector(in, "children", >children) calls 
start_vector(in, tag, >count) which may left v->children.count uninitialized
  ... then deserialize_String_vector() calls calloc(v->count, sizeof(*v->data)) 
which may face insufficient memory if v->count is a big number
  ... or at least lead to significant delay in executing the further 
for(i=0;icount;i++)

case COMPLETION_STRINGLIST_STAT: // get_children2 response
  the same as the above case

case COMPLETION_STRING: // create response
  described in the main part of the ticket
  deserialize_String(in, "path", >path) may left v->path uninitialized
  ... then deallocate_String(>path) will free by uninitialized v->data

case COMPLETION_ACLLIST: // get_acl response
  deserialize_ACL_vector(in, "acl", >acl) calls start_vector(in, tag, 
>count) which may left v->acl.count uninitialized
  ... then deserialize_ACL_vector() calls calloc(v->count, sizeof(*v->data)) 
which may face insufficient memory if v->count is a big number
  ... or at least lead to significant delay in executing the further 
for(i=0;icount;i++)

On the other hand:
case COMPLETION_STAT -- looks safe as soon as _free()_ is not called for the 
_struct Stat_.

> Local automatic variable is left uninitialized and then freed.
> --
>
> Key: ZOOKEEPER-2890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
> automatic variable *_struct CreateResponse res_* which is +left 
> uninitialized+ and passed to the function _deserialize_GetACLResponse()_ and 
> then to _deallocate_GetACLResponse()_.
> The _deserialize_ function, which is called the first, is expected to assign 
> the _res_ variable with a value from the parsed _struct iarchive *ia_. But, 
> if _ia_ contains for example insufficient amount of bytes the 
> _deserialize_String()_ function refuses of assigning a value to _res_, and 
> _res_ stays uninitialized (the true case is described below). Then, the 
> _deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
> with arguments. If incidentally the memory region in the program stack under 
> the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
> address+.
> The true case: this happens when an active _multi_ request with _create_ 
> sub-request is completed on call to _zookeeper_close()_ with the so called 
> "Fake response" which is fabricated by the function _free_completions()_. 
> Such response includes only the header but +zero bytes for the body+. The 
> significant condition is that the _create_ request is not a stand-alone one, 
> but namely a sub-request within the _multi_ request. In this case the 
> _deserialize_response()_ is called recursively (for each sub-request), and 
> when it is called for the _create_ subrequest (from the nested 
> _deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
> the _if (failed)_ condition branches to the _else_ part. Note that in the 
> stand-alone create-request case this does not occur.
> *I suspect this may happen not only due to call to _zookeeper_close()_ but on 
> reception of a true multi-response from the server* containing insufficient 
> number of bytes (I'm not sure if it can be a proper response from the server 
> with an error overall status and empty or insufficient payload).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

ZooKeeper-trunk-jdk8 - Build # 1191 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-jdk8/1191/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 60.34 MB...]
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:19350
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit] at java.lang.Thread.run(Thread.java:748)
[junit] 2017-09-05 11:56:09,728 [myid:] - INFO  [New I/O boss 
#2601:ClientCnxnSocketNetty@208] - channel is told closing
[junit] 2017-09-05 11:56:09,728 [myid:127.0.0.1:19350] - INFO  
[main-SendThread(127.0.0.1:19350):ClientCnxn$SendThread@1231] - channel for 
sessionid 0x105cb50fba30001 is lost, closing socket connection and attempting 
reconnect
[junit] 2017-09-05 11:56:09,770 [myid:127.0.0.1:19427] - INFO  
[main-SendThread(127.0.0.1:19427):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:19427. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-09-05 11:56:09,771 [myid:] - INFO  [New I/O boss 
#5304:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:19427
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit] at java.lang.Thread.run(Thread.java:748)
[junit] 2017-09-05 11:56:09,821 [myid:] - WARN  [New I/O boss 
#5304:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 
0x864c4fbd] EXCEPTION: java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:19427
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:19427
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit] at java.lang.Thread.run(Thread.java:748)
[junit] 2017-09-05 11:56:09,821 [myid:] 

[GitHub] zookeeper pull request #358: branch-3.4 -- bugfix -- ZOOKEEPER-2890

[xoiss]

Github user xoiss closed the pull request at:

https://github.com/apache/zookeeper/pull/358


---

[GitHub] zookeeper issue #358: branch-3.4 -- bugfix -- ZOOKEEPER-2890

[xoiss]

Github user xoiss commented on the issue:

https://github.com/apache/zookeeper/pull/358
  
Sorry, I wanna propose it not for the master branch -- closed.


---

[GitHub] zookeeper pull request #359: branch-3.4 -- bugfix -- ZOOKEEPER-2890

[xoiss]

GitHub user xoiss opened a pull request:

https://github.com/apache/zookeeper/pull/359

branch-3.4 -- bugfix -- ZOOKEEPER-2890

Fixes https://issues.apache.org/jira/browse/ZOOKEEPER-2890

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xoiss/zookeeper 
branch-3.4-bugfix-zookeeper-2890

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/359.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #359


commit 2dc3664e52a0f1ae82c5c4fdc800921548bfc087
Author: xoiss 
Date:   2017-09-05T11:28:36Z

ZOOKEEPER-2890 - fix freing by uninitialized address.




---

ZooKeeper_branch35_jdk8 - Build # 660 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch35_jdk8/660/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 64.49 MB...]
[junit] 2017-09-05 12:09:32,170 [myid:] - INFO  [ProcessThread(sid:0 
cport:14161)::PrepRequestProcessor@155] - PrepRequestProcessor exited loop!
[junit] 2017-09-05 12:09:32,170 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@169] - SyncRequestProcessor exited!
[junit] 2017-09-05 12:09:32,170 [myid:] - INFO  
[main:FinalRequestProcessor@481] - shutdown of request processor complete
[junit] 2017-09-05 12:09:32,171 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean 
[org.apache.ZooKeeperService:name0=StandaloneServer_port14161,name1=InMemoryDataTree]
[junit] 2017-09-05 12:09:32,171 [myid:] - INFO  [main:MBeanRegistry@128] - 
Unregister MBean [org.apache.ZooKeeperService:name0=StandaloneServer_port14161]
[junit] 2017-09-05 12:09:32,171 [myid:] - INFO  
[main:FourLetterWordMain@87] - connecting to 127.0.0.1 14161
[junit] 2017-09-05 12:09:32,171 [myid:] - INFO  [main:JMXEnv@146] - 
ensureOnly:[]
[junit] 2017-09-05 12:09:32,179 [myid:] - INFO  [main:ClientBase@611] - 
fdcount after test is: 1404 at start it was 1415
[junit] 2017-09-05 12:09:32,180 [myid:] - INFO  [main:ZKTestCase$1@68] - 
SUCCEEDED testWatcherAutoResetWithLocal
[junit] 2017-09-05 12:09:32,180 [myid:] - INFO  [main:ZKTestCase$1@63] - 
FINISHED testWatcherAutoResetWithLocal
[junit] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
420.356 sec, Thread: 2, Class: org.apache.zookeeper.test.NioNettySuiteTest
[junit] 2017-09-05 12:09:32,421 [myid:127.0.0.1:13979] - INFO  
[main-SendThread(127.0.0.1:13979):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:13979. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-09-05 12:09:32,421 [myid:127.0.0.1:13979] - WARN  
[main-SendThread(127.0.0.1:13979):ClientCnxn$SendThread@1235] - Session 
0x10797c48606 for server 127.0.0.1/127.0.0.1:13979, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-09-05 12:09:32,734 [myid:127.0.0.1:14038] - INFO  
[main-SendThread(127.0.0.1:14038):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:14038. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-09-05 12:09:32,735 [myid:127.0.0.1:14038] - WARN  
[main-SendThread(127.0.0.1:14038):ClientCnxn$SendThread@1235] - Session 
0x10797c6c8a4 for server 127.0.0.1/127.0.0.1:14038, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-09-05 12:09:32,770 [myid:127.0.0.1:14044] - INFO  
[main-SendThread(127.0.0.1:14044):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:14044. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-09-05 12:09:32,771 [myid:127.0.0.1:14044] - WARN  
[main-SendThread(127.0.0.1:14044):ClientCnxn$SendThread@1235] - Session 
0x30797c6c8b1 for server 127.0.0.1/127.0.0.1:14044, unexpected error, 
closing socket connection and attempting reconnect
[junit] java.net.ConnectException: Connection refused
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
[junit] at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
[junit] 2017-09-05 12:09:32,968 [myid:] - INFO  
[SessionTracker:SessionTrackerImpl@158] - SessionTrackerImpl exited loop!
[junit] 2017-09-05 12:09:33,194 [myid:127.0.0.1:13915] - INFO  
[main-SendThread(127.0.0.1:13915):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:13915. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-09-05 12:09:33,195 [myid:127.0.0.1:13915] - WARN  

Failed: ZOOKEEPER- PreCommit Build #993

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/993/

###
## LAST 60 LINES OF THE CONSOLE 
###
GitHub pull request #355 to apache/zookeeper
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Archive the artifacts? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #993
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #993
[description-setter] Could not determine description.
Putting comment on the pull request
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Updated] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2890:
-
Description: 
Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
automatic variable *_struct CreateResponse res_* which is +left uninitialized+ 
and passed to the function _deserialize_GetACLResponse()_ and then to 
_deallocate_GetACLResponse()_.

The _deserialize_ function, which is called the first, is expected to assign 
the _res_ variable with a value from the parsed _struct iarchive *ia_. But, if 
_ia_ contains for example insufficient amount of bytes the 
_deserialize_String()_ function refuses of assigning a value to _res_, and 
_res_ stays uninitialized (the true case is described below). Then, the 
_deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
with arguments. If incidentally the memory region in the program stack under 
the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
address+.

The true case: this happens when an active _multi_ request with _create_ 
sub-request is completed on call to _zookeeper_close()_ with the so called 
"Fake response" which is fabricated by the function _free_completions()_. Such 
response includes only the header but +zero bytes for the body+. The 
significant condition is that the _create_ request is not a stand-alone one, 
but namely a sub-request within the _multi_ request. In this case the 
_deserialize_response()_ is called recursively (for each sub-request), and when 
it is called for the _create_ subrequest (from the nested 
_deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
the _if (failed)_ condition branches to the _else_ part. Note that in the 
stand-alone create-request case this does not occur.

*I suspect this may happen not only due to call to _zookeeper_close()_ but on 
reception of a true multi-response from the server* containing insufficient 
number of bytes (I'm not sure if it can be a proper response from the server 
with an error overall status and empty or insufficient payload).

  was:
Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
automatic variable *_struct CreateResponse res_* which is +left uninitialized+ 
and passed to the function _deserialize_GetACLResponse()_ and then to 
_deallocate_GetACLResponse()_.

The _deserialize_ function, which is called the first, is expected to assign 
the _res_ variable with a value from the parsed _struct iarchive *ia_. But, if 
_ia_ contains for example insufficient amount of bytes the 
_deserialize_String()_ function refuses of assigning a value to _res_, and 
_res_ stays uninitialized (the true case is described below). Then, the 
_deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
with arguments. If incidentally the memory region in the program stack under 
the _res_ was not equal to NULL, the last call +leads to _free()_ for an 
invalid address+.

The true case: this happens for example when an active _create_ request (or 
create sub-request within the _multi_ request) is completed on call to 
_zookeeper_close()_ with the so called "Fake response" which is fabricated by 
the function _free_completions()_. Such response includes only the header but 
+zero bytes for the body+.

*I suspect this may happen not only due to call to _zookeeper_close()_ but on 
reception of a true response from the server* containing insufficient number of 
bytes (I'm not sure if it can be a proper response from the server with an 
error status and empty payload).

Also *I suspect the same case will take place with different requests, but not 
only the _create_*. Indeed, almost all cases in the _deserialize_response()_ 
shall be verified as soon as they also use uninitialized _res_-s and 
_deallocate_ them. Still I have not checked others except the _create_ request 
with _COMPLETION_STRING_ response.


> Local automatic variable is left uninitialized and then freed.
> --
>
> Key: ZOOKEEPER-2890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
> automatic variable *_struct CreateResponse res_* which is +left 
> uninitialized+ and passed to the function _deserialize_GetACLResponse()_ and 
> then to _deallocate_GetACLResponse()_.
> The 

[GitHub] zookeeper pull request #358: branch-3.4 -- bugfix -- ZOOKEEPER-2890

[xoiss]

GitHub user xoiss opened a pull request:

https://github.com/apache/zookeeper/pull/358

branch-3.4 -- bugfix -- ZOOKEEPER-2890

Fixes https://issues.apache.org/jira/browse/ZOOKEEPER-2890

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xoiss/zookeeper 
branch-3.4-bugfix-zookeeper-2890

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/358.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #358


commit a752eee7837d241d79151fa6053571df7a00761e
Author: Patrick D. Hunt 
Date:   2013-10-24T05:11:59Z

ZOOKEEPER-1744. clientPortAddress breaks "zkServer.sh status" (Nick Ohanian 
via phunt)

git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1535279 
13f79535-47bb-0310-9956-ffa450edef68

commit 3dae9c4770f77e28fa6bba07e58133f1530f381a
Author: Thawan Kooburat 
Date:   2013-10-24T18:52:31Z

ZOOKEEPER-1798. Fix race condition in testNormalObserverRun (thawan, fpj 
via thawan)

git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1535497 
13f79535-47bb-0310-9956-ffa450edef68

commit 2bdf4678c70ff7639761c459708922cb2309b078
Author: Patrick D. Hunt 
Date:   2013-11-12T06:30:06Z

ZOOKEEPER-1812. ZooInspector reconnection always fails if first connection 
fails (Benjamin Jaton via phunt)

git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1540960 
13f79535-47bb-0310-9956-ffa450edef68

commit ffd23df5f0f5038bee5c3b05f54f12a80f8098b1
Author: Camille Fournier 
Date:   2013-11-13T02:22:39Z

ZOOKEEPER-1666. Avoid Reverse DNS lookup if the hostname in 
  connection string is literal IP address. (George Cao via camille)


git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1541359 
13f79535-47bb-0310-9956-ffa450edef68

commit bf1814b9943588ac25b46aa158d994f5900b9202
Author: Thawan Kooburat 
Date:   2013-11-14T04:43:46Z

ZOOKEEPER-1798. Fix race condition in testNormalObserverRun (Part 2) 
(thawan, fpj via thawan)

git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1541814 
13f79535-47bb-0310-9956-ffa450edef68

commit 6503f12e144bc6b0652f7adf558bdc28c984e6c0
Author: Flavio Paiva Junqueira 
Date:   2013-11-15T18:12:07Z

ZOOKEEPER-1786. ZooKeeper data model documentation is incorrect
  (Niraj Tolia via fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1542356 
13f79535-47bb-0310-9956-ffa450edef68

commit a452d791cff92d65937142c3475a9894264376f6
Author: Flavio Paiva Junqueira 
Date:   2013-11-16T10:06:19Z

ZOOKEEPER-1808. Add version to FLE notifications for 3.4 branch (fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1542489 
13f79535-47bb-0310-9956-ffa450edef68

commit b770aeb0c8bcb28a9b653b1dc0edac623c75b509
Author: Flavio Paiva Junqueira 
Date:   2013-11-17T11:42:22Z

ZOOKEEPER-1597. Windows build failing (michim via fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1542710 
13f79535-47bb-0310-9956-ffa450edef68

commit 7f069c0e084a74f6de661267b828ff3fde30bd11
Author: Flavio Paiva Junqueira 
Date:   2013-11-23T18:21:38Z

ZOOKEEPER-1817. Fix don't care for b3.4 (fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1544858 
13f79535-47bb-0310-9956-ffa450edef68

commit c62f2ffa7b9fb026004f66d788ebf1e293d2fdf8
Author: Flavio Paiva Junqueira 
Date:   2013-11-26T23:43:29Z

ZOOKEEPER-1653. zookeeper fails to start because of inconsistent epoch 
(michim via fpj)



git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1545883 
13f79535-47bb-0310-9956-ffa450edef68

commit 605c8d1b0db0baa57582bb39229c80bfec832ae3
Author: Flavio Paiva Junqueira 
Date:   2013-11-27T23:23:11Z

ZOOKEEPER-1821. very ugly warning when compiling load_gen.c (german blanco 
via fpj)


git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1546231 
13f79535-47bb-0310-9956-ffa450edef68

commit a1cdea63de42ed1eaba224570ad0304c224b21c1
Author: Michi Mutsuzaki 
Date:   2013-12-04T04:20:53Z

ZOOKEEPER-1632. fix memory leaks in cli_st (fpj via michim)


git-svn-id: 
https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1547703 
13f79535-47bb-0310-9956-ffa450edef68

commit f23b85bde1bf53b284a85a8ec6c6251cd63e9feb
Author: Flavio Paiva Junqueira 
Date:   2013-12-07T10:17:54Z

ZOOKEEPER-1459. Standalone 

Failed: ZOOKEEPER- PreCommit Build #994

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/994/

###
## LAST 60 LINES OF THE CONSOLE 
###
GitHub pull request #358 to apache/zookeeper
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Archive the artifacts? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #994
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #994
[description-setter] Could not determine description.
Putting comment on the pull request
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Commented] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153532#comment-16153532
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2890:
---

Github user xoiss closed the pull request at:

https://github.com/apache/zookeeper/pull/358


> Local automatic variable is left uninitialized and then freed.
> --
>
> Key: ZOOKEEPER-2890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
> automatic variable *_struct CreateResponse res_* which is +left 
> uninitialized+ and passed to the function _deserialize_GetACLResponse()_ and 
> then to _deallocate_GetACLResponse()_.
> The _deserialize_ function, which is called the first, is expected to assign 
> the _res_ variable with a value from the parsed _struct iarchive *ia_. But, 
> if _ia_ contains for example insufficient amount of bytes the 
> _deserialize_String()_ function refuses of assigning a value to _res_, and 
> _res_ stays uninitialized (the true case is described below). Then, the 
> _deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
> with arguments. If incidentally the memory region in the program stack under 
> the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
> address+.
> The true case: this happens when an active _multi_ request with _create_ 
> sub-request is completed on call to _zookeeper_close()_ with the so called 
> "Fake response" which is fabricated by the function _free_completions()_. 
> Such response includes only the header but +zero bytes for the body+. The 
> significant condition is that the _create_ request is not a stand-alone one, 
> but namely a sub-request within the _multi_ request. In this case the 
> _deserialize_response()_ is called recursively (for each sub-request), and 
> when it is called for the _create_ subrequest (from the nested 
> _deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
> the _if (failed)_ condition branches to the _else_ part. Note that in the 
> stand-alone create-request case this does not occur.
> *I suspect this may happen not only due to call to _zookeeper_close()_ but on 
> reception of a true multi-response from the server* containing insufficient 
> number of bytes (I'm not sure if it can be a proper response from the server 
> with an error overall status and empty or insufficient payload).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153531#comment-16153531
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2890:
---

Github user xoiss commented on the issue:

https://github.com/apache/zookeeper/pull/358
  
Sorry, I wanna propose it not for the master branch -- closed.


> Local automatic variable is left uninitialized and then freed.
> --
>
> Key: ZOOKEEPER-2890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
> automatic variable *_struct CreateResponse res_* which is +left 
> uninitialized+ and passed to the function _deserialize_GetACLResponse()_ and 
> then to _deallocate_GetACLResponse()_.
> The _deserialize_ function, which is called the first, is expected to assign 
> the _res_ variable with a value from the parsed _struct iarchive *ia_. But, 
> if _ia_ contains for example insufficient amount of bytes the 
> _deserialize_String()_ function refuses of assigning a value to _res_, and 
> _res_ stays uninitialized (the true case is described below). Then, the 
> _deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
> with arguments. If incidentally the memory region in the program stack under 
> the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
> address+.
> The true case: this happens when an active _multi_ request with _create_ 
> sub-request is completed on call to _zookeeper_close()_ with the so called 
> "Fake response" which is fabricated by the function _free_completions()_. 
> Such response includes only the header but +zero bytes for the body+. The 
> significant condition is that the _create_ request is not a stand-alone one, 
> but namely a sub-request within the _multi_ request. In this case the 
> _deserialize_response()_ is called recursively (for each sub-request), and 
> when it is called for the _create_ subrequest (from the nested 
> _deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
> the _if (failed)_ condition branches to the _else_ part. Note that in the 
> stand-alone create-request case this does not occur.
> *I suspect this may happen not only due to call to _zookeeper_close()_ but on 
> reception of a true multi-response from the server* containing insufficient 
> number of bytes (I'm not sure if it can be a proper response from the server 
> with an error overall status and empty or insufficient payload).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153535#comment-16153535
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2890:
---

GitHub user xoiss opened a pull request:

https://github.com/apache/zookeeper/pull/359

branch-3.4 -- bugfix -- ZOOKEEPER-2890

Fixes https://issues.apache.org/jira/browse/ZOOKEEPER-2890

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xoiss/zookeeper 
branch-3.4-bugfix-zookeeper-2890

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/359.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #359


commit 2dc3664e52a0f1ae82c5c4fdc800921548bfc087
Author: xoiss 
Date:   2017-09-05T11:28:36Z

ZOOKEEPER-2890 - fix freing by uninitialized address.




> Local automatic variable is left uninitialized and then freed.
> --
>
> Key: ZOOKEEPER-2890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
> automatic variable *_struct CreateResponse res_* which is +left 
> uninitialized+ and passed to the function _deserialize_GetACLResponse()_ and 
> then to _deallocate_GetACLResponse()_.
> The _deserialize_ function, which is called the first, is expected to assign 
> the _res_ variable with a value from the parsed _struct iarchive *ia_. But, 
> if _ia_ contains for example insufficient amount of bytes the 
> _deserialize_String()_ function refuses of assigning a value to _res_, and 
> _res_ stays uninitialized (the true case is described below). Then, the 
> _deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
> with arguments. If incidentally the memory region in the program stack under 
> the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
> address+.
> The true case: this happens when an active _multi_ request with _create_ 
> sub-request is completed on call to _zookeeper_close()_ with the so called 
> "Fake response" which is fabricated by the function _free_completions()_. 
> Such response includes only the header but +zero bytes for the body+. The 
> significant condition is that the _create_ request is not a stand-alone one, 
> but namely a sub-request within the _multi_ request. In this case the 
> _deserialize_response()_ is called recursively (for each sub-request), and 
> when it is called for the _create_ subrequest (from the nested 
> _deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
> the _if (failed)_ condition branches to the _else_ part. Note that in the 
> stand-alone create-request case this does not occur.
> *I suspect this may happen not only due to call to _zookeeper_close()_ but on 
> reception of a true multi-response from the server* containing insufficient 
> number of bytes (I'm not sure if it can be a proper response from the server 
> with an error overall status and empty or insufficient payload).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Failed: ZOOKEEPER- PreCommit Build #995

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/995/

###
## LAST 60 LINES OF THE CONSOLE 
###
GitHub pull request #359 to apache/zookeeper
[EnvInject] - Loading node environment variables.
ERROR: SEVERE ERROR occurs
org.jenkinsci.lib.envinject.EnvInjectException: java.io.IOException: Remote 
call on H5 failed
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:86)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.setUpEnvironment(EnvInjectListener.java:43)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.createLauncher(AbstractBuild.java:528)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:448)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:405)
Caused by: java.io.IOException: Remote call on H5 failed
at hudson.remoting.Channel.call(Channel.java:838)
at hudson.FilePath.act(FilePath.java:1081)
at 
org.jenkinsci.plugins.envinject.service.EnvInjectActionSetter.addEnvVarsToRun(EnvInjectActionSetter.java:59)
at 
org.jenkinsci.plugins.envinject.EnvInjectListener.loadEnvironmentVariablesNode(EnvInjectListener.java:83)
... 7 more
Caused by: java.lang.OutOfMemoryError: Java heap space
ERROR: Step ?Archive the artifacts? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #995
ERROR: Step ?Publish JUnit test result report? failed: no workspace for 
PreCommit-ZOOKEEPER-github-pr-build #995
[description-setter] Could not determine description.
Putting comment on the pull request
[EnvInject] - [ERROR] - SEVERE ERROR occurs: Remote call on H5 failed
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
No tests ran.

[jira] [Updated] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2890:
-
Description: 
Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
automatic variable *_struct CreateResponse res_* which is +left uninitialized+ 
and passed to the function _deserialize_GetACLResponse()_ and then to 
_deallocate_GetACLResponse()_.

The _deserialize_ function, which is called the first, is expected to assign 
the _res_ variable with a value from the parsed _struct iarchive *ia_. But, if 
_ia_ contains for example insufficient amount of bytes the 
_deserialize_String()_ function refuses of assigning a value to _res_, and 
_res_ stays uninitialized (the true case is described below). Then, the 
_deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
with arguments. If incidentally the memory region in the program stack under 
the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
address+.

The true case: this happens when an active _multi_ request with _create_ 
sub-request is completed on call to _zookeeper_close()_ with the so called 
"Fake response" which is fabricated by the function _free_completions()_. Such 
response includes only the header but +zero bytes for the body+. The 
significant condition is that the _create_ request is not a stand-alone one, 
but namely a sub-request within the _multi_ request. In this case the 
_deserialize_response()_ is called recursively (for each sub-request), and when 
it is called for the _create_ subrequest (from the nested 
_deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
the _if (failed)_ condition branches to the _else_ part. Note that in the 
stand-alone create-request case this does not occur.

*I suspect this may happen not only due to call to _zookeeper_close()_ but on 
reception of a true multi-response from the server* containing insufficient 
number of bytes (I'm not sure if it can be a proper response from the server 
with an error overall status and empty or insufficient payload).

This is a proposed fix: https://github.com/apache/zookeeper/pull/359

  was:
Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
automatic variable *_struct CreateResponse res_* which is +left uninitialized+ 
and passed to the function _deserialize_GetACLResponse()_ and then to 
_deallocate_GetACLResponse()_.

The _deserialize_ function, which is called the first, is expected to assign 
the _res_ variable with a value from the parsed _struct iarchive *ia_. But, if 
_ia_ contains for example insufficient amount of bytes the 
_deserialize_String()_ function refuses of assigning a value to _res_, and 
_res_ stays uninitialized (the true case is described below). Then, the 
_deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
with arguments. If incidentally the memory region in the program stack under 
the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
address+.

The true case: this happens when an active _multi_ request with _create_ 
sub-request is completed on call to _zookeeper_close()_ with the so called 
"Fake response" which is fabricated by the function _free_completions()_. Such 
response includes only the header but +zero bytes for the body+. The 
significant condition is that the _create_ request is not a stand-alone one, 
but namely a sub-request within the _multi_ request. In this case the 
_deserialize_response()_ is called recursively (for each sub-request), and when 
it is called for the _create_ subrequest (from the nested 
_deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
the _if (failed)_ condition branches to the _else_ part. Note that in the 
stand-alone create-request case this does not occur.

*I suspect this may happen not only due to call to _zookeeper_close()_ but on 
reception of a true multi-response from the server* containing insufficient 
number of bytes (I'm not sure if it can be a proper response from the server 
with an error overall status and empty or insufficient payload).


> Local automatic variable is left uninitialized and then freed.
> --
>
> Key: ZOOKEEPER-2890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_response()_*, in _case COMPLETION_STRING_, 

[jira] [Created] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[Alexander A. Strelets (JIRA)]

Alexander A. Strelets created ZOOKEEPER-2890:


 Summary: Local automatic variable is left uninitialized and then 
freed.
 Key: ZOOKEEPER-2890
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
 Project: ZooKeeper
  Issue Type: Bug
  Components: c client
Affects Versions: 3.4.10
 Environment: Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4
Reporter: Alexander A. Strelets
Priority: Critical
 Fix For: 3.4.10


Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
automatic variable *_struct CreateResponse res_* which is +left uninitialized+ 
and passed to the function _deserialize_GetACLResponse()_ and then to 
_deallocate_GetACLResponse()_.

The _deserialize_ function, which is called the first, is expected to assign 
the _res_ variable with a value from the parsed _struct iarchive *ia_. But, if 
_ia_ contains for example insufficient amount of bytes the 
_deserialize_String()_ function refuses of assigning a value to _res_, and 
_res_ stays uninitialized (the true case is described below). Then, the 
_deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
with arguments. If incidentally the memory region in the program stack under 
the _res_ was not equal to NULL, the last call +leads to _free()_ for an 
invalid address+.

The true case: this happens for example when an active _create_ request (or 
create sub-request within the _multi_ request) is completed on call to 
_zookeeper_close()_ with the so called "Fake response" which is fabricated by 
the function _free_completions()_. Such response includes only the header but 
+zero bytes for the body+.

*I suspect this may happen not only due to call to _zookeeper_close()_ but on 
reception of a true response from the server* containing insufficient number of 
bytes (I'm not sure if it can be a proper response from the server with an 
error status and empty payload).

Also *I suspect the same case will take place with different requests, but not 
only the _create_*. Indeed, almost all cases in the _deserialize_response()_ 
shall be verified as soon as they also use uninitialized _res_-s and 
_deallocate_ them. Still I have not checked others except the _create_ request 
with _COMPLETION_STRING_ response.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ZOOKEEPER-2891) SIGABRT from assert during fake completion on zookeeper_close.

[Alexander A. Strelets (JIRA)]

Alexander A. Strelets created ZOOKEEPER-2891:


 Summary: SIGABRT from assert during fake completion on 
zookeeper_close.
 Key: ZOOKEEPER-2891
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2891
 Project: ZooKeeper
  Issue Type: Bug
  Components: c client
Affects Versions: 3.4.10
 Environment: Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4
Reporter: Alexander A. Strelets
Priority: Critical
 Fix For: 3.4.10


Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the so 
called "Fake response" which is fabricated by the function _free_completions()_ 
for example when _zookeeper_close()_ is called while there is a pending _multi_ 
request.

Such fake response includes only the header but zero bytes for the body. Due to 
this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is called 
repeatedly for each {{completion_list_t *entry = dequeue_completion(clist)}}, 
does not assign the _mhdr_ and keeps _mhdr.done == 0_ as it was originally 
initialized. Consequently the _while (!mhdr.done)_ does not ever end, and 
finally falls into the _assert(entry)_ with _entry == NULL_ when all 
sub-requests are "completed".

I propose to instruct the _deserialize_multi()_ function to break the loop on 
_entry == NULL_ if it was called for an unsuccessfull overal status of the 
multi response, and in particular for the fake response having _ZCLOSING_ 
(-116) status. I have introduced the _rc0_ parameter for this.


*Another issue* with this function is that even if the while-loop exited 
properly, this function returns _rc == 0_, and this return code +overrides+ the 
true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in the 
_deserialize_response()_ function. So, the _multi_ response callback +handler 
would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which is strictly 
wrong.

To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
zero (which is _ZOK_ indeed).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] zookeeper pull request #360: branch-3.4 -- bugfix -- ZOOKEEPER-2891

[xoiss]

GitHub user xoiss opened a pull request:

https://github.com/apache/zookeeper/pull/360

branch-3.4 -- bugfix -- ZOOKEEPER-2891

Fixes https://issues.apache.org/jira/browse/ZOOKEEPER-2891

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xoiss/zookeeper 
branch-3.4-bugfix-zookeeper-2891

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/360.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #360


commit 2dc3664e52a0f1ae82c5c4fdc800921548bfc087
Author: xoiss 
Date:   2017-09-05T11:28:36Z

ZOOKEEPER-2890 - fix freing by uninitialized address.

commit b6da551a38bcfe834038c44f94da0bbfb2c881a5
Author: xoiss 
Date:   2017-09-05T12:48:33Z

ZOOKEEPER-2891 - fix endless loop and assertion on fake multi response.




---

[jira] [Updated] (ZOOKEEPER-2891) SIGABRT from assert during fake completion on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2891:
-
Description: 
Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the so 
called "Fake response" which is fabricated by the function _free_completions()_ 
for example when _zookeeper_close()_ is called while there is a pending _multi_ 
request.

Such fake response includes only the header but zero bytes for the body. Due to 
this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is called 
repeatedly for each {{completion_list_t *entry = dequeue_completion(clist)}}, 
does not assign the _mhdr_ and keeps _mhdr.done == 0_ as it was originally 
initialized. Consequently the _while (!mhdr.done)_ does not ever end, and 
finally falls into the _assert(entry)_ with _entry == NULL_ when all 
sub-requests are "completed". ~// Normally on my platform assert raises 
SIGABRT.~

I propose to instruct the _deserialize_multi()_ function to break the loop on 
_entry == NULL_ if it was called for an unsuccessfull overal status of the 
multi response, and in particular for the fake response having _ZCLOSING_ 
(-116) status. I have introduced the _rc0_ parameter for this.


*Another issue* with this function is that even if the while-loop exited 
properly, this function returns _rc == 0_, and this return code +overrides+ the 
true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in the 
_deserialize_response()_ function. So, the _multi_ response callback +handler 
would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which is strictly 
wrong.

To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
zero (which is _ZOK_ indeed).

This is a proposed fix: https://github.com/apache/zookeeper/pull/360

  was:
Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the so 
called "Fake response" which is fabricated by the function _free_completions()_ 
for example when _zookeeper_close()_ is called while there is a pending _multi_ 
request.

Such fake response includes only the header but zero bytes for the body. Due to 
this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is called 
repeatedly for each {{completion_list_t *entry = dequeue_completion(clist)}}, 
does not assign the _mhdr_ and keeps _mhdr.done == 0_ as it was originally 
initialized. Consequently the _while (!mhdr.done)_ does not ever end, and 
finally falls into the _assert(entry)_ with _entry == NULL_ when all 
sub-requests are "completed". ~// Normally on my platform assert raises 
SIGABRT.~

I propose to instruct the _deserialize_multi()_ function to break the loop on 
_entry == NULL_ if it was called for an unsuccessfull overal status of the 
multi response, and in particular for the fake response having _ZCLOSING_ 
(-116) status. I have introduced the _rc0_ parameter for this.


*Another issue* with this function is that even if the while-loop exited 
properly, this function returns _rc == 0_, and this return code +overrides+ the 
true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in the 
_deserialize_response()_ function. So, the _multi_ response callback +handler 
would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which is strictly 
wrong.

To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
zero (which is _ZOK_ indeed).


> SIGABRT from assert during fake completion on zookeeper_close.
> --
>
> Key: ZOOKEEPER-2891
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2891
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the 
> so called "Fake response" which is fabricated by the function 
> _free_completions()_ for example when _zookeeper_close()_ is called while 
> there is a pending _multi_ request.
> Such fake response includes only the header but zero bytes for the body. Due 
> to this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is 
> called repeatedly for each {{completion_list_t *entry = 
> dequeue_completion(clist)}}, does not assign the _mhdr_ and keeps _mhdr.done 
> == 0_ as it was originally initialized. Consequently the _while (!mhdr.done)_ 
> does not ever end, and finally falls into the _assert(entry)_ with _entry == 
> NULL_ when all sub-requests are "completed". ~// Normally on my platform 
> assert raises SIGABRT.~
> I 

[jira] [Updated] (ZOOKEEPER-2891) SIGABRT from assert during fake completion on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2891:
-
Description: 
Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the so 
called "Fake response" which is fabricated by the function _free_completions()_ 
for example when _zookeeper_close()_ is called while there is a pending _multi_ 
request.

Such fake response includes only the header but zero bytes for the body. Due to 
this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is called 
repeatedly for each {{completion_list_t *entry = dequeue_completion(clist)}}, 
does not assign the _mhdr_ and keeps _mhdr.done == 0_ as it was originally 
initialized. Consequently the _while (!mhdr.done)_ does not ever end, and 
finally falls into the _assert(entry)_ with _entry == NULL_ when all 
sub-requests are "completed". ~// Normally on my platform assert raises 
SIGABRT.~

I propose to instruct the _deserialize_multi()_ function to break the loop on 
_entry == NULL_ if it was called for an unsuccessfull overal status of the 
multi response, and in particular for the fake response having _ZCLOSING_ 
(-116) status. I have introduced the _rc0_ parameter for this.


*Another issue* with this function is that even if the while-loop exited 
properly, this function returns _rc == 0_, and this return code +overrides+ the 
true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in the 
_deserialize_response()_ function. So, the _multi_ response callback +handler 
would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which is strictly 
wrong.

To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
zero (which is _ZOK_ indeed).

  was:
Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the so 
called "Fake response" which is fabricated by the function _free_completions()_ 
for example when _zookeeper_close()_ is called while there is a pending _multi_ 
request.

Such fake response includes only the header but zero bytes for the body. Due to 
this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is called 
repeatedly for each {{completion_list_t *entry = dequeue_completion(clist)}}, 
does not assign the _mhdr_ and keeps _mhdr.done == 0_ as it was originally 
initialized. Consequently the _while (!mhdr.done)_ does not ever end, and 
finally falls into the _assert(entry)_ with _entry == NULL_ when all 
sub-requests are "completed".

I propose to instruct the _deserialize_multi()_ function to break the loop on 
_entry == NULL_ if it was called for an unsuccessfull overal status of the 
multi response, and in particular for the fake response having _ZCLOSING_ 
(-116) status. I have introduced the _rc0_ parameter for this.


*Another issue* with this function is that even if the while-loop exited 
properly, this function returns _rc == 0_, and this return code +overrides+ the 
true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in the 
_deserialize_response()_ function. So, the _multi_ response callback +handler 
would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which is strictly 
wrong.

To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
zero (which is _ZOK_ indeed).


> SIGABRT from assert during fake completion on zookeeper_close.
> --
>
> Key: ZOOKEEPER-2891
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2891
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the 
> so called "Fake response" which is fabricated by the function 
> _free_completions()_ for example when _zookeeper_close()_ is called while 
> there is a pending _multi_ request.
> Such fake response includes only the header but zero bytes for the body. Due 
> to this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is 
> called repeatedly for each {{completion_list_t *entry = 
> dequeue_completion(clist)}}, does not assign the _mhdr_ and keeps _mhdr.done 
> == 0_ as it was originally initialized. Consequently the _while (!mhdr.done)_ 
> does not ever end, and finally falls into the _assert(entry)_ with _entry == 
> NULL_ when all sub-requests are "completed". ~// Normally on my platform 
> assert raises SIGABRT.~
> I propose to instruct the _deserialize_multi()_ function to break the loop on 
> _entry == NULL_ if it was called for an 

[jira] [Commented] (ZOOKEEPER-2891) SIGABRT from assert during fake completion on zookeeper_close.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153585#comment-16153585
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2891:
---

GitHub user xoiss opened a pull request:

https://github.com/apache/zookeeper/pull/360

branch-3.4 -- bugfix -- ZOOKEEPER-2891

Fixes https://issues.apache.org/jira/browse/ZOOKEEPER-2891

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xoiss/zookeeper 
branch-3.4-bugfix-zookeeper-2891

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/360.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #360


commit 2dc3664e52a0f1ae82c5c4fdc800921548bfc087
Author: xoiss 
Date:   2017-09-05T11:28:36Z

ZOOKEEPER-2890 - fix freing by uninitialized address.

commit b6da551a38bcfe834038c44f94da0bbfb2c881a5
Author: xoiss 
Date:   2017-09-05T12:48:33Z

ZOOKEEPER-2891 - fix endless loop and assertion on fake multi response.




> SIGABRT from assert during fake completion on zookeeper_close.
> --
>
> Key: ZOOKEEPER-2891
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2891
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the 
> so called "Fake response" which is fabricated by the function 
> _free_completions()_ for example when _zookeeper_close()_ is called while 
> there is a pending _multi_ request.
> Such fake response includes only the header but zero bytes for the body. Due 
> to this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is 
> called repeatedly for each {{completion_list_t *entry = 
> dequeue_completion(clist)}}, does not assign the _mhdr_ and keeps _mhdr.done 
> == 0_ as it was originally initialized. Consequently the _while (!mhdr.done)_ 
> does not ever end, and finally falls into the _assert(entry)_ with _entry == 
> NULL_ when all sub-requests are "completed". ~// Normally on my platform 
> assert raises SIGABRT.~
> I propose to instruct the _deserialize_multi()_ function to break the loop on 
> _entry == NULL_ if it was called for an unsuccessfull overal status of the 
> multi response, and in particular for the fake response having _ZCLOSING_ 
> (-116) status. I have introduced the _rc0_ parameter for this.
> *Another issue* with this function is that even if the while-loop exited 
> properly, this function returns _rc == 0_, and this return code +overrides+ 
> the true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in 
> the _deserialize_response()_ function. So, the _multi_ response callback 
> +handler would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which 
> is strictly wrong.
> To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
> zero (which is _ZOK_ indeed).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ZOOKEEPER-2845) Data inconsistency issue due to retain database in leader election

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153628#comment-16153628
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2845:
---

Github user revans2 commented on the issue:

https://github.com/apache/zookeeper/pull/310
  
@lvfangmin any update on getting a pull request for the actual fix?


> Data inconsistency issue due to retain database in leader election
> --
>
> Key: ZOOKEEPER-2845
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2845
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: quorum
>Affects Versions: 3.4.10, 3.5.3, 3.6.0
>Reporter: Fangmin Lv
>Assignee: Fangmin Lv
>Priority: Critical
>
> In ZOOKEEPER-2678, the ZKDatabase is retained to reduce the unavailable time 
> during leader election. In ZooKeeper ensemble, it's possible that the 
> snapshot is ahead of txn file (due to slow disk on the server, etc), or the 
> txn file is ahead of snapshot due to no commit message being received yet. 
> If snapshot is ahead of txn file, since the SyncRequestProcessor queue will 
> be drained during shutdown, the snapshot and txn file will keep consistent 
> before leader election happening, so this is not an issue.
> But if txn is ahead of snapshot, it's possible that the ensemble will have 
> data inconsistent issue, here is the simplified scenario to show the issue:
> Let's say we have a 3 servers in the ensemble, server A and B are followers, 
> and C is leader, and all the snapshot and txn are up to T0:
> 1. A new request reached to leader C to create Node N, and it's converted to 
> txn T1 
> 2. Txn T1 was synced to disk in C, but just before the proposal reaching out 
> to the followers, A and B restarted, so the T1 didn't exist in A and B
> 3. A and B formed a new quorum after restart, let's say B is the leader
> 4. C changed to looking state due to no enough followers, it will sync with 
> leader B with last Zxid T0, which will have an empty diff sync
> 5. Before C take snapshot it restarted, it replayed the txns on disk which 
> includes T1, now it will have Node N, but A and B doesn't have it.
> Also I included the a test case to reproduce this issue consistently. 
> We have a totally different RetainDB version which will avoid this issue by 
> doing consensus between snapshot and txn files before leader election, will 
> submit for review.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] zookeeper issue #310: [ZOOKEEPER-2845][Test] Test used to reproduce the data...

[revans2]

Github user revans2 commented on the issue:

https://github.com/apache/zookeeper/pull/310
  
@lvfangmin any update on getting a pull request for the actual fix?


---

[jira] [Commented] (ZOOKEEPER-2891) SIGABRT from assert during fake completion on zookeeper_close.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153637#comment-16153637
 ] 

Hadoop QA commented on ZOOKEEPER-2891:
--

+1 overall.  GitHub Pull Request  Build
  

+1 @author.  The patch does not contain any @author tags.

+0 tests included.  The patch appears to be a documentation patch that 
doesn't require tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 3.0.1) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/996//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/996//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/996//console

This message is automatically generated.

> SIGABRT from assert during fake completion on zookeeper_close.
> --
>
> Key: ZOOKEEPER-2891
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2891
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the 
> so called "Fake response" which is fabricated by the function 
> _free_completions()_ for example when _zookeeper_close()_ is called while 
> there is a pending _multi_ request.
> Such fake response includes only the header but zero bytes for the body. Due 
> to this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is 
> called repeatedly for each {{completion_list_t *entry = 
> dequeue_completion(clist)}}, does not assign the _mhdr_ and keeps _mhdr.done 
> == 0_ as it was originally initialized. Consequently the _while (!mhdr.done)_ 
> does not ever end, and finally falls into the _assert(entry)_ with _entry == 
> NULL_ when all sub-requests are "completed". ~// Normally on my platform 
> assert raises SIGABRT.~
> I propose to instruct the _deserialize_multi()_ function to break the loop on 
> _entry == NULL_ if it was called for an unsuccessfull overal status of the 
> multi response, and in particular for the fake response having _ZCLOSING_ 
> (-116) status. I have introduced the _rc0_ parameter for this.
> *Another issue* with this function is that even if the while-loop exited 
> properly, this function returns _rc == 0_, and this return code +overrides+ 
> the true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in 
> the _deserialize_response()_ function. So, the _multi_ response callback 
> +handler would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which 
> is strictly wrong.
> To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
> zero (which is _ZOK_ indeed).
> This is a proposed fix: https://github.com/apache/zookeeper/pull/360



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] zookeeper pull request #356: ZOOKEEPER-2572: Fix potential resource leak in ...

[maoling]

Github user maoling commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/356#discussion_r137003935
  
--- Diff: 
src/java/main/org/apache/zookeeper/server/persistence/FileTxnLog.java ---
@@ -409,13 +409,15 @@ public void truncate(long zxid) throws IOException {
 try {
 Files.delete(itr.logFile.toPath());
 } catch (NoSuchFileException e) {
+LOG.info("An NoSuchFileException was thrown when 
delete file {}" +
+", but will continue. Assume this file has been 
deleted successfully.", itr.logFile);
 }
 }
 } finally {
-close(itr);
 if (raf != null) {
 raf.close();
--- End diff --

IMHO, `raf.close();`  should be surrounded with `try-catch` ?


---

[jira] [Commented] (ZOOKEEPER-2572) Potential resource leak in FileTxnLog.truncate

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153734#comment-16153734
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2572:
---

Github user maoling commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/356#discussion_r137003935
  
--- Diff: 
src/java/main/org/apache/zookeeper/server/persistence/FileTxnLog.java ---
@@ -409,13 +409,15 @@ public void truncate(long zxid) throws IOException {
 try {
 Files.delete(itr.logFile.toPath());
 } catch (NoSuchFileException e) {
+LOG.info("An NoSuchFileException was thrown when 
delete file {}" +
+", but will continue. Assume this file has been 
deleted successfully.", itr.logFile);
 }
 }
 } finally {
-close(itr);
 if (raf != null) {
 raf.close();
--- End diff --

IMHO, `raf.close();`  should be surrounded with `try-catch` ?


> Potential resource leak in FileTxnLog.truncate
> --
>
> Key: ZOOKEEPER-2572
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2572
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.9, 3.5.2
>Reporter: Michael Han
>Assignee: gaoshu
> Fix For: 3.5.4, 3.6.0, 3.4.11
>
>
> In FileTxnLog.truncate, we have:
> {code}
> public boolean truncate(long zxid) throws IOException {
> FileTxnIterator itr = null;
> try {
> itr = new FileTxnIterator(this.logDir, zxid);
> PositionInputStream input = itr.inputStream;
> if(input == null) {
> throw new IOException("No log files found to truncate! This 
> could " +
> "happen if you still have snapshots from an old setup 
> or " +
> "log files were deleted accidentally or dataLogDir 
> was changed in zoo.cfg.");
> }
> long pos = input.getPosition();
> // now, truncate at the current position
> RandomAccessFile raf=new RandomAccessFile(itr.logFile,"rw");
> raf.setLength(pos);
> raf.close();
> while(itr.goToNextLog()) {
> if (!itr.logFile.delete()) {
> LOG.warn("Unable to truncate {}", itr.logFile);
> }
> }
> } finally {
> close(itr);
> }
> return true;
> }
> {code}
> {{raf}} here can be potentially in a state of not closed after leaving the 
> method, if there is an (IO) exception thrown from setLength.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ZOOKEEPER-2809) Unnecessary stack-trace in server when the client disconnect unexpectedly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153740#comment-16153740
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2809:
---

Github user mfenes commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/355#discussion_r137005402
  
--- Diff: src/java/main/org/apache/zookeeper/server/NIOServerCnxn.java ---
@@ -374,14 +373,12 @@ void doIO(SelectionKey k) throws InterruptedException 
{
 // expecting close to log session closure
 close();
 } catch (EndOfStreamException e) {
-LOG.warn("caught end of stream exception",e); // tell user why
-
+LOG.warn(e.getMessage());
--- End diff --

Yes, it would make sense to log the stack trace at debug level for 
EndOfStreamException too, so that we don't get less information in the log 
after this change is applied. The reason why I did not include it first, was to 
keep this change a pure backport from master and branch-3.5 to branch-3.4 as 
the original change does not include this additional logging statement. I've 
added it now, but probably then we should also add it to master as well if we 
wish to keep it.


> Unnecessary stack-trace in server when the client disconnect unexpectedly
> -
>
> Key: ZOOKEEPER-2809
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2809
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: server
>Affects Versions: 3.4.8
>Reporter: Paul Millar
>Assignee: Mark Fenes
>Priority: Minor
> Fix For: 3.5.0
>
>
> In ZK 3.4.x, if the client disconnects unexpectedly then the server logs this 
> with a stack-trace (see 
> src/java/main/org/apache/zookeeper/server/NIOServerCnxn.java:356).
> This is unfortunate as we are using an embedded ZK server in our project (in 
> a test environment) and we consider all stack-traces as bugs.
> I noticed that ZK 3.5 and later no longer log a stack-trace.  This change is 
> due to commit 6206b495 (in branch-3.5), which adds ZOOKEEPER-1504 and seems 
> to fix this issue almost as a side-effect; a similar change in master has the 
> same effect.
> I was wondering if the change in how EndOfStreamException is logged (i.e., 
> logging the message without a stack-trace) could be back-ported to 3.4 
> branch, so could be included in the next 3.4 release.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] zookeeper pull request #355: ZOOKEEPER-2809: Unnecessary stack-trace in serv...

[mfenes]

Github user mfenes commented on a diff in the pull request:

https://github.com/apache/zookeeper/pull/355#discussion_r137005402
  
--- Diff: src/java/main/org/apache/zookeeper/server/NIOServerCnxn.java ---
@@ -374,14 +373,12 @@ void doIO(SelectionKey k) throws InterruptedException 
{
 // expecting close to log session closure
 close();
 } catch (EndOfStreamException e) {
-LOG.warn("caught end of stream exception",e); // tell user why
-
+LOG.warn(e.getMessage());
--- End diff --

Yes, it would make sense to log the stack trace at debug level for 
EndOfStreamException too, so that we don't get less information in the log 
after this change is applied. The reason why I did not include it first, was to 
keep this change a pure backport from master and branch-3.5 to branch-3.4 as 
the original change does not include this additional logging statement. I've 
added it now, but probably then we should also add it to master as well if we 
wish to keep it.


---