ZooKeeper-trunk-WinVS2008 - Build # 2483 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper-trunk-WinVS2008/2483/

###
## LAST 60 LINES OF THE CONSOLE 
###
Started by timer
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
[EnvInject] - Loading node environment variables.
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
Building remotely on windows-2012-2 (Windows) in workspace 
f:\jenkins\jenkins-slave\workspace\ZooKeeper-trunk-WinVS2008
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url git://git.apache.org/zookeeper.git # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
Fetching upstream changes from git://git.apache.org/zookeeper.git
 > git --version # timeout=10
 > git fetch --tags --progress git://git.apache.org/zookeeper.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse "refs/remotes/origin/master^{commit}" # timeout=10
 > git rev-parse "refs/remotes/origin/origin/master^{commit}" # timeout=10
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
Checking out Revision 86577c9d81bd40067bc084e49a7172884ac9ae49 
(refs/remotes/origin/master)
Commit message: "ZOOKEEPER-2880: Rename README.txt to README.md"
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 86577c9d81bd40067bc084e49a7172884ac9ae49
 > git rev-list 86577c9d81bd40067bc084e49a7172884ac9ae49 # timeout=10
No emails were triggered.
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
[ZooKeeper-trunk-WinVS2008] $ cmd.exe /C 
"F:\jenkins\tools\ant\latest\bin\ant.bat -Dtest.output=yes 
-Dtest.junit.output.format=xml clean compile_jute && exit %%ERRORLEVEL%%"
java.lang.UnsupportedClassVersionError: org/apache/tools/ant/launch/Launcher : 
Unsupported major.minor version 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
Exception in thread "main" Build step 'Invoke Ant' marked build as failure
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found
No JDK named ‘JDK 1.8 (unlimited security) 64-bit Windows only’ found



###
## FAILED TESTS (if any) 
##
No tests ran.

ZooKeeper_branch34_jdk7 - Build # 1646 - Failure

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch34_jdk7/1646/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 31.11 MB...]
[junit] 2017-09-09 02:43:03,501 [myid:] - INFO  
[main:PrepRequestProcessor@769] - Shutting down
[junit] 2017-09-09 02:43:03,501 [myid:] - INFO  
[main:SyncRequestProcessor@208] - Shutting down
[junit] 2017-09-09 02:43:03,502 [myid:] - INFO  [ProcessThread(sid:0 
cport:11221)::PrepRequestProcessor@144] - PrepRequestProcessor exited loop!
[junit] 2017-09-09 02:43:03,502 [myid:] - INFO  
[SyncThread:0:SyncRequestProcessor@186] - SyncRequestProcessor exited!
[junit] 2017-09-09 02:43:03,503 [myid:] - INFO  
[main:FinalRequestProcessor@403] - shutdown of request processor complete
[junit] 2017-09-09 02:43:03,503 [myid:] - INFO  
[main:FourLetterWordMain@65] - connecting to 127.0.0.1 11221
[junit] 2017-09-09 02:43:03,504 [myid:] - INFO  [main:JMXEnv@147] - 
ensureOnly:[]
[junit] 2017-09-09 02:43:03,506 [myid:] - INFO  [main:ClientBase@489] - 
STARTING server
[junit] 2017-09-09 02:43:03,506 [myid:] - INFO  [main:ClientBase@410] - 
CREATING server instance 127.0.0.1:11221
[junit] 2017-09-09 02:43:03,507 [myid:] - INFO  
[main:ServerCnxnFactory@116] - Using 
org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
[junit] 2017-09-09 02:43:03,507 [myid:] - INFO  
[main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11221
[junit] 2017-09-09 02:43:03,507 [myid:] - INFO  [main:ClientBase@385] - 
STARTING server instance 127.0.0.1:11221
[junit] 2017-09-09 02:43:03,508 [myid:] - INFO  [main:ZooKeeperServer@173] 
- Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 
6 datadir 
/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/build/test/tmp/test8111936303576622803.junit.dir/version-2
 snapdir 
/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/build/test/tmp/test8111936303576622803.junit.dir/version-2
[junit] 2017-09-09 02:43:03,512 [myid:] - ERROR [main:ZooKeeperServer@468] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR or SHUTDOWN server state changes
[junit] 2017-09-09 02:43:03,513 [myid:] - INFO  
[main:FourLetterWordMain@65] - connecting to 127.0.0.1 11221
[junit] 2017-09-09 02:43:03,513 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@215] - 
Accepted socket connection from /127.0.0.1:40258
[junit] 2017-09-09 02:43:03,514 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxn@892] - Processing 
stat command from /127.0.0.1:40258
[junit] 2017-09-09 02:43:03,515 [myid:] - INFO  
[Thread-4:NIOServerCnxn$StatCommand@683] - Stat command output
[junit] 2017-09-09 02:43:03,515 [myid:] - INFO  
[Thread-4:NIOServerCnxn@1040] - Closed socket connection for client 
/127.0.0.1:40258 (no session established for client)
[junit] 2017-09-09 02:43:03,515 [myid:] - INFO  [main:JMXEnv@230] - 
ensureParent:[InMemoryDataTree, StandaloneServer_port]
[junit] 2017-09-09 02:43:03,518 [myid:] - INFO  [main:JMXEnv@247] - 
expect:InMemoryDataTree
[junit] 2017-09-09 02:43:03,518 [myid:] - INFO  [main:JMXEnv@251] - 
found:InMemoryDataTree 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221,name1=InMemoryDataTree
[junit] 2017-09-09 02:43:03,518 [myid:] - INFO  [main:JMXEnv@247] - 
expect:StandaloneServer_port
[junit] 2017-09-09 02:43:03,518 [myid:] - INFO  [main:JMXEnv@251] - 
found:StandaloneServer_port 
org.apache.ZooKeeperService:name0=StandaloneServer_port11221
[junit] 2017-09-09 02:43:03,519 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@58] - Memory used 35693
[junit] 2017-09-09 02:43:03,519 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@63] - Number of threads 20
[junit] 2017-09-09 02:43:03,519 [myid:] - INFO  
[main:JUnit4ZKTestRunner$LoggedInvokeMethod@78] - FINISHED TEST METHOD testQuota
[junit] 2017-09-09 02:43:03,520 [myid:] - INFO  [main:ClientBase@566] - 
tearDown starting
[junit] 2017-09-09 02:43:03,580 [myid:] - INFO  [main:ZooKeeper@687] - 
Session: 0x1068f4f988d closed
[junit] 2017-09-09 02:43:03,580 [myid:] - INFO  [main:ClientBase@536] - 
STOPPING server
[junit] 2017-09-09 02:43:03,580 [myid:] - INFO  
[main-EventThread:ClientCnxn$EventThread@520] - EventThread shut down for 
session: 0x1068f4f988d
[junit] 2017-09-09 02:43:03,580 [myid:] - INFO  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11221:NIOServerCnxnFactory@242] - 
NIOServerCnxn factory exited run method
[junit] 2017-09-09 02:43:03,581 [myid:] - INFO  [main:ZooKeeperServer@501] 
- shutting down
[junit] 2017-09-09 02:43:03,581 [myid:] - ERROR [main:ZooKeeperServer@468] 
- ZKShutdownHandler is not registered, so ZooKeeper server won't take any 
action on ERROR 

Failed: ZOOKEEPER- PreCommit Build #1004

[Apache Jenkins Server]

Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1004/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 35.03 MB...]
 [exec] +0 tests included.  The patch appears to be a documentation 
patch that doesn't require tests.
 [exec] 
 [exec] +1 javadoc.  The javadoc tool did not generate any warning 
messages.
 [exec] 
 [exec] +1 javac.  The applied patch does not increase the total number 
of javac compiler warnings.
 [exec] 
 [exec] +1 findbugs.  The patch does not introduce any new Findbugs 
(version 3.0.1) warnings.
 [exec] 
 [exec] +1 release audit.  The applied patch does not increase the 
total number of release audit warnings.
 [exec] 
 [exec] -1 core tests.  The patch failed core unit tests.
 [exec] 
 [exec] +1 contrib tests.  The patch passed contrib unit tests.
 [exec] 
 [exec] Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1004//testReport/
 [exec] Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1004//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
 [exec] Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1004//console
 [exec] 
 [exec] This message is automatically generated.
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Adding comment to Jira.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] Comment added.
 [exec] 996e13c72c5a18eee377ece594f74a662b3e8bc7 logged out
 [exec] 
 [exec] 
 [exec] 
==
 [exec] 
==
 [exec] Finished build.
 [exec] 
==
 [exec] 
==
 [exec] 
 [exec] 
 [exec] mv: 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 and 
'/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/patchprocess'
 are the same file

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/PreCommit-ZOOKEEPER-github-pr-build/build.xml:1706:
 exec returned: 1

Total time: 43 minutes 12 seconds
Build step 'Execute shell' marked build as failure
Archiving artifacts
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Recording test results
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
[description-setter] Description set: ZOOKEEPER-2894
Putting comment on the pull request
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7
Setting JDK_1_7_LATEST__HOME=/home/jenkins/tools/java/latest1.7



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Commented] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158655#comment-16158655
 ] 

Hadoop QA commented on ZOOKEEPER-2894:
--

-1 overall.  GitHub Pull Request  Build
  

+1 @author.  The patch does not contain any @author tags.

+0 tests included.  The patch appears to be a documentation patch that 
doesn't require tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 3.0.1) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

-1 core tests.  The patch failed core unit tests.

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1004//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1004//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: 
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/1004//console

This message is automatically generated.

> Memory and completions leak on zookeeper_close.
> ---
>
> Key: ZOOKEEPER-2894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2894
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
> Attachments: zk-will-free-zh-and-lose-completions.png
>
>
> ZooKeeper C Client *+single thread+* build
> First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
> two ways:
> a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
> called through _zookeeper_process()_
> b) and from other places -- i.e., when the call-stack does not pass through 
> any of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_
> The issue described here below is +specific only to the case (b)+. So, it's 
> Ok with the case (a).
> When _zookeeper_close()_ is called in the (b) way, the following happens:
> 1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
> they all are removed from this queue and each of them is "completed" with 
> personal fake response having status ZCLOSING. Such fake responses are put 
> into _zh.completions_to_process_ queue. It's Ok
> 2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
> completion callbacks are called, nor dynamic memory allocated for fake 
> responses is freed+
> 3. Different structures within _zh_ are dismissed and finally _zh_ is freed
> This is illustrated on the screenshot attached to this ticket: you may see 
> that the next instruction to execute will be _free(zh)_ while 
> _zh.completions_to_process_ queue is not empty (see the "Variables" tab to 
> the right).
> Alternatively, the same situation but in the case (a) is handled properly -- 
> i.e., all completion callback handlers are truly called with ZCLOSING and the 
> memory is freed, both for subcases (a.1) when there is a failure like 
> connection-timeout, connection-closed, etc., or (a.2) there is not failure. 
> The reason is that any callback handler (completion or watcher) in the case 
> (a) is called from the _process_completions()_ function which runs in the 
> loop until _zh.completions_to_process_ queue gets empty. So, this function 
> guarantees this queue to be completely processed even if new completions 
> occur during reaction on previously queued completions.
> *Consequently:*
> 1. At least there is definitely the +memory leak+ in the case (b) -- all the 
> fake responses put into _zh.completions_to_process_ queue are lost after 
> _free(zh)_
> 2. And it looks like a great misbehavior not to call completions on sent 
> requests in the case (b) while they are called with ZCLOSING in the case (a) 
> -- so, I think it's not "by design" but a +completions leak+
> +To reproduce the case (b) do the following:+
> - open ZooKeeper session, connect to a server, receive and process 
> connected-watch, etc.
> - then somewhere +from the main events loop+ call for example _zoo_acreate()_ 
> with valid arguments -- it shall return ZOK
> - then, +immediately after it returned+, call _zookeeper_close()_
> - note that completion callback handler for _zoo_acreate()_ *will not be 
> called*
> +To reproduce the case (a) do the following:+
> - the same as above, open 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Attachment: zk-will-free-zh-and-lose-completions.png

> Memory and completions leak on zookeeper_close.
> ---
>
> Key: ZOOKEEPER-2894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2894
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
> Attachments: zk-will-free-zh-and-lose-completions.png
>
>
> ZooKeeper C Client *+single thread+* build
> First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
> two ways:
> a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
> called through _zookeeper_process()_
> b) and from other places -- i.e., when the call-stack does not pass through 
> any of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_
> The issue described here below is +specific only to the case (b)+. So, it's 
> Ok with the case (a).
> When _zookeeper_close()_ is called in the (b) way, the following happens:
> 1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
> they all are removed from this queue and each of them is "completed" with 
> personal fake response having status ZCLOSING. Such fake responses are put 
> into _zh.completions_to_process_ queue. It's Ok
> 2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
> completion callbacks are called, nor dynamic memory allocated for fake 
> responses is freed+
> 3. Different structures within _zh_ are dismissed and finally _zh_ is freed
> This is illustrated on the screenshot attached to this ticket: you may see 
> that the next instruction to execute will be _free(zh)_ while 
> _zh.completions_to_process_ queue is not empty (see the "Variables" tab to 
> the right).
> Alternatively, the same situation but in the case (a) is handled properly -- 
> i.e., all completion callback handlers are truly called with ZCLOSING and the 
> memory is freed, both for subcases (a.1) when there is a failure like 
> connection-timeout, connection-closed, etc., or (a.2) there is not failure. 
> The reason is that any callback handler (completion or watcher) in the case 
> (a) is called from the _process_completions()_ function which runs in the 
> loop until _zh.completions_to_process_ queue gets empty. So, this function 
> guarantees this queue to be completely processed even if new completions 
> occur during reaction on previously queued completions.
> *Consequently:*
> 1. At least there is definitely the +memory leak+ in the case (b) -- all the 
> fake responses put into _zh.completions_to_process_ queue are lost after 
> _free(zh)_
> 2. And it looks like a great misbehavior not to call completions on sent 
> requests in the case (b) while they are called with ZCLOSING in the case (a) 
> -- so, I think it's not "by design" but a +completions leak+
> +To reproduce the case (b) do the following:+
> - open ZooKeeper session, connect to a server, receive and process 
> connected-watch, etc.
> - then somewhere +from the main events loop+ call for example _zoo_acreate()_ 
> with valid arguments -- it shall return ZOK
> - then, +immediately after it returned+, call _zookeeper_close()_
> - note that completion callback handler for _zoo_acreate()_ *will not be 
> called*
> +To reproduce the case (a) do the following:+
> - the same as above, open ZooKeeper session, connect to a server, receive and 
> process connected-watch, etc.
> - the same as above, somewhere from the main events loop call for example 
> _zoo_acreate()_ with valid arguments -- it shall return ZOK
> - but now don't call _zookeeper_close()_ immediately -- wait for completion 
> callback on the commenced request
> - when _zoo_acreate()_ completes, +from within its completion callback 
> handler+, call another _zoo_acreate()_ and immediately after it returned call 
> _zookeeper_close()_
> - note that completion callback handler for the second _zoo_acreate()_ *will 
> be called with ZCLOSING, unlike the case (b) described above*
> To fix this I propose calling to _process_completions()_ from 
> _destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int 
> rc)_.
> This is a proposed fix: https://github.com/apache/zookeeper/pull/363
> [upd]
> There are another tickets with about the same problem: ZOOKEEPER-1493, 
> ZOOKEEPER-2073 (the "same" just according to their titles).
> However, as I can see, the 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

Alternatively, the same situation but in the case (a) is handled properly -- 
i.e., all completion callback handlers are truly called with ZCLOSING and the 
memory is freed, both for subcases (a.1) when there is a failure like 
connection-timeout, connection-closed, etc., or (a.2) there is not failure. The 
reason is that any callback handler (completion or watcher) in the case (a) is 
called from the _process_completions()_ function which runs in the loop until 
_zh.completions_to_process_ queue gets empty. So, this function guarantees this 
queue to be completely processed even if new completions occur during reaction 
on previously queued completions.

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b) -- all the 
fake responses put into _zh.completions_to_process_ queue are lost after 
_free(zh)_
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) while they are called with ZCLOSING in the case (a) -- 
so, I think it's not "by design" but a +completions leak+

+To reproduce the case (b) do the following:+
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then somewhere +from the main events loop+ call for example _zoo_acreate()_ 
with valid arguments -- it shall return ZOK
- then, +immediately after it returned+, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ *will not be called*

+To reproduce the case (a) do the following:+
- the same as above, open ZooKeeper session, connect to a server, receive and 
process connected-watch, etc.
- the same as above, somewhere from the main events loop call for example 
_zoo_acreate()_ with valid arguments -- it shall return ZOK
- but now don't call _zookeeper_close()_ immediately -- wait for completion 
callback on the commenced request
- when _zoo_acreate()_ completes, +from within its completion callback 
handler+, call another _zoo_acreate()_ and immediately after it returned call 
_zookeeper_close()_
- note that completion callback handler for the second _zoo_acreate()_ *will be 
called with ZCLOSING, unlike the case (b) described above*

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

This is a proposed fix: https://github.com/apache/zookeeper/pull/363

[upd]
There are another tickets with about the same problem: ZOOKEEPER-1493, 
ZOOKEEPER-2073 (the "same" just according to their titles).
However, as I can see, the corresponding patches were applied on the branch 
3.4.10, but the effect still persists -- so, this ticket does not duplicate the 
mentioned two.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

Alternatively, the same situation but in the case (a) is handled properly -- 
i.e., all completion callback handlers are truly called with ZCLOSING and the 
memory is freed, both for subcases (a.1) when there is a failure like 
connection-timeout, connection-closed, etc., or (a.2) there is not failure. The 
reason is that any callback handler (completion or watcher) in the case (a) is 
called from the _process_completions()_ function which runs in the loop until 
_zh.completions_to_process_ queue gets empty. So, this function guarantees this 
queue to be completely processed even if new completions occur during reaction 
on previously queued completions.

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b) -- all the 
fake responses put into _zh.completions_to_process_ queue are lost after 
_free(zh)_
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) while they are called with ZCLOSING in the case (a) -- 
so, I think it's not "by design" but a +completions leak+

+To reproduce the case (b) do the following:+
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then somewhere +from the main events loop+ call for example _zoo_acreate()_ 
with valid arguments -- it shall return ZOK
- then, +immediately after it returned+, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ *will not be called*

+To reproduce the case (a) do the following:+
- the same as above, open ZooKeeper session, connect to a server, receive and 
process connected-watch, etc.
- the same as above, somewhere from the main events loop call for example 
_zoo_acreate()_ with valid arguments -- it shall return ZOK
- but now don't call _zookeeper_close()_ immediately -- wait for completion 
callback on the commenced request
- when _zoo_acreate()_ completes, +from within its completion callback 
handler+, call another _zoo_acreate()_ and immediately after it returned call 
_zookeeper_close()_
- note that completion callback handler for the second _zoo_acreate()_ *will be 
called with ZCLOSING, unlike the case (b) described above*

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

This is a proposed fix: https://github.com/apache/zookeeper/pull/363

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Labels: easyfix  (was: )

> Memory and completions leak on zookeeper_close.
> ---
>
> Key: ZOOKEEPER-2894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2894
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> ZooKeeper C Client *+single thread+* build
> First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
> two ways:
> a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
> called through _zookeeper_process()_
> b) and from other places -- i.e., when the call-stack does not pass through 
> any of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_
> The issue described here below is +specific only to the case (b)+. So, it's 
> Ok with the case (a).
> When _zookeeper_close()_ is called in the (b) way, the following happens:
> 1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
> they all are removed from this queue and each of them is "completed" with 
> personal fake response having status ZCLOSING. Such fake responses are put 
> into _zh.completions_to_process_ queue. It's Ok
> 2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
> completion callbacks are called, nor dynamic memory allocated for fake 
> responses is freed+
> 3. Different structures within _zh_ are dismissed and finally _zh_ is freed
> This is illustrated on the screenshot attached to this ticket: you may see 
> that the next instruction to execute will be _free(zh)_ while 
> _zh.completions_to_process_ queue is not empty (see the "Variables" tab to 
> the right).
> Alternatively, the same situation but in the case (a) is handled properly -- 
> i.e., all completion callback handlers are truly called with ZCLOSING and the 
> memory is freed, both for subcases (a.1) when there is a failure like 
> connection-timeout, connection-closed, etc., or (a.2) there is not failure. 
> The reason is that any callback handler (completion or watcher) in the case 
> (a) is called from the _process_completions()_ function which runs in the 
> loop until _zh.completions_to_process_ queue gets empty. So, this function 
> guarantees this queue to be completely processed even if new completions 
> occur during reaction on previously queued completions.
> *Consequently:*
> 1. At least there is definitely the +memory leak+ in the case (b) -- all the 
> fake responses put into _zh.completions_to_process_ queue are lost after 
> _free(zh)_
> 2. And it looks like a great misbehavior not to call completions on sent 
> requests in the case (b) while they are called with ZCLOSING in the case (a) 
> -- so, I think it's not "by design" but a +completions leak+
> +To reproduce the case (b) do the following:+
> - open ZooKeeper session, connect to a server, receive and process 
> connected-watch, etc.
> - then somewhere +from the main events loop+ call for example _zoo_acreate()_ 
> with valid arguments -- it shall return ZOK
> - then, +immediately after it returned+, call _zookeeper_close()_
> - note that completion callback handler for _zoo_acreate()_ *will not be 
> called*
> +To reproduce the case (a) do the following:+
> - the same as above, open ZooKeeper session, connect to a server, receive and 
> process connected-watch, etc.
> - the same as above, somewhere from the main events loop call for example 
> _zoo_acreate()_ with valid arguments -- it shall return ZOK
> - but now don't call _zookeeper_close()_ immediately -- wait for completion 
> callback on the commenced request
> - when _zoo_acreate()_ completes, +from within its completion callback 
> handler+, call another _zoo_acreate()_ and immediately after it returned call 
> _zookeeper_close()_
> - note that completion callback handler for the second _zoo_acreate()_ *will 
> be called with ZCLOSING, unlike the case (b) described above*
> To fix this I propose calling to _process_completions()_ from 
> _destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int 
> rc)_.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158587#comment-16158587
 ] 

ASF GitHub Bot commented on ZOOKEEPER-2894:
---

GitHub user xoiss opened a pull request:

https://github.com/apache/zookeeper/pull/363

branch-3.4 -- bugfix -- ZOOKEEPER-2894

Fixes https://issues.apache.org/jira/browse/ZOOKEEPER-2894

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xoiss/zookeeper 
branch-3.4-bugfix-zookeeper-2894

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/363.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #363


commit ba56555b39edacd3be75ad0b07cc7ce86861be16
Author: xoiss 
Date:   2017-09-08T12:49:06Z

ZOOKEEPER-2894 - fix memory and completion leak on zookeeper_close.




> Memory and completions leak on zookeeper_close.
> ---
>
> Key: ZOOKEEPER-2894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2894
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> ZooKeeper C Client *+single thread+* build
> First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
> two ways:
> a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
> called through _zookeeper_process()_
> b) and from other places -- i.e., when the call-stack does not pass through 
> any of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_
> The issue described here below is +specific only to the case (b)+. So, it's 
> Ok with the case (a).
> When _zookeeper_close()_ is called in the (b) way, the following happens:
> 1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
> they all are removed from this queue and each of them is "completed" with 
> personal fake response having status ZCLOSING. Such fake responses are put 
> into _zh.completions_to_process_ queue. It's Ok
> 2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
> completion callbacks are called, nor dynamic memory allocated for fake 
> responses is freed+
> 3. Different structures within _zh_ are dismissed and finally _zh_ is freed
> This is illustrated on the screenshot attached to this ticket: you may see 
> that the next instruction to execute will be _free(zh)_ while 
> _zh.completions_to_process_ queue is not empty (see the "Variables" tab to 
> the right).
> Alternatively, the same situation but in the case (a) is handled properly -- 
> i.e., all completion callback handlers are truly called with ZCLOSING and the 
> memory is freed, both for subcases (a.1) when there is a failure like 
> connection-timeout, connection-closed, etc., or (a.2) there is not failure. 
> The reason is that any callback handler (completion or watcher) in the case 
> (a) is called from the _process_completions()_ function which runs in the 
> loop until _zh.completions_to_process_ queue gets empty. So, this function 
> guarantees this queue to be completely processed even if new completions 
> occur during reaction on previously queued completions.
> *Consequently:*
> 1. At least there is definitely the +memory leak+ in the case (b) -- all the 
> fake responses put into _zh.completions_to_process_ queue are lost after 
> _free(zh)_
> 2. And it looks like a great misbehavior not to call completions on sent 
> requests in the case (b) while they are called with ZCLOSING in the case (a) 
> -- so, I think it's not "by design" but a +completions leak+
> +To reproduce the case (b) do the following:+
> - open ZooKeeper session, connect to a server, receive and process 
> connected-watch, etc.
> - then somewhere +from the main events loop+ call for example _zoo_acreate()_ 
> with valid arguments -- it shall return ZOK
> - then, +immediately after it returned+, call _zookeeper_close()_
> - note that completion callback handler for _zoo_acreate()_ *will not be 
> called*
> +To reproduce the case (a) do the following:+
> - the same as above, open ZooKeeper session, connect to a server, receive and 
> process connected-watch, etc.
> - the same as above, somewhere from the main events loop call for example 
> _zoo_acreate()_ with valid arguments -- it shall return ZOK
> - but now don't call _zookeeper_close()_ immediately -- wait for completion 
> callback on the commenced 

[GitHub] zookeeper pull request #363: branch-3.4 -- bugfix -- ZOOKEEPER-2894

[xoiss]

GitHub user xoiss opened a pull request:

https://github.com/apache/zookeeper/pull/363

branch-3.4 -- bugfix -- ZOOKEEPER-2894

Fixes https://issues.apache.org/jira/browse/ZOOKEEPER-2894

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xoiss/zookeeper 
branch-3.4-bugfix-zookeeper-2894

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zookeeper/pull/363.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #363


commit ba56555b39edacd3be75ad0b07cc7ce86861be16
Author: xoiss 
Date:   2017-09-08T12:49:06Z

ZOOKEEPER-2894 - fix memory and completion leak on zookeeper_close.




---

ZooKeeper_branch35_jdk8 - Build # 663 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/ZooKeeper_branch35_jdk8/663/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 65.40 MB...]
[junit] 2017-09-08 12:13:19,675 [myid:] - WARN  [New I/O boss 
#1971:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 
0x3bdd42ae] EXCEPTION: java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24751
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24751
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit] at java.lang.Thread.run(Thread.java:748)
[junit] 2017-09-08 12:13:19,675 [myid:] - INFO  [New I/O boss 
#1971:ClientCnxnSocketNetty@208] - channel is told closing
[junit] 2017-09-08 12:13:19,676 [myid:127.0.0.1:24751] - INFO  
[main-SendThread(127.0.0.1:24751):ClientCnxn$SendThread@1231] - channel for 
sessionid 0x107a73ae4ff is lost, closing socket connection and attempting 
reconnect
[junit] 2017-09-08 12:13:19,819 [myid:127.0.0.1:24736] - INFO  
[main-SendThread(127.0.0.1:24736):ClientCnxn$SendThread@1113] - Opening socket 
connection to server 127.0.0.1/127.0.0.1:24736. Will not attempt to 
authenticate using SASL (unknown error)
[junit] 2017-09-08 12:13:19,820 [myid:] - INFO  [New I/O boss 
#1341:ClientCnxnSocketNetty$1@127] - future isn't success, cause: {}
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24736
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit] at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit] at java.lang.Thread.run(Thread.java:748)
[junit] 2017-09-08 12:13:19,821 [myid:] - WARN  [New I/O boss 
#1341:ClientCnxnSocketNetty$ZKClientHandler@439] - Exception caught: [id: 
0xec722b3f] EXCEPTION: java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24736
[junit] java.net.ConnectException: Connection refused: 
127.0.0.1/127.0.0.1:24736
[junit] at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit] at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
[junit] at 
org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[junit] at 
org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
[junit] at 
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[junit] at 
org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[junit] at 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

Alternatively, the same situation but in the case (a) is handled properly -- 
i.e., all completion callback handlers are truly called with ZCLOSING and the 
memory is freed, both for subcases (a.1) when there is a failure like 
connection-timeout, connection-closed, etc., or (a.2) there is not failure. The 
reason is that any callback handler (completion or watcher) in the case (a) is 
called from the _process_completions()_ function which runs in the loop until 
_zh.completions_to_process_ queue gets empty. So, this function guarantees this 
queue to be completely processed even if new completions occur during reaction 
on previously queued completions.

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b) -- all the 
fake responses put into _zh.completions_to_process_ queue are lost after 
_free(zh)_
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) while they are called with ZCLOSING in the case (a) -- 
so, I think it's not "by design" but a +completions leak+

+To reproduce the case (b) do the following:+
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then somewhere +from the main events loop+ call for example _zoo_acreate()_ 
with valid arguments -- it shall return ZOK
- then, +immediately after it returned+, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ *will not be called*

+To reproduce the case (a) do the following:+
- the same as above, open ZooKeeper session, connect to a server, receive and 
process connected-watch, etc.
- the same as above, somewhere from the main events loop call for example 
_zoo_acreate()_ with valid arguments -- it shall return ZOK
- but now don't call _zookeeper_close()_ immediately -- wait for completion 
callback on the commenced request
- when _zoo_acreate()_ completes, +from within its completion callback 
handler+, call another _zoo_acreate()_ and immediately after it returned call 
_zookeeper_close()_
- note that completion callback handler for the second _zoo_acreate()_ *will be 
called with ZCLOSING, unlike the case (b) described above*

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

Alternatively, the same situation but in the case (a) is handled properly -- 
i.e., all completion callback handlers are truly called with ZCLOSING and the 
memory is freed, both for subcases (a.1) when there is a failure like 
connection-timeout, connection-closed, etc., or (a.2) there is not failure. The 
reason is that any callback handler (completion or watcher) in the case (a) is 
called from the _process_completions()_ function which runs in the loop until 
_zh.completions_to_process_ queue gets empty. So, this function guarantees this 
queue to be completely processed even if new completions occur during reaction 
on previously queued completions.

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b) -- all the 
fake responses put into _zh.completions_to_process_ queue are lost after 
_free(zh)_
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) while they are called with ZCLOSING in the case (a) -- 
so, I think it's not "by design" but a +completions leak+

+To reproduce the case (b) do the following:+
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then somewhere +from the main events loop+ call for example _zoo_acreate()_ 
with valid arguments -- it shall return ZOK
- then, +immediately after it returned+, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

+To reproduce the case (a) do the following:+
- the same as above, open ZooKeeper session, connect to a server, receive and 
process connected-watch, etc.
- the same as above, somewhere from the main events loop call for example 
_zoo_acreate()_ with valid arguments -- it shall return ZOK
- but now don't call _zookeeper_close()_ immediately -- wait for completion 
callback on the commenced request
- when _zoo_acreate()_ completes, +from within its completion callback 
handler+, call another _zoo_acreate()_ and immediately after it returned call 
_zookeeper_close()_
- note that completion callback handler for the second _zoo_acreate()_ will be 
called with ZCLOSING, unlike the case (b) described above

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

Alternatively, the same situation but in the case (a) is handled properly -- 
i.e., all completion callback handlers are truly called with ZCLOSING and the 
memory is freed, both for subcases (a.1) when there is a failure like 
connection-timeout, connection-closed, etc., or (a.2) there is not failure. The 
reason is that any callback handler (completion or watcher) in the case (a) is 
called from the _process_completions()_ function which runs in the loop until 
_zh.completions_to_process_ queue gets empty. So, this function guarantees this 
queue to be completely processed even if new completions occur during reaction 
on previously queued completions.

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b) -- all the 
fake responses put into _zh.completions_to_process_ queue are lost after 
_free(zh)_
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) while they are called with ZCLOSING in the case (a) -- 
so, I think it's not "by design" but a +completions leak+

+To reproduce the case (b) do the following:+
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then somewhere from the main events loop call for example _zoo_acreate()_ 
with valid arguments -- it shall return ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

+To reproduce the case (a) do the following:+
- the same as above, open ZooKeeper session, connect to a server, receive and 
process connected-watch, etc.
- the same as above, somewhere from the main events loop call for example 
_zoo_acreate()_ with valid arguments -- it shall return ZOK
- but now don't call _zookeeper_close()_ immediately -- wait for completion 
callback on the commenced request
- when _zoo_acreate()_ completes, +from within its completion callback 
handler+, call another _zoo_acreate()_ and immediately after it returned call 
_zookeeper_close()_
- note that completion callback handler for the second _zoo_acreate()_ will be 
called with ZCLOSING, unlike the case (b) described above

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

Alternatively, the same situation but in the case (a) is handled properly -- 
i.e., all completion callback handlers are truly called with ZCLOSING and the 
memory is freed, both for subcases (a.1) when there is a failure like 
connection-timeout, connection-closed, etc., or (a.2) there is not failure. The 
reason is that any callback handler (completion or watcher) in the case (a) is 
called from the _process_completions()_ function which runs in the loop until 
_zh.completions_to_process_ queue gets empty. So, this function guarantees this 
queue to be completely processed even if new completions occur during reaction 
on previously queued completions.

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b) -- all the 
fake responses put into _zh.completions_to_process_ queue are lost after 
_free(zh)_
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) while they are called with ZCLOSING in the case (a) -- 
so, I think it's not "by design" but a +completions leak+

+To reproduce the case (b) do the following:+
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then somewhere from the main events loop call for example _zoo_acreate()_ 
with valid arguments -- it shall return ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

+To reproduce the case (a) do the following:+
- the same as above open ZooKeeper session, connect to a server, receive and 
process connected-watch, etc.
- the same as above somewhere from the main events loop call for example 
_zoo_acreate()_ with valid arguments -- it shall return ZOK
- but now don't call _zookeeper_close()_ immediately -- wait for completion 
callback on the issued request
- when _zoo_acreate()_ completes, from within the completion callback handler 
call another _zoo_acreate()_ and immediately after it returned call 
_zookeeper_close()_
- note that completion callback handler for the second _zoo_acreate()_ will be 
called with ZCLOSING, unlike the case (b) described above

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. +If there are requests waiting for responses in _zh.sent_requests_ queue+, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b) -- all the 
fake responses put into _zh.completions_to_process_ queue are lost after 
_free(zh)_
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) while they are called with ZCLOSING in the case (a) -- 
so, I think it's not "by design" but a +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b) -- all the 
fake responses put into _zh.completions_to_process_ queue are lost after 
_free(zh)_
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) while they are called with ZCLOSING in the case (a) -- 
so, I think it's not "by design" but a +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.


> Memory and completions leak 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b) -- all the 
fake responses put into _zh.completions_to_process_ queue are lost after 
_free(zh)_
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) while they are called with ZCLOSING in the case (a) -- 
so, I think it's not "by design" but a +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b)
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) as soon as they are called with ZCLOSING in the case 
(a) -- so, the +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.


> Memory and completions leak on zookeeper_close.
> ---
>
> Key: ZOOKEEPER-2894
> 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
_zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b)
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) as soon as they are called with ZCLOSING in the case 
(a) -- so, the +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
the _zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b)
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) as soon as they are called with ZCLOSING in the case 
(a) -- so, the +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.


> Memory and completions leak on zookeeper_close.
> ---
>
> Key: ZOOKEEPER-2894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2894
> Project: ZooKeeper
>  Issue Type: 

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
the _zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b)
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) as soon as they are called with ZCLOSING in the case 
(a) -- so, the +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in the _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
the _zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b)
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) as soon as they are called with ZCLOSING in the case 
(a) -- so, the +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.


> Memory and completions leak on zookeeper_close.
> ---
>
> Key: ZOOKEEPER-2894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2894
> Project: ZooKeeper
>  

[jira] [Updated] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2894:
-
Description: 
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in the _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with 
personal fake response having status ZCLOSING. Such fake responses are put into 
the _zh.completions_to_process_ queue. It's Ok
2. But then, _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b)
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) as soon as they are called with ZCLOSING in the case 
(a) -- so, the +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.

  was:
ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in the _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with the 
fake response having status ZCLOSING. Such fake responses are put into the 
_zh.completions_to_process_ queue. It's Ok
2. But then, the _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b)
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) as soon as they are called with ZCLOSING in the case 
(a) -- so, the +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.


> Memory and completions leak on zookeeper_close.
> ---
>
> Key: ZOOKEEPER-2894
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2894
> Project: ZooKeeper
>  

[jira] [Created] (ZOOKEEPER-2894) Memory and completions leak on zookeeper_close.

[Alexander A. Strelets (JIRA)]

Alexander A. Strelets created ZOOKEEPER-2894:


 Summary: Memory and completions leak on zookeeper_close.
 Key: ZOOKEEPER-2894
 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2894
 Project: ZooKeeper
  Issue Type: Bug
  Components: c client
Affects Versions: 3.4.10
 Environment: Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4
Reporter: Alexander A. Strelets
Priority: Critical
 Fix For: 3.4.10


ZooKeeper C Client *+single thread+* build

First of all, ZooKeeper C Client design allows calling _zookeeper_close()_ in 
two ways:
a) from a ZooKeeper callback handler (completion or watcher) which in turn is 
called through _zookeeper_process()_
b) and from other places -- i.e., when the call-stack does not pass through any 
of zookeeper mechanics prior to enter into mentioned _zookeeper_close()_

The issue described here below is +specific only to the case (b)+. So, it's Ok 
with the case (a).

When _zookeeper_close()_ is called in the (b) way, the following happens:
1. If there are requests waiting for responses in the _zh.sent_requests_ queue, 
they all are removed from this queue and each of them is "completed" with the 
fake response having status ZCLOSING. Such fake responses are put into the 
_zh.completions_to_process_ queue. It's Ok
2. But then, the _zh.completions_to_process_ queue is left unhandled. +Neither 
completion callbacks are called, nor dynamic memory allocated for fake 
responses is freed.+
3. Different structures within _zh_ are dismissed and finally _zh_ is freed

This is illustrated on the screenshot attached to this ticket: you may see that 
the next instruction to execute will be _free(zh)_ while 
_zh.completions_to_process_ queue is not empty (see the "Variables" tab to the 
right).

*Consequently:*
1. At least there is definitely the +memory leak+ in the case (b)
2. And it looks like a great misbehavior not to call completions on sent 
requests in the case (b) as soon as they are called with ZCLOSING in the case 
(a) -- so, the +completions leak+

To reproduce the case (b) do the following:
- open ZooKeeper session, connect to a server, receive and process 
connected-watch, etc.
- then call for example _zoo_acreate()_ with valid arguments -- it shall return 
ZOK
- then, immediately after it returned, call _zookeeper_close()_
- note that completion callback handler for _zoo_acreate()_ will not be called

To reproduce the case (a) do about the same, but call _zookeeper_close()_ from 
a watcher or another completion callback handler.

To fix this I propose calling to _process_completions()_ from 
_destroy(zhandle_t *zh)_ as it is done in _handle_error(zhandle_t *zh,int rc)_.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ZOOKEEPER-2891) SIGABRT from assert during fake completion on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2891:
-
Environment: 
Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4

  was:
Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4

ZooKeeper C Client *single thread* build


> SIGABRT from assert during fake completion on zookeeper_close.
> --
>
> Key: ZOOKEEPER-2891
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2891
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the 
> so called "Fake response" which is fabricated by the function 
> _free_completions()_ for example when _zookeeper_close()_ is called while 
> there is a pending _multi_ request.
> Such fake response includes only the header but zero bytes for the body. Due 
> to this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is 
> called repeatedly for each {{completion_list_t *entry = 
> dequeue_completion(clist)}}, does not assign the _mhdr_ and keeps _mhdr.done 
> == 0_ as it was originally initialized. Consequently the _while (!mhdr.done)_ 
> does not ever end, and finally falls into the _assert(entry)_ with _entry == 
> NULL_ when all sub-requests are "completed". ~// Normally on my platform 
> assert raises SIGABRT.~
> I propose to instruct the _deserialize_multi()_ function to break the loop on 
> _entry == NULL_ if it was called for an unsuccessfull overal status of the 
> multi response, and in particular for the fake response having _ZCLOSING_ 
> (-116) status. I have introduced the _rc0_ parameter for this.
> *Another issue* with this function is that even if the while-loop exited 
> properly, this function returns _rc == 0_, and this return code +overrides+ 
> the true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in 
> the _deserialize_response()_ function. So, the _multi_ response callback 
> +handler would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which 
> is strictly wrong.
> To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
> zero (which is _ZOK_ indeed).
> This is a proposed fix: https://github.com/apache/zookeeper/pull/360
> [upd]
> It looks like about the same problem is described in ZOOKEEPER-1636
> However, the patch proposed in this ticket also remedies the second linked 
> problem: reporting ZCLOSING status (as required) to the multi-request 
> completion handler.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2890:
-
Description: 
ZooKeeper C Client *+single thread+* build

Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
automatic variable *_struct CreateResponse res_* which is +left uninitialized+ 
and passed to the function _deserialize_GetACLResponse()_ and then to 
_deallocate_GetACLResponse()_.

The _deserialize_ function, which is called the first, is expected to assign 
the _res_ variable with a value from the parsed _struct iarchive *ia_. But, if 
_ia_ contains for example insufficient amount of bytes the 
_deserialize_String()_ function refuses of assigning a value to _res_, and 
_res_ stays uninitialized (the true case is described below). Then, the 
_deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
with arguments. If incidentally the memory region in the program stack under 
the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
address+.

The true case: this happens when an active _multi_ request with _create_ 
sub-request is completed on call to _zookeeper_close()_ with the so called 
"Fake response" which is fabricated by the function _free_completions()_. Such 
response includes only the header but +zero bytes for the body+. The 
significant condition is that the _create_ request is not a stand-alone one, 
but namely a sub-request within the _multi_ request. In this case the 
_deserialize_response()_ is called recursively (for each sub-request), and when 
it is called for the _create_ subrequest (from the nested 
_deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
the _if (failed)_ condition branches to the _else_ part. Note that in the 
stand-alone create-request case this does not occur.

*I suspect this may happen not only due to call to _zookeeper_close()_ but on 
reception of a true multi-response from the server* containing insufficient 
number of bytes (I'm not sure if it can be a proper response from the server 
with an error overall status and empty or insufficient payload).

This is a proposed fix: https://github.com/apache/zookeeper/pull/359

  was:
Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
automatic variable *_struct CreateResponse res_* which is +left uninitialized+ 
and passed to the function _deserialize_GetACLResponse()_ and then to 
_deallocate_GetACLResponse()_.

The _deserialize_ function, which is called the first, is expected to assign 
the _res_ variable with a value from the parsed _struct iarchive *ia_. But, if 
_ia_ contains for example insufficient amount of bytes the 
_deserialize_String()_ function refuses of assigning a value to _res_, and 
_res_ stays uninitialized (the true case is described below). Then, the 
_deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
with arguments. If incidentally the memory region in the program stack under 
the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
address+.

The true case: this happens when an active _multi_ request with _create_ 
sub-request is completed on call to _zookeeper_close()_ with the so called 
"Fake response" which is fabricated by the function _free_completions()_. Such 
response includes only the header but +zero bytes for the body+. The 
significant condition is that the _create_ request is not a stand-alone one, 
but namely a sub-request within the _multi_ request. In this case the 
_deserialize_response()_ is called recursively (for each sub-request), and when 
it is called for the _create_ subrequest (from the nested 
_deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
the _if (failed)_ condition branches to the _else_ part. Note that in the 
stand-alone create-request case this does not occur.

*I suspect this may happen not only due to call to _zookeeper_close()_ but on 
reception of a true multi-response from the server* containing insufficient 
number of bytes (I'm not sure if it can be a proper response from the server 
with an error overall status and empty or insufficient payload).

This is a proposed fix: https://github.com/apache/zookeeper/pull/359


> Local automatic variable is left uninitialized and then freed.
> --
>
> Key: ZOOKEEPER-2890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  

[jira] [Updated] (ZOOKEEPER-2891) SIGABRT from assert during fake completion on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2891:
-
Description: 
ZooKeeper C Client *+single thread+* build

Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the so 
called "Fake response" which is fabricated by the function _free_completions()_ 
for example when _zookeeper_close()_ is called while there is a pending _multi_ 
request.

Such fake response includes only the header but zero bytes for the body. Due to 
this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is called 
repeatedly for each {{completion_list_t *entry = dequeue_completion(clist)}}, 
does not assign the _mhdr_ and keeps _mhdr.done == 0_ as it was originally 
initialized. Consequently the _while (!mhdr.done)_ does not ever end, and 
finally falls into the _assert(entry)_ with _entry == NULL_ when all 
sub-requests are "completed". ~// Normally on my platform assert raises 
SIGABRT.~

I propose to instruct the _deserialize_multi()_ function to break the loop on 
_entry == NULL_ if it was called for an unsuccessfull overal status of the 
multi response, and in particular for the fake response having _ZCLOSING_ 
(-116) status. I have introduced the _rc0_ parameter for this.


*Another issue* with this function is that even if the while-loop exited 
properly, this function returns _rc == 0_, and this return code +overrides+ the 
true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in the 
_deserialize_response()_ function. So, the _multi_ response callback +handler 
would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which is strictly 
wrong.

To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
zero (which is _ZOK_ indeed).

This is a proposed fix: https://github.com/apache/zookeeper/pull/360

[upd]
It looks like about the same problem is described in ZOOKEEPER-1636
However, the patch proposed in this ticket also remedies the second linked 
problem: reporting ZCLOSING status (as required) to the multi-request 
completion handler.

  was:
Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the so 
called "Fake response" which is fabricated by the function _free_completions()_ 
for example when _zookeeper_close()_ is called while there is a pending _multi_ 
request.

Such fake response includes only the header but zero bytes for the body. Due to 
this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is called 
repeatedly for each {{completion_list_t *entry = dequeue_completion(clist)}}, 
does not assign the _mhdr_ and keeps _mhdr.done == 0_ as it was originally 
initialized. Consequently the _while (!mhdr.done)_ does not ever end, and 
finally falls into the _assert(entry)_ with _entry == NULL_ when all 
sub-requests are "completed". ~// Normally on my platform assert raises 
SIGABRT.~

I propose to instruct the _deserialize_multi()_ function to break the loop on 
_entry == NULL_ if it was called for an unsuccessfull overal status of the 
multi response, and in particular for the fake response having _ZCLOSING_ 
(-116) status. I have introduced the _rc0_ parameter for this.


*Another issue* with this function is that even if the while-loop exited 
properly, this function returns _rc == 0_, and this return code +overrides+ the 
true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in the 
_deserialize_response()_ function. So, the _multi_ response callback +handler 
would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which is strictly 
wrong.

To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
zero (which is _ZOK_ indeed).

This is a proposed fix: https://github.com/apache/zookeeper/pull/360

[upd]
It looks like about the same problem is described in ZOOKEEPER-1636
However, the patch proposed in this ticket also remedies the second linked 
problem: reporting ZCLOSING status (as required) to the multi-request 
completion handler.


> SIGABRT from assert during fake completion on zookeeper_close.
> --
>
> Key: ZOOKEEPER-2891
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2891
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> ZooKeeper C Client *+single thread+* build
> Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the 
> so called "Fake response" which is fabricated by the function 
> _free_completions()_ for example 

[jira] [Updated] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2890:
-
Environment: 
Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4

ZooKeeper C Client *+single thread+* build

  was:
Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4


> Local automatic variable is left uninitialized and then freed.
> --
>
> Key: ZOOKEEPER-2890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
> ZooKeeper C Client *+single thread+* build
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
> automatic variable *_struct CreateResponse res_* which is +left 
> uninitialized+ and passed to the function _deserialize_GetACLResponse()_ and 
> then to _deallocate_GetACLResponse()_.
> The _deserialize_ function, which is called the first, is expected to assign 
> the _res_ variable with a value from the parsed _struct iarchive *ia_. But, 
> if _ia_ contains for example insufficient amount of bytes the 
> _deserialize_String()_ function refuses of assigning a value to _res_, and 
> _res_ stays uninitialized (the true case is described below). Then, the 
> _deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
> with arguments. If incidentally the memory region in the program stack under 
> the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
> address+.
> The true case: this happens when an active _multi_ request with _create_ 
> sub-request is completed on call to _zookeeper_close()_ with the so called 
> "Fake response" which is fabricated by the function _free_completions()_. 
> Such response includes only the header but +zero bytes for the body+. The 
> significant condition is that the _create_ request is not a stand-alone one, 
> but namely a sub-request within the _multi_ request. In this case the 
> _deserialize_response()_ is called recursively (for each sub-request), and 
> when it is called for the _create_ subrequest (from the nested 
> _deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
> the _if (failed)_ condition branches to the _else_ part. Note that in the 
> stand-alone create-request case this does not occur.
> *I suspect this may happen not only due to call to _zookeeper_close()_ but on 
> reception of a true multi-response from the server* containing insufficient 
> number of bytes (I'm not sure if it can be a proper response from the server 
> with an error overall status and empty or insufficient payload).
> This is a proposed fix: https://github.com/apache/zookeeper/pull/359



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ZOOKEEPER-2890) Local automatic variable is left uninitialized and then freed.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2890:
-
Environment: 
Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4

  was:
Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4

ZooKeeper C Client *+single thread+* build


> Local automatic variable is left uninitialized and then freed.
> --
>
> Key: ZOOKEEPER-2890
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2890
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_response()_*, in _case COMPLETION_STRING_, uses local 
> automatic variable *_struct CreateResponse res_* which is +left 
> uninitialized+ and passed to the function _deserialize_GetACLResponse()_ and 
> then to _deallocate_GetACLResponse()_.
> The _deserialize_ function, which is called the first, is expected to assign 
> the _res_ variable with a value from the parsed _struct iarchive *ia_. But, 
> if _ia_ contains for example insufficient amount of bytes the 
> _deserialize_String()_ function refuses of assigning a value to _res_, and 
> _res_ stays uninitialized (the true case is described below). Then, the 
> _deallocate_ function calls _deallocate_String()_ passing uninitialized _res_ 
> with arguments. If incidentally the memory region in the program stack under 
> the _res_ was not equal to NULL, the last call +leads to _free()_ by invalid 
> address+.
> The true case: this happens when an active _multi_ request with _create_ 
> sub-request is completed on call to _zookeeper_close()_ with the so called 
> "Fake response" which is fabricated by the function _free_completions()_. 
> Such response includes only the header but +zero bytes for the body+. The 
> significant condition is that the _create_ request is not a stand-alone one, 
> but namely a sub-request within the _multi_ request. In this case the 
> _deserialize_response()_ is called recursively (for each sub-request), and 
> when it is called for the _create_ subrequest (from the nested 
> _deserialize_multi()_) the _failed_ parameter is assigned with false (0), so 
> the _if (failed)_ condition branches to the _else_ part. Note that in the 
> stand-alone create-request case this does not occur.
> *I suspect this may happen not only due to call to _zookeeper_close()_ but on 
> reception of a true multi-response from the server* containing insufficient 
> number of bytes (I'm not sure if it can be a proper response from the server 
> with an error overall status and empty or insufficient payload).
> This is a proposed fix: https://github.com/apache/zookeeper/pull/359



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ZOOKEEPER-2891) SIGABRT from assert during fake completion on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2891:
-
Environment: 
Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4

ZooKeeper C Client *single thread* build

  was:
Linux ubuntu 4.4.0-87-generic
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

https://github.com/apache/zookeeper.git
branch-3.4


> SIGABRT from assert during fake completion on zookeeper_close.
> --
>
> Key: ZOOKEEPER-2891
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2891
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
> ZooKeeper C Client *single thread* build
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the 
> so called "Fake response" which is fabricated by the function 
> _free_completions()_ for example when _zookeeper_close()_ is called while 
> there is a pending _multi_ request.
> Such fake response includes only the header but zero bytes for the body. Due 
> to this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is 
> called repeatedly for each {{completion_list_t *entry = 
> dequeue_completion(clist)}}, does not assign the _mhdr_ and keeps _mhdr.done 
> == 0_ as it was originally initialized. Consequently the _while (!mhdr.done)_ 
> does not ever end, and finally falls into the _assert(entry)_ with _entry == 
> NULL_ when all sub-requests are "completed". ~// Normally on my platform 
> assert raises SIGABRT.~
> I propose to instruct the _deserialize_multi()_ function to break the loop on 
> _entry == NULL_ if it was called for an unsuccessfull overal status of the 
> multi response, and in particular for the fake response having _ZCLOSING_ 
> (-116) status. I have introduced the _rc0_ parameter for this.
> *Another issue* with this function is that even if the while-loop exited 
> properly, this function returns _rc == 0_, and this return code +overrides+ 
> the true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in 
> the _deserialize_response()_ function. So, the _multi_ response callback 
> +handler would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which 
> is strictly wrong.
> To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
> zero (which is _ZOK_ indeed).
> This is a proposed fix: https://github.com/apache/zookeeper/pull/360
> [upd]
> It looks like about the same problem is described in ZOOKEEPER-1636
> However, the patch proposed in this ticket also remedies the second linked 
> problem: reporting ZCLOSING status (as required) to the multi-request 
> completion handler.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ZOOKEEPER-2891) SIGABRT from assert during fake completion on zookeeper_close.

[Alexander A. Strelets (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ZOOKEEPER-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander A. Strelets updated ZOOKEEPER-2891:
-
Description: 
Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the so 
called "Fake response" which is fabricated by the function _free_completions()_ 
for example when _zookeeper_close()_ is called while there is a pending _multi_ 
request.

Such fake response includes only the header but zero bytes for the body. Due to 
this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is called 
repeatedly for each {{completion_list_t *entry = dequeue_completion(clist)}}, 
does not assign the _mhdr_ and keeps _mhdr.done == 0_ as it was originally 
initialized. Consequently the _while (!mhdr.done)_ does not ever end, and 
finally falls into the _assert(entry)_ with _entry == NULL_ when all 
sub-requests are "completed". ~// Normally on my platform assert raises 
SIGABRT.~

I propose to instruct the _deserialize_multi()_ function to break the loop on 
_entry == NULL_ if it was called for an unsuccessfull overal status of the 
multi response, and in particular for the fake response having _ZCLOSING_ 
(-116) status. I have introduced the _rc0_ parameter for this.


*Another issue* with this function is that even if the while-loop exited 
properly, this function returns _rc == 0_, and this return code +overrides+ the 
true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in the 
_deserialize_response()_ function. So, the _multi_ response callback +handler 
would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which is strictly 
wrong.

To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
zero (which is _ZOK_ indeed).

This is a proposed fix: https://github.com/apache/zookeeper/pull/360

[upd]
It looks like about the same problem is described in ZOOKEEPER-1636
However, the patch proposed in this ticket also remedies the second linked 
problem: reporting ZCLOSING status (as required) to the multi-request 
completion handler.

  was:
Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the so 
called "Fake response" which is fabricated by the function _free_completions()_ 
for example when _zookeeper_close()_ is called while there is a pending _multi_ 
request.

Such fake response includes only the header but zero bytes for the body. Due to 
this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is called 
repeatedly for each {{completion_list_t *entry = dequeue_completion(clist)}}, 
does not assign the _mhdr_ and keeps _mhdr.done == 0_ as it was originally 
initialized. Consequently the _while (!mhdr.done)_ does not ever end, and 
finally falls into the _assert(entry)_ with _entry == NULL_ when all 
sub-requests are "completed". ~// Normally on my platform assert raises 
SIGABRT.~

I propose to instruct the _deserialize_multi()_ function to break the loop on 
_entry == NULL_ if it was called for an unsuccessfull overal status of the 
multi response, and in particular for the fake response having _ZCLOSING_ 
(-116) status. I have introduced the _rc0_ parameter for this.


*Another issue* with this function is that even if the while-loop exited 
properly, this function returns _rc == 0_, and this return code +overrides+ the 
true status value with {{rc = deserialize_multi(xid, cptr, ia, rc)}} in the 
_deserialize_response()_ function. So, the _multi_ response callback +handler 
would be called with _rc == ZOK_ instead of _rc == ZCLOSING_+ which is strictly 
wrong.

To fix this I propose initializing _rc_ with the introduced _rc0_ instead of 
zero (which is _ZOK_ indeed).

This is a proposed fix: https://github.com/apache/zookeeper/pull/360


> SIGABRT from assert during fake completion on zookeeper_close.
> --
>
> Key: ZOOKEEPER-2891
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2891
> Project: ZooKeeper
>  Issue Type: Bug
>  Components: c client
>Affects Versions: 3.4.10
> Environment: Linux ubuntu 4.4.0-87-generic
> gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
> https://github.com/apache/zookeeper.git
> branch-3.4
>Reporter: Alexander A. Strelets
>Priority: Critical
>  Labels: easyfix
> Fix For: 3.4.10
>
>
> Function *_deserialize_multi()_* hits *_assert(entry)_* when called for the 
> so called "Fake response" which is fabricated by the function 
> _free_completions()_ for example when _zookeeper_close()_ is called while 
> there is a pending _multi_ request.
> Such fake response includes only the header but zero bytes for the body. Due 
> to this {{deserialize_MultiHeader(ia, "multiheader", )}}, which is 
> called repeatedly for each {{completion_list_t *entry = 
> dequeue_completion(clist)}}, does not