[jira] Commented: (ZOOKEEPER-474) add compile, test, and improved package targets to zkperl build.xml
[ https://issues.apache.org/jira/browse/ZOOKEEPER-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12741503#action_12741503 ] Chris Darroch commented on ZOOKEEPER-474: - Nothing yet, sorry. Might be a couple of weeks before I get a chance to look; it's extra busy at work at the moment. add compile, test, and improved package targets to zkperl build.xml --- Key: ZOOKEEPER-474 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-474 Project: Zookeeper Issue Type: Improvement Components: contrib Affects Versions: 3.2.0 Reporter: Chris Darroch Assignee: Chris Darroch Fix For: 3.2.1, 3.3.0 Attachments: zk474_testout.txt, ZOOKEEPER-474.patch This patch adds compile and test targets to the zkperl build.xml, and tweaks the package target a little to use the manifest file. For me, ant compile, ant test, and ant clean all work (from scratch, in each case) when using Ant in the local src/contrib/zkperl directory. Further, ant package in the top-level directory seems to continue to build zkperl along with everything else, and leaves out the build.xml and t/zkServer.sh files, which is appropriate. From what I can see, the top-level build.xml doesn't actually invoke the test-contrib target, so I'm not sure if there's a way to integrate the zkperl tests into the main hudson automated test process, but that would be ideal, if at all possible. I feel like I've seen comments to the effect that the zkpython tests are run automatically, but I'm not sure if that's actually true or not. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-474) add compile, test, and improved package targets to zkperl build.xml
[ https://issues.apache.org/jira/browse/ZOOKEEPER-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-474: Interesting. I'll try to take a look shortly. The ironic thing is that you appear to have some intriguing but harmless compile warnings, and then the tests fail in one of the smallest tests (where we just try to work around the single log file from the C client), but pass all the complex ones! So that's mostly good news, I think. add compile, test, and improved package targets to zkperl build.xml --- Key: ZOOKEEPER-474 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-474 Project: Zookeeper Issue Type: Improvement Components: contrib Affects Versions: 3.2.0 Reporter: Chris Darroch Assignee: Chris Darroch Fix For: 3.2.1, 3.3.0 Attachments: zk474_testout.txt, ZOOKEEPER-474.patch This patch adds compile and test targets to the zkperl build.xml, and tweaks the package target a little to use the manifest file. For me, ant compile, ant test, and ant clean all work (from scratch, in each case) when using Ant in the local src/contrib/zkperl directory. Further, ant package in the top-level directory seems to continue to build zkperl along with everything else, and leaves out the build.xml and t/zkServer.sh files, which is appropriate. From what I can see, the top-level build.xml doesn't actually invoke the test-contrib target, so I'm not sure if there's a way to integrate the zkperl tests into the main hudson automated test process, but that would be ideal, if at all possible. I feel like I've seen comments to the effect that the zkpython tests are run automatically, but I'm not sure if that's actually true or not. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-482) ignore sigpipe in testRetry to avoid silent immediate failure
ignore sigpipe in testRetry to avoid silent immediate failure - Key: ZOOKEEPER-482 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-482 Project: Zookeeper Issue Type: Bug Components: c client, tests Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 The testRetry test silently exits for me periodically, especially, it seems, on newer hardware. It also spits out from log messages clutter the test output. The silent exits turn out to be because SIGPIPE is sometimes delivered during the sleep(1) in createClient(), the second time createClient() is called. Since SIGPIPE is not being ignored and there is no signal handler, the process exists immediately. This leaves the test suite in a broken state, with the test ZooKeeper process still running because zkServer.sh stop is not run by tearDown(). You have to manually kill the ZK server and retry the tests; sometimes they succeed and sometimes they don't. I described SIGPIPE handling a little in ZOOKEEPER-320. The appropriate thing, I think, is for the client application to ignore or handle SIGPIPE. In this case, that falls to the test processes. The attached patch fixes the issue for me with testRetry. The patch uses sigaction() to ignore SIGPIPE in TestClientRetry.cc and, for good measure (although I never saw it actually fail for me), TestClient.cc, since that file also uses sleep() extensively. I also removed a couple of unused functions and a macro definition from TestClientRetry.cc, just to simply matters, and turned off log output, which makes the testRetry output much, much cleaner (otherwise you get a lot of log output spamming into the nice clean cppunit output :-). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-482) ignore sigpipe in testRetry to avoid silent immediate failure
[ https://issues.apache.org/jira/browse/ZOOKEEPER-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-482: Release Note: Ignore SIGPIPE signal to C test suite does not fail suddenly. Status: Patch Available (was: Open) ignore sigpipe in testRetry to avoid silent immediate failure - Key: ZOOKEEPER-482 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-482 Project: Zookeeper Issue Type: Bug Components: c client, tests Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-482.patch The testRetry test silently exits for me periodically, especially, it seems, on newer hardware. It also spits out from log messages clutter the test output. The silent exits turn out to be because SIGPIPE is sometimes delivered during the sleep(1) in createClient(), the second time createClient() is called. Since SIGPIPE is not being ignored and there is no signal handler, the process exists immediately. This leaves the test suite in a broken state, with the test ZooKeeper process still running because zkServer.sh stop is not run by tearDown(). You have to manually kill the ZK server and retry the tests; sometimes they succeed and sometimes they don't. I described SIGPIPE handling a little in ZOOKEEPER-320. The appropriate thing, I think, is for the client application to ignore or handle SIGPIPE. In this case, that falls to the test processes. The attached patch fixes the issue for me with testRetry. The patch uses sigaction() to ignore SIGPIPE in TestClientRetry.cc and, for good measure (although I never saw it actually fail for me), TestClient.cc, since that file also uses sleep() extensively. I also removed a couple of unused functions and a macro definition from TestClientRetry.cc, just to simply matters, and turned off log output, which makes the testRetry output much, much cleaner (otherwise you get a lot of log output spamming into the nice clean cppunit output :-). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-482) ignore sigpipe in testRetry to avoid silent immediate failure
[ https://issues.apache.org/jira/browse/ZOOKEEPER-482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-482: Attachment: ZOOKEEPER-482.patch ignore sigpipe in testRetry to avoid silent immediate failure - Key: ZOOKEEPER-482 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-482 Project: Zookeeper Issue Type: Bug Components: c client, tests Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-482.patch The testRetry test silently exits for me periodically, especially, it seems, on newer hardware. It also spits out from log messages clutter the test output. The silent exits turn out to be because SIGPIPE is sometimes delivered during the sleep(1) in createClient(), the second time createClient() is called. Since SIGPIPE is not being ignored and there is no signal handler, the process exists immediately. This leaves the test suite in a broken state, with the test ZooKeeper process still running because zkServer.sh stop is not run by tearDown(). You have to manually kill the ZK server and retry the tests; sometimes they succeed and sometimes they don't. I described SIGPIPE handling a little in ZOOKEEPER-320. The appropriate thing, I think, is for the client application to ignore or handle SIGPIPE. In this case, that falls to the test processes. The attached patch fixes the issue for me with testRetry. The patch uses sigaction() to ignore SIGPIPE in TestClientRetry.cc and, for good measure (although I never saw it actually fail for me), TestClient.cc, since that file also uses sleep() extensively. I also removed a couple of unused functions and a macro definition from TestClientRetry.cc, just to simply matters, and turned off log output, which makes the testRetry output much, much cleaner (otherwise you get a lot of log output spamming into the nice clean cppunit output :-). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-471) update zkperl for 3.2.x branch
[ https://issues.apache.org/jira/browse/ZOOKEEPER-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-471: Attachment: ZOOKEEPER-471.patch Some additional improvements which will support (I hope) a Ant-based test harness (to be uploaded soon, with any luck). update zkperl for 3.2.x branch -- Key: ZOOKEEPER-471 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-471 Project: Zookeeper Issue Type: Improvement Components: contrib Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-471.patch Update the documentation, version number, and one test in the zkperl contrib module to match 3.2.0. The only code change here involves changing one authorization test to use the 'ip' instead of the now-removed 'host' scheme. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-471) update zkperl for 3.2.x branch
[ https://issues.apache.org/jira/browse/ZOOKEEPER-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-471: Attachment: (was: ZOOKEEPER-471.patch) update zkperl for 3.2.x branch -- Key: ZOOKEEPER-471 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-471 Project: Zookeeper Issue Type: Improvement Components: contrib Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-471.patch Update the documentation, version number, and one test in the zkperl contrib module to match 3.2.0. The only code change here involves changing one authorization test to use the 'ip' instead of the now-removed 'host' scheme. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-474) add compile, test, and improved package targets to zkperl build.xml
add compile, test, and improved package targets to zkperl build.xml --- Key: ZOOKEEPER-474 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-474 Project: Zookeeper Issue Type: Improvement Components: contrib Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-474.patch This patch adds compile and test targets to the zkperl build.xml, and tweaks the package target a little to use the manifest file. For me, ant compile, ant test, and ant clean all work (from scratch, in each case) when using Ant in the local src/contrib/zkperl directory. Further, ant package in the top-level directory seems to continue to build zkperl along with everything else, and leaves out the build.xml and t/zkServer.sh files, which is appropriate. From what I can see, the top-level build.xml doesn't actually invoke the test-contrib target, so I'm not sure if there's a way to integrate the zkperl tests into the main hudson automated test process, but that would be ideal, if at all possible. I feel like I've seen comments to the effect that the zkpython tests are run automatically, but I'm not sure if that's actually true or not. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-474) add compile, test, and improved package targets to zkperl build.xml
[ https://issues.apache.org/jira/browse/ZOOKEEPER-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-474: Attachment: ZOOKEEPER-474.patch add compile, test, and improved package targets to zkperl build.xml --- Key: ZOOKEEPER-474 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-474 Project: Zookeeper Issue Type: Improvement Components: contrib Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-474.patch This patch adds compile and test targets to the zkperl build.xml, and tweaks the package target a little to use the manifest file. For me, ant compile, ant test, and ant clean all work (from scratch, in each case) when using Ant in the local src/contrib/zkperl directory. Further, ant package in the top-level directory seems to continue to build zkperl along with everything else, and leaves out the build.xml and t/zkServer.sh files, which is appropriate. From what I can see, the top-level build.xml doesn't actually invoke the test-contrib target, so I'm not sure if there's a way to integrate the zkperl tests into the main hudson automated test process, but that would be ideal, if at all possible. I feel like I've seen comments to the effect that the zkpython tests are run automatically, but I'm not sure if that's actually true or not. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-468) avoid compile warning in send_auth_info()
avoid compile warning in send_auth_info() - Key: ZOOKEEPER-468 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-468 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-468.patch Older compilers may complain that rc may be used without initialization in send_auth_info(), if -Wall is specified. The fix is a simple initialization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-468) avoid compile warning in send_auth_info()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-468: Attachment: ZOOKEEPER-468.patch avoid compile warning in send_auth_info() - Key: ZOOKEEPER-468 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-468 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-468.patch Older compilers may complain that rc may be used without initialization in send_auth_info(), if -Wall is specified. The fix is a simple initialization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-468) avoid compile warning in send_auth_info()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-468: Release Note: Fix minor compile warning in send_auth_info(). Status: Patch Available (was: Open) avoid compile warning in send_auth_info() - Key: ZOOKEEPER-468 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-468 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-468.patch Older compilers may complain that rc may be used without initialization in send_auth_info(), if -Wall is specified. The fix is a simple initialization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-468) avoid compile warning in send_auth_info()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-468: Priority: Minor (was: Major) avoid compile warning in send_auth_info() - Key: ZOOKEEPER-468 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-468 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-468.patch Older compilers may complain that rc may be used without initialization in send_auth_info(), if -Wall is specified. The fix is a simple initialization. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-469) make sure CPPUNIT_CFLAGS isn't overwritten
[ https://issues.apache.org/jira/browse/ZOOKEEPER-469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-469: Attachment: ZOOKEEPER-469.patch make sure CPPUNIT_CFLAGS isn't overwritten -- Key: ZOOKEEPER-469 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-469 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-469.patch When compiling with --with-cppunit-prefix, CPPUNIT_CFLAGS is set by the AM_PATH_CPPUNIT macro. In configure.ac, it is then reset in order to set the -DZKSERVER_CMD command line argument. Instead, that argument should be added to CPPUNIT_CFLAGS so that things like a custom -I include location set by AM_PATH_CPPUNIT are not lost. Otherwise, a custom cppunit installation is not properly supported, despite the --with-cppunit-prefix option. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-469) make sure CPPUNIT_CFLAGS isn't overwritten
[ https://issues.apache.org/jira/browse/ZOOKEEPER-469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-469: Status: Patch Available (was: Open) I've pulled this from ZOOKEEPER-316 because it was getting lost there, I think, and is technically unrelated to the problem first reported there. This should be a much more straightforward fix for a problem that doesn't relate to any licensing issues for cppunit, for example. Sorry for any confusion. make sure CPPUNIT_CFLAGS isn't overwritten -- Key: ZOOKEEPER-469 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-469 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-469.patch When compiling with --with-cppunit-prefix, CPPUNIT_CFLAGS is set by the AM_PATH_CPPUNIT macro. In configure.ac, it is then reset in order to set the -DZKSERVER_CMD command line argument. Instead, that argument should be added to CPPUNIT_CFLAGS so that things like a custom -I include location set by AM_PATH_CPPUNIT are not lost. Otherwise, a custom cppunit installation is not properly supported, despite the --with-cppunit-prefix option. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-316) configure option --without-cppunit does not work
[ https://issues.apache.org/jira/browse/ZOOKEEPER-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12731012#action_12731012 ] Chris Darroch commented on ZOOKEEPER-316: - I'm moving the patch file to ZOOKEEPER-469 because it's really a separate issue; sorry for any confusion. configure option --without-cppunit does not work Key: ZOOKEEPER-316 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-316 Project: Zookeeper Issue Type: Bug Components: c client, tests Affects Versions: 3.1.0 Reporter: Mahadev konar configure option --without-cppunit does not work. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-316) configure option --without-cppunit does not work
[ https://issues.apache.org/jira/browse/ZOOKEEPER-316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-316: Attachment: (was: ZOOKEEPER-316.patch) configure option --without-cppunit does not work Key: ZOOKEEPER-316 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-316 Project: Zookeeper Issue Type: Bug Components: c client, tests Affects Versions: 3.1.0 Reporter: Mahadev konar configure option --without-cppunit does not work. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-470) include unistd.h for sleep() in c tests
[ https://issues.apache.org/jira/browse/ZOOKEEPER-470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-470: Status: Patch Available (was: Open) include unistd.h for sleep() in c tests --- Key: ZOOKEEPER-470 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-470 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-470.patch Include unistd.h for sleep() calls in C tests to ensure successful compilation on some platforms. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-470) include unistd.h for sleep() in c tests
include unistd.h for sleep() in c tests --- Key: ZOOKEEPER-470 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-470 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-470.patch Include unistd.h for sleep() calls in C tests to ensure successful compilation on some platforms. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-470) include unistd.h for sleep() in c tests
[ https://issues.apache.org/jira/browse/ZOOKEEPER-470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-470: Attachment: ZOOKEEPER-470.patch include unistd.h for sleep() in c tests --- Key: ZOOKEEPER-470 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-470 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-470.patch Include unistd.h for sleep() calls in C tests to ensure successful compilation on some platforms. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-471) update zkperl for 3.2.x branch
[ https://issues.apache.org/jira/browse/ZOOKEEPER-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-471: Attachment: ZOOKEEPER-471.patch update zkperl for 3.2.x branch -- Key: ZOOKEEPER-471 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-471 Project: Zookeeper Issue Type: Improvement Components: contrib Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-471.patch Update the documentation, version number, and one test in the zkperl contrib module to match 3.2.0. The only code change here involves changing one authorization test to use the 'ip' instead of the now-removed 'host' scheme. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-471) update zkperl for 3.2.x branch
update zkperl for 3.2.x branch -- Key: ZOOKEEPER-471 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-471 Project: Zookeeper Issue Type: Improvement Components: contrib Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-471.patch Update the documentation, version number, and one test in the zkperl contrib module to match 3.2.0. The only code change here involves changing one authorization test to use the 'ip' instead of the now-removed 'host' scheme. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-471) update zkperl for 3.2.x branch
[ https://issues.apache.org/jira/browse/ZOOKEEPER-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-471: Status: Patch Available (was: Open) update zkperl for 3.2.x branch -- Key: ZOOKEEPER-471 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-471 Project: Zookeeper Issue Type: Improvement Components: contrib Affects Versions: 3.2.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.2.1 Attachments: ZOOKEEPER-471.patch Update the documentation, version number, and one test in the zkperl contrib module to match 3.2.0. The only code change here involves changing one authorization test to use the 'ip' instead of the now-removed 'host' scheme. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-466) crash on zookeeper_close() when using auth with empty cert
crash on zookeeper_close() when using auth with empty cert -- Key: ZOOKEEPER-466 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-466 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 The free_auth_info() function calls deallocate_Buffer(auth-auth) on every element in the auth list; that function frees any memory pointed to by auth-auth.buff if that field is non-NULL. In zoo_add_auth(), when certLen is zero (or cert is NULL), auth.buff is set to 0, but then not assigned to authinfo-auth when auth.buff is NULL. The result is uninitialized data in auth-auth.buff in free_auth_info(), and potential crashes. The attached patch adds a test which attempts to duplicate this error; it works for me but may not always on all systems as it depends on the uninitialized data being non-zero; there's not really a simple way I can see to trigger this in the current test framework. The patch also fixes the problem, I believe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-466) crash on zookeeper_close() when using auth with empty cert
[ https://issues.apache.org/jira/browse/ZOOKEEPER-466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-466: Attachment: (was: ZOOKEEPER-466.patch) crash on zookeeper_close() when using auth with empty cert -- Key: ZOOKEEPER-466 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-466 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-466.patch The free_auth_info() function calls deallocate_Buffer(auth-auth) on every element in the auth list; that function frees any memory pointed to by auth-auth.buff if that field is non-NULL. In zoo_add_auth(), when certLen is zero (or cert is NULL), auth.buff is set to 0, but then not assigned to authinfo-auth when auth.buff is NULL. The result is uninitialized data in auth-auth.buff in free_auth_info(), and potential crashes. The attached patch adds a test which attempts to duplicate this error; it works for me but may not always on all systems as it depends on the uninitialized data being non-zero; there's not really a simple way I can see to trigger this in the current test framework. The patch also fixes the problem, I believe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-466) crash on zookeeper_close() when using auth with empty cert
[ https://issues.apache.org/jira/browse/ZOOKEEPER-466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-466: Attachment: ZOOKEEPER-466.patch crash on zookeeper_close() when using auth with empty cert -- Key: ZOOKEEPER-466 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-466 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-466.patch The free_auth_info() function calls deallocate_Buffer(auth-auth) on every element in the auth list; that function frees any memory pointed to by auth-auth.buff if that field is non-NULL. In zoo_add_auth(), when certLen is zero (or cert is NULL), auth.buff is set to 0, but then not assigned to authinfo-auth when auth.buff is NULL. The result is uninitialized data in auth-auth.buff in free_auth_info(), and potential crashes. The attached patch adds a test which attempts to duplicate this error; it works for me but may not always on all systems as it depends on the uninitialized data being non-zero; there's not really a simple way I can see to trigger this in the current test framework. The patch also fixes the problem, I believe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-466) crash on zookeeper_close() when using auth with empty cert
[ https://issues.apache.org/jira/browse/ZOOKEEPER-466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-466: Attachment: ZOOKEEPER-466.patch crash on zookeeper_close() when using auth with empty cert -- Key: ZOOKEEPER-466 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-466 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-466.patch The free_auth_info() function calls deallocate_Buffer(auth-auth) on every element in the auth list; that function frees any memory pointed to by auth-auth.buff if that field is non-NULL. In zoo_add_auth(), when certLen is zero (or cert is NULL), auth.buff is set to 0, but then not assigned to authinfo-auth when auth.buff is NULL. The result is uninitialized data in auth-auth.buff in free_auth_info(), and potential crashes. The attached patch adds a test which attempts to duplicate this error; it works for me but may not always on all systems as it depends on the uninitialized data being non-zero; there's not really a simple way I can see to trigger this in the current test framework. The patch also fixes the problem, I believe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-466) crash on zookeeper_close() when using auth with empty cert
[ https://issues.apache.org/jira/browse/ZOOKEEPER-466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-466: Release Note: Fixes crash in zookeeper_close() when zoo_add_auth() used with empty cert string. Status: Patch Available (was: Open) crash on zookeeper_close() when using auth with empty cert -- Key: ZOOKEEPER-466 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-466 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-466.patch The free_auth_info() function calls deallocate_Buffer(auth-auth) on every element in the auth list; that function frees any memory pointed to by auth-auth.buff if that field is non-NULL. In zoo_add_auth(), when certLen is zero (or cert is NULL), auth.buff is set to 0, but then not assigned to authinfo-auth when auth.buff is NULL. The result is uninitialized data in auth-auth.buff in free_auth_info(), and potential crashes. The attached patch adds a test which attempts to duplicate this error; it works for me but may not always on all systems as it depends on the uninitialized data being non-zero; there's not really a simple way I can see to trigger this in the current test framework. The patch also fixes the problem, I believe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-466) crash on zookeeper_close() when using auth with empty cert
[ https://issues.apache.org/jira/browse/ZOOKEEPER-466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-466: Attachment: ZOOKEEPER-466.patch crash on zookeeper_close() when using auth with empty cert -- Key: ZOOKEEPER-466 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-466 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-466.patch, ZOOKEEPER-466.patch The free_auth_info() function calls deallocate_Buffer(auth-auth) on every element in the auth list; that function frees any memory pointed to by auth-auth.buff if that field is non-NULL. In zoo_add_auth(), when certLen is zero (or cert is NULL), auth.buff is set to 0, but then not assigned to authinfo-auth when auth.buff is NULL. The result is uninitialized data in auth-auth.buff in free_auth_info(), and potential crashes. The attached patch adds a test which attempts to duplicate this error; it works for me but may not always on all systems as it depends on the uninitialized data being non-zero; there's not really a simple way I can see to trigger this in the current test framework. The patch also fixes the problem, I believe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-466) crash on zookeeper_close() when using auth with empty cert
[ https://issues.apache.org/jira/browse/ZOOKEEPER-466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-466: Attachment: (was: ZOOKEEPER-466.patch) crash on zookeeper_close() when using auth with empty cert -- Key: ZOOKEEPER-466 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-466 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.2.0 Reporter: Chris Darroch Fix For: 3.2.1 Attachments: ZOOKEEPER-466.patch The free_auth_info() function calls deallocate_Buffer(auth-auth) on every element in the auth list; that function frees any memory pointed to by auth-auth.buff if that field is non-NULL. In zoo_add_auth(), when certLen is zero (or cert is NULL), auth.buff is set to 0, but then not assigned to authinfo-auth when auth.buff is NULL. The result is uninitialized data in auth-auth.buff in free_auth_info(), and potential crashes. The attached patch adds a test which attempts to duplicate this error; it works for me but may not always on all systems as it depends on the uninitialized data being non-zero; there's not really a simple way I can see to trigger this in the current test framework. The patch also fixes the problem, I believe. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-311) handle small path lengths in zoo_create()
[
https://issues.apache.org/jira/browse/ZOOKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Darroch updated ZOOKEEPER-311:
Attachment: (was: ZOOKEEPER-311.patch)
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.0.0, 3.0.1, 3.1.0
Reporter: Chris Darroch
Assignee: Chris Darroch
Priority: Minor
Fix For: 4.0.0
Attachments: ZOOKEEPER-311.patch, ZOOKEEPER-311.patch
The synchronous completion for zoo_create() contains the following code:\\
{noformat}
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
{noformat}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but
again, nothing will happen (even if strlen(res.path) is 0, which is unlikely
since new node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is
its description (the maximum length of real path you would want) in
zookeeper.h, and the example usage in the Programmer's Guide:
{noformat}
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL,
buffer, sizeof(buffer)-1);
{noformat}
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last
byte, leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to
zoo_set(). The path_buffer_len would be treated as the full length of the
buffer (as the code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-311) handle small path lengths in zoo_create()
[
https://issues.apache.org/jira/browse/ZOOKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Darroch updated ZOOKEEPER-311:
Attachment: ZOOKEEPER-311.patch
Updated to current trunk, with some tests added for extra crunchy goodness.
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.0.0, 3.0.1, 3.1.0, 3.1.1, 3.2.0
Reporter: Chris Darroch
Assignee: Chris Darroch
Priority: Minor
Fix For: 3.2.1
Attachments: ZOOKEEPER-311.patch, ZOOKEEPER-311.patch
The synchronous completion for zoo_create() contains the following code:\\
{noformat}
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
{noformat}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but
again, nothing will happen (even if strlen(res.path) is 0, which is unlikely
since new node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is
its description (the maximum length of real path you would want) in
zookeeper.h, and the example usage in the Programmer's Guide:
{noformat}
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL,
buffer, sizeof(buffer)-1);
{noformat}
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last
byte, leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to
zoo_set(). The path_buffer_len would be treated as the full length of the
buffer (as the code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-311) handle small path lengths in zoo_create()
[
https://issues.apache.org/jira/browse/ZOOKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Darroch updated ZOOKEEPER-311:
Attachment: (was: ZOOKEEPER-311.patch)
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.0.0, 3.0.1, 3.1.0, 3.1.1, 3.2.0
Reporter: Chris Darroch
Assignee: Chris Darroch
Priority: Minor
Fix For: 3.2.1
Attachments: ZOOKEEPER-311.patch, ZOOKEEPER-311.patch
The synchronous completion for zoo_create() contains the following code:\\
{noformat}
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
{noformat}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but
again, nothing will happen (even if strlen(res.path) is 0, which is unlikely
since new node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is
its description (the maximum length of real path you would want) in
zookeeper.h, and the example usage in the Programmer's Guide:
{noformat}
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL,
buffer, sizeof(buffer)-1);
{noformat}
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last
byte, leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to
zoo_set(). The path_buffer_len would be treated as the full length of the
buffer (as the code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-311) handle small path lengths in zoo_create()
[
https://issues.apache.org/jira/browse/ZOOKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Darroch updated ZOOKEEPER-311:
Fix Version/s: (was: 4.0.0)
3.2.1
Affects Version/s: 3.1.1
3.2.0
Release Note: The zoo_create() function is revised to take path_buffer
and path_buffer_len arguments. The path_buffer will be filled with as much of
the actual path of the new node fits into the buffer, including a null
terminator. Unlike previous behavior, if path_buffer_len (formerly
max_realpath_len) is 1, the buffer will be filled with a single null
terminator. (was: The zoo_create() function is revised to take path_buffer and
path_buffer_len arguments. The path_buffer will be filled with as much of the
actual path of the new node fits into the buffer, including a null terminator.
Unlike 3.x behaviour, if path_buffer_len (formerly max_realpath_len) is 1, the
buffer will be filled with a single null terminator.)
Status: Patch Available (was: Open)
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.2.0, 3.1.1, 3.1.0, 3.0.1, 3.0.0
Reporter: Chris Darroch
Assignee: Chris Darroch
Priority: Minor
Fix For: 3.2.1
Attachments: ZOOKEEPER-311.patch, ZOOKEEPER-311.patch
The synchronous completion for zoo_create() contains the following code:\\
{noformat}
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
{noformat}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but
again, nothing will happen (even if strlen(res.path) is 0, which is unlikely
since new node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is
its description (the maximum length of real path you would want) in
zookeeper.h, and the example usage in the Programmer's Guide:
{noformat}
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL,
buffer, sizeof(buffer)-1);
{noformat}
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last
byte, leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to
zoo_set(). The path_buffer_len would be treated as the full length of the
buffer (as the code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-311) handle small path lengths in zoo_create()
[
https://issues.apache.org/jira/browse/ZOOKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Darroch updated ZOOKEEPER-311:
Attachment: ZOOKEEPER-311.patch
This revision is a good demonstration of why one should never trust a
programmer, especially if there's a degree of hubris involved (It's just a
small patch, what could go wrong?) In testing, I found I had an off-by-one
error when calculating where to put the trailing null. Sigh.
I agree this shouldn't be committed without some tests. I'm at virtually 100%
capacity at the moment with just my paid work so it may take me a while to do
this; it is my next to-do, though. If someone else wants to write a few tests
in the meantime, that would also be great.
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.0.0, 3.0.1, 3.1.0
Reporter: Chris Darroch
Assignee: Chris Darroch
Priority: Minor
Fix For: 4.0.0
Attachments: ZOOKEEPER-311.patch, ZOOKEEPER-311.patch,
ZOOKEEPER-311.patch
The synchronous completion for zoo_create() contains the following code:\\
{noformat}
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
{noformat}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but
again, nothing will happen (even if strlen(res.path) is 0, which is unlikely
since new node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is
its description (the maximum length of real path you would want) in
zookeeper.h, and the example usage in the Programmer's Guide:
{noformat}
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL,
buffer, sizeof(buffer)-1);
{noformat}
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last
byte, leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to
zoo_set(). The path_buffer_len would be treated as the full length of the
buffer (as the code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-378) contribution of Net::ZooKeeper Perl module
[ https://issues.apache.org/jira/browse/ZOOKEEPER-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12704428#action_12704428 ] Chris Darroch commented on ZOOKEEPER-378: - Thanks for the commit! I'll try to submit patch updates to keep in sync with any C API changes. contribution of Net::ZooKeeper Perl module -- Key: ZOOKEEPER-378 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-378 Project: Zookeeper Issue Type: New Feature Reporter: Chris Darroch Assignee: Chris Darroch Fix For: 3.2.0 Attachments: ZOOKEEPER-378.patch, ZOOKEEPER-378.patch As per the conversation in ZOOKEEPER-364, I submitted a Software Grant Agreement to the ASF last week for Net::ZooKeeper by fax and surface mail; it might take a few more days for it to show up in the private grants.txt file, I believe. I also created a Net::ZooKeeper package, version 0.33, with licenses and a NOTICE file as required by the ASF for any ASF project. It is attached to this issue and also available at http://people.apache.org/~chrisd/projects/net_zookeeper/. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-378) contribution of Net::ZooKeeper Perl module
[ https://issues.apache.org/jira/browse/ZOOKEEPER-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-378: Attachment: (was: Net-ZooKeeper-0.33.tar.gz) contribution of Net::ZooKeeper Perl module -- Key: ZOOKEEPER-378 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-378 Project: Zookeeper Issue Type: New Feature Affects Versions: 3.0.0, 3.0.1, 3.1.0, 3.1.1 Reporter: Chris Darroch Assignee: Chris Darroch Attachments: ZOOKEEPER-378.patch As per the conversation in ZOOKEEPER-364, I submitted a Software Grant Agreement to the ASF last week for Net::ZooKeeper by fax and surface mail; it might take a few more days for it to show up in the private grants.txt file, I believe. I also created a Net::ZooKeeper package, version 0.33, with licenses and a NOTICE file as required by the ASF for any ASF project. It is attached to this issue and also available at http://people.apache.org/~chrisd/projects/net_zookeeper/. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-378) contribution of Net::ZooKeeper Perl module
[ https://issues.apache.org/jira/browse/ZOOKEEPER-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-378: Attachment: ZOOKEEPER-378.patch Patch file against trunk. contribution of Net::ZooKeeper Perl module -- Key: ZOOKEEPER-378 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-378 Project: Zookeeper Issue Type: New Feature Affects Versions: 3.0.0, 3.0.1, 3.1.0, 3.1.1 Reporter: Chris Darroch Assignee: Chris Darroch Attachments: ZOOKEEPER-378.patch As per the conversation in ZOOKEEPER-364, I submitted a Software Grant Agreement to the ASF last week for Net::ZooKeeper by fax and surface mail; it might take a few more days for it to show up in the private grants.txt file, I believe. I also created a Net::ZooKeeper package, version 0.33, with licenses and a NOTICE file as required by the ASF for any ASF project. It is attached to this issue and also available at http://people.apache.org/~chrisd/projects/net_zookeeper/. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-378) contribution of Net::ZooKeeper Perl module
[ https://issues.apache.org/jira/browse/ZOOKEEPER-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12702517#action_12702517 ] Chris Darroch commented on ZOOKEEPER-378: - Down the road a little, it would be nice to get the build.xml file to do a little more; e.g., build after the C client libs are built, and then run ZK_TEST_HOSTS=localhost:22181 make test after running src/c/tests/zkServer.sh startClean. Probably worth a separate Jira issue, though. contribution of Net::ZooKeeper Perl module -- Key: ZOOKEEPER-378 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-378 Project: Zookeeper Issue Type: New Feature Affects Versions: 3.2.0, 4.0.0 Reporter: Chris Darroch Assignee: Chris Darroch Attachments: ZOOKEEPER-378.patch As per the conversation in ZOOKEEPER-364, I submitted a Software Grant Agreement to the ASF last week for Net::ZooKeeper by fax and surface mail; it might take a few more days for it to show up in the private grants.txt file, I believe. I also created a Net::ZooKeeper package, version 0.33, with licenses and a NOTICE file as required by the ASF for any ASF project. It is attached to this issue and also available at http://people.apache.org/~chrisd/projects/net_zookeeper/. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-378) contribution of Net::ZooKeeper Perl module
[ https://issues.apache.org/jira/browse/ZOOKEEPER-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-378: Affects Version/s: (was: 3.1.1) (was: 3.0.1) (was: 3.1.0) (was: 3.0.0) 4.0.0 3.2.0 Release Note: Net::ZooKeeper Perl module included under src/contribs/zkperl. Status: Patch Available (was: Open) Contribution of Net::ZooKeeper Perl module version 0.33. contribution of Net::ZooKeeper Perl module -- Key: ZOOKEEPER-378 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-378 Project: Zookeeper Issue Type: New Feature Affects Versions: 3.2.0, 4.0.0 Reporter: Chris Darroch Assignee: Chris Darroch Attachments: ZOOKEEPER-378.patch As per the conversation in ZOOKEEPER-364, I submitted a Software Grant Agreement to the ASF last week for Net::ZooKeeper by fax and surface mail; it might take a few more days for it to show up in the private grants.txt file, I believe. I also created a Net::ZooKeeper package, version 0.33, with licenses and a NOTICE file as required by the ASF for any ASF project. It is attached to this issue and also available at http://people.apache.org/~chrisd/projects/net_zookeeper/. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-378) contribution of Net::ZooKeeper Perl module
contribution of Net::ZooKeeper Perl module -- Key: ZOOKEEPER-378 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-378 Project: Zookeeper Issue Type: New Feature Affects Versions: 3.2.0, 4.0.0 Reporter: Chris Darroch Attachments: Net-ZooKeeper-0.33.tar.gz As per the conversation in ZOOKEEPER-364, I submitted a Software Grant Agreement to the ASF last week for Net::ZooKeeper by fax and surface mail; it might take a few more days for it to show up in the private grants.txt file, I believe. I also created a Net::ZooKeeper package, version 0.33, with licenses and a NOTICE file as required by the ASF for any ASF project. It is attached to this issue and also available at http://people.apache.org/~chrisd/projects/net_zookeeper/. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-378) contribution of Net::ZooKeeper Perl module
[ https://issues.apache.org/jira/browse/ZOOKEEPER-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12701020#action_12701020 ] Chris Darroch commented on ZOOKEEPER-378: - Glad it looks OK. My only comment is that I'm fairly strongly attached to the Net::ZooKeeper name because it conforms to the well-established Perl module naming scheme and has been approved by the CPAN community. By all means check it into SVN under a zkperl directory or whatever you like; if you want to change the module name, though, we should have a conversation about that. But I think you're just talking about the directory name in SVN. :-) Cheers. contribution of Net::ZooKeeper Perl module -- Key: ZOOKEEPER-378 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-378 Project: Zookeeper Issue Type: New Feature Affects Versions: 3.0.0, 3.0.1, 3.1.0, 3.1.1 Reporter: Chris Darroch Assignee: Chris Darroch Attachments: Net-ZooKeeper-0.33.tar.gz As per the conversation in ZOOKEEPER-364, I submitted a Software Grant Agreement to the ASF last week for Net::ZooKeeper by fax and surface mail; it might take a few more days for it to show up in the private grants.txt file, I believe. I also created a Net::ZooKeeper package, version 0.33, with licenses and a NOTICE file as required by the ASF for any ASF project. It is attached to this issue and also available at http://people.apache.org/~chrisd/projects/net_zookeeper/. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-316) configure option --without-cppunit does not work
[ https://issues.apache.org/jira/browse/ZOOKEEPER-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12699766#action_12699766 ] Chris Darroch commented on ZOOKEEPER-316: - One step forward (not a complete solution) would be to add cppunit.m4 to svn. For example, src/c/m4/cppunit.m4 can then be included when building from scratch with trunk using aclocal -I m4. That gets rid of the undefined macro error for AM_PATH_CPPUNIT if you don't have cppunit installed in a default location. configure option --without-cppunit does not work Key: ZOOKEEPER-316 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-316 Project: Zookeeper Issue Type: Bug Components: c client, tests Affects Versions: 3.1.0 Reporter: Mahadev konar configure option --without-cppunit does not work. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-316) configure option --without-cppunit does not work
[ https://issues.apache.org/jira/browse/ZOOKEEPER-316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-316: Attachment: ZOOKEEPER-316.patch This patch makes sure CPPUNIT_CFLAGS isn't overwritten; if one has cppunit in a non-standard location it's important to have the -I flags passed into the compiler. configure option --without-cppunit does not work Key: ZOOKEEPER-316 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-316 Project: Zookeeper Issue Type: Bug Components: c client, tests Affects Versions: 3.1.0 Reporter: Mahadev konar Attachments: ZOOKEEPER-316.patch configure option --without-cppunit does not work. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-316) configure option --without-cppunit does not work
[ https://issues.apache.org/jira/browse/ZOOKEEPER-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12699774#action_12699774 ] Chris Darroch commented on ZOOKEEPER-316: - I don't know about cppunit.m4, but AM_PATH_CPPUNIT isn't available without it -- maybe it's installed by default on some systems, but not any of mine, at least. configure option --without-cppunit does not work Key: ZOOKEEPER-316 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-316 Project: Zookeeper Issue Type: Bug Components: c client, tests Affects Versions: 3.1.0 Reporter: Mahadev konar Attachments: ZOOKEEPER-316.patch configure option --without-cppunit does not work. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-36) REST access to ZooKeeper
[ https://issues.apache.org/jira/browse/ZOOKEEPER-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12699514#action_12699514 ] Chris Darroch commented on ZOOKEEPER-36: I took a quick look at the spec; nice! I like that it follows the spirit of the HTTP spec in the way I envisioned. A couple of minor thoughts: I think it would be nice to avoid POST entirely, if only because POST is so over-used that it no longer has any particular meaning except send stuff to server. Suppose you used the trailing slash in the URI to distinguish meanings? Having hacked on mod_dir at various points this occurs to me as a possibility. For example: PUT /foo - zoo_set(/foo) PUT /foo/ - zoo_create(/foo) The same would work to overload GET: GET /foo - zoo_get(/foo) GET /foo/ - zoo_get_children(/foo) I especially like the GET cases because they correspond nicely to how URIs are mapped to files and directories. As for running code, I'm working (slowly) on a mod_zookeeper and you'll be the first to know if I get it to a releasable state. The basic purpose is similar to mod_dbd: provide connections that other modules can use however they choose. REST access to ZooKeeper Key: ZOOKEEPER-36 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-36 Project: Zookeeper Issue Type: New Feature Components: contrib Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.2.0 Attachments: rest_2.tar.gz, SPEC.txt Moved from SourceForge to Apache. http://sourceforge.net/tracker/index.php?func=detailaid=1961763group_id=209147atid=1008547 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-311) handle small path lengths in zoo_create()
[
https://issues.apache.org/jira/browse/ZOOKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12697996#action_12697996
]
Chris Darroch commented on ZOOKEEPER-311:
-
Mostly because the existing logic is a little counter-intuitive. I spent quite
a bit of time looking at it while trying to determine it's behaviour in various
cases --- e.g., what happens if my buffer is 1 byte long? Because I was
writing a wrapper library (Net::ZooKeeper) and not a client application, I
didn't know what max path len the user might supply and I wanted to ensure
consistent behaviour.
Typical C functions that take a buffer and fill it (e.g., snprintf()) rarely do
nothing if the buffer is 1 byte long; instead, they fill it with a null
terminator. So that's surprising and counterintuitive.
What I ended up doing in Net::ZooKeeper was preventing users from supplying
max path len values less than 2.
My other concern is really just about future-proofing. Suppose in a future
release ZooKeeper returns null paths from create() calls for some reason ---
because of some new feature (virtual nodes?) or whatever. Anyway, in this
case, the user might reasonably expect a 1-byte buffer to be (marginally)
useful. Or perhaps the code gets copied and re-used in another context where
1-byte buffers make more sense.
The code rewrite, although simple, I think clarifies what's going on in a way
that will help prevent buffer problems in the future --- always a sore point
when coding in C. First we calculate the len of the buffer that would be
required for the full path, including space for the null terminator. If that's
longer than what we've been given, we truncate to what we have. If there's
anything to do at this point (i.e., the supplied buffer had at least one byte),
then we do it.
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.0.0, 3.0.1, 3.1.0
Reporter: Chris Darroch
Assignee: Chris Darroch
Priority: Minor
Fix For: 4.0.0
Attachments: ZOOKEEPER-311.patch, ZOOKEEPER-311.patch
The synchronous completion for zoo_create() contains the following code:\\
{noformat}
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
{noformat}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but
again, nothing will happen (even if strlen(res.path) is 0, which is unlikely
since new node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is
its description (the maximum length of real path you would want) in
zookeeper.h, and the example usage in the Programmer's Guide:
{noformat}
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL,
buffer, sizeof(buffer)-1);
{noformat}
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last
byte, leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to
zoo_set(). The path_buffer_len would be treated as the full length of the
buffer (as the code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-364) command line interface for zookeeper.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12697997#action_12697997 ] Chris Darroch commented on ZOOKEEPER-364: - Well (and this should move to a new Jira issue, I guess) if you're willing to adopt Net::ZooKeeper I'm happy to fill out a Software Grant next week and submit that. I don't think there should be any license or copyright issues as I already have a ICLA on file and all the work is exclusively mine. command line interface for zookeeper. - Key: ZOOKEEPER-364 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-364 Project: Zookeeper Issue Type: New Feature Affects Versions: 3.0.0, 3.0.1, 3.1.0, 3.1.1 Reporter: Mahadev konar Fix For: 3.2.0 currently we have a shell based interface for zookeeper (which again isnt well published). we should have a well published cli based interface for zookeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-364) command line interface for zookeeper.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12695668#action_12695668 ] Chris Darroch commented on ZOOKEEPER-364: - Just FYI on the Perl side of things, CPAN is just a distribution channel -- the Perl module could easily be donated to the ASF and shipped as part of ZooKeeper, if there was interest. Since Perl users tend to look at CPAN first, though, I wanted to put it there as well. Things like mod_perl are generally available from both apache.org and cpan.org but maintained by the ASF. Since most Linux/Unix folks will have Perl installed by default -- and Net::ZooKeeper really is only going to be simple to build on Linux/Unix anyway, because of the pthread requirement in the C API -- it's not hard to build a command-line interface; in fact, pretty simple. Ephemerals, sequences, watches, and ACLs all are supported by the module. That said, personally, I think lots of people will want a Python binding as well -- I just happened to need a Perl one first. I caution that building it took more of my time than I expected; getting things like watches to work took a number of tries. Because the multi-threaded C API runs two private threads in the background to handle IO (especially pings), you can't allow them to just arbitrarily make callbacks upon a watch event notification into Perl code -- that could fry the interpreter's state over in the main thread. And you want to be sure that if the user decides to abandon some higher-level object that represents the watch (e.g., lets it go out of scope before the event notification occurs) then when the watch event does come in you haven't thrown away the private structure the callback is expecting to update. There may be a niftier way to handle this with Python since Python supports threading better than Perl 5.x, by using the single-threaded stub adapter and doing all the calls to zookeeper_interest(), zookeeper_process(), etc. from the Python module, i.e., implementing the event loops in Python threads spawned by the module. Still, it's not a cakewalk, I suspect. (I was kidding about the Parrot implementation, but I'm also intrigued by the possibility that 5 years from now these sorts of conversations might be moot, if one could write a module in Parrot code that implemented the event loop and was then usable with Parrot-compiled versions of Python, Ruby, Perl, etc. When time permits (ha!) I've been meaning to pull down Parrot 1.0 and peek at its concurrency and extension mechanisms.) command line interface for zookeeper. - Key: ZOOKEEPER-364 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-364 Project: Zookeeper Issue Type: New Feature Affects Versions: 3.0.0, 3.0.1, 3.1.0, 3.1.1 Reporter: Mahadev konar Fix For: 3.2.0 currently we have a shell based interface for zookeeper (which again isnt well published). we should have a well published cli based interface for zookeeper. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-36) REST access to ZooKeeper
[ https://issues.apache.org/jira/browse/ZOOKEEPER-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12695669#action_12695669 ] Chris Darroch commented on ZOOKEEPER-36: Another option might be the expand on the mod_shmap and mod_socache_zookeeper httpd modules I wrote a while back. The latter maintains a ZooKeeper client connection for each httpd child process -- these are shared across all HTTP requests handled by the process, so (as with the code attached to this issue, I think) ephemeral nodes aren't supported, nor are ACLs, watches, etc. The code is available under the Apache license at http://people.apache.org/~chrisd/projects/shared_map/. The shared-map module can harness a variety of small object cache providers to various parts of the URL namespace and then perform GET/PUT/DELETE against them. For the mod_socache_zookeeper provider these map to zoo_get(), zoo_set()/zoo_create(), and zoo_delete(). Nodes are created automatically when a PUT is made for a non-extant node. I need to refactor mod_socache_zookeeper and create a mod_zookeeper which deals with the business of starting/stopping ZooKeeper connections for each httpd child process, something like mod_dbd does for SQL DB connections. That will allow other modules to then acquire the ZK connection and make zoo_*() requests directly; mod_socache_zookeeper and mod_slotmem_zookeeper (yet to be written) then just devolve into the business of mapping URLs to specific ZK calls. For a REST-style interface that supported things like ACLs, sequences, stat data, etc. one could write a separate module (mod_zookeeper_rest or whatever) which supports a more complex mapping than is available through just the socache or slotmem APIs. However the REST interface is implemented, it would be nice, I think, to use HEAD - zoo_exists(), GET - zoo_get(), PUT - zoo_set()/zoo_create(), and DELETE - zoo_delete(). There's such a natural mapping of HTTP methods to ZK methods that it would seem to call out for use, as opposed to using bags of CGI arguments to POST requests or what have you. REST access to ZooKeeper Key: ZOOKEEPER-36 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-36 Project: Zookeeper Issue Type: New Feature Components: contrib Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.2.0 Attachments: rest_2.tar.gz Moved from SourceForge to Apache. http://sourceforge.net/tracker/index.php?func=detailaid=1961763group_id=209147atid=1008547 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-36) REST access to ZooKeeper
[
https://issues.apache.org/jira/browse/ZOOKEEPER-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12695670#action_12695670
]
Chris Darroch commented on ZOOKEEPER-36:
Sorry, just to throw in a couple of additional thoughts; using httpd modules
means all you need is a conventional Apache httpd instance. Requests look like
(using the current minimalistic shmap/socache modules):
{noformat}
GET /node1/node2 HTTP/1.0
DELETE /node1/node2 HTTP/1.0
PUT /node1/node3 HTTP/1.0
Content-Length: 5
12345
{noformat}
REST access to ZooKeeper
Key: ZOOKEEPER-36
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-36
Project: Zookeeper
Issue Type: New Feature
Components: contrib
Reporter: Patrick Hunt
Assignee: Patrick Hunt
Fix For: 3.2.0
Attachments: rest_2.tar.gz
Moved from SourceForge to Apache.
http://sourceforge.net/tracker/index.php?func=detailaid=1961763group_id=209147atid=1008547
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-78) added a high level protocol/feature - for easy Leader Election or exclusive Write Lock creation
[
https://issues.apache.org/jira/browse/ZOOKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12694793#action_12694793
]
Chris Darroch commented on ZOOKEEPER-78:
Just took a quick peek to see what would be either rework these in a
ZooKeeper::Lock Perl module to go along with Net::ZooKeeper, or to expose the C
version through Net::ZooKeeper. I may well be missing something after such a
brief review, but in zoo_mutex_operation() I would be inclined to add some
return value checks.
For example, if zoo_create() here returns a failure code, I believe it's
possible for the retbuf to contain whatever random data it had before the call.
In that case, getName() might succeed if the data contains a '/' and return
a string --- possibly one that doesn't happen to be null-terminated within the
retbuf buffer.
{noformat}
char retbuf[len+20];
snprintf(buf, len, %s/%s, path, prefix);
ret = zoo_create(zh, buf, NULL, 0, mutex-acl,
ZOO_EPHEMERAL|ZOO_SEQUENCE, retbuf, (len+20));
mutex-id = getName(retbuf);
{noformat}
There are some calls to zoo_get_children(), etc., which I think might be
usefully checked for failure return codes as well. Sorry not to provide a
patch yet.
added a high level protocol/feature - for easy Leader Election or exclusive
Write Lock creation
---
Key: ZOOKEEPER-78
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-78
Project: Zookeeper
Issue Type: New Feature
Components: java client
Affects Versions: 3.0.0
Reporter: james strachan
Assignee: james strachan
Fix For: 3.2.0
Attachments: patch_with_including_Benjamin's_fix.patch,
using_zookeeper_facade.patch, ZOOKEEPER-78.patch, ZOOKEEPER-78.patch,
ZOOKEEPER-78.patch, ZOOKEEPER-78.patch
Here's a patch which adds a little WriteLock helper class for performing
leader elections or creating exclusive locks in some directory znode. Note
its an early cut; am sure we can improve it over time. The aim is to avoid
folks having to use the low level ZK stuff but provide a simpler high level
abstraction.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-343) add tests that specifically verify the zkmain and qpmain classes
[ https://issues.apache.org/jira/browse/ZOOKEEPER-343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12685414#action_12685414 ] Chris Darroch commented on ZOOKEEPER-343: - Great patch; thank you -- this would have helped me avoid the regression from ZOOKEEPER-326/330/341. I ran the unit tests after ZOOKEEPER-330 and everything looked good; there was one non-reproducible error which I went away after the first test run and which I chalked up to my ancient CPU's slow speed. A classic case of PEBKAC where the computer knew better than I did that there really was a problem. add tests that specifically verify the zkmain and qpmain classes Key: ZOOKEEPER-343 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-343 Project: Zookeeper Issue Type: Improvement Components: tests Reporter: Patrick Hunt Assignee: Patrick Hunt Fix For: 3.2.0 Attachments: ZOOKEEPER-343.patch We are missing tests for these two main() routines. Add tests that verify standalone and quorum (2 servers is probably enough) by starting and connecting a client. Use on-disk configuration files to configure these. (ie verify starting with actual config files) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-333) helgrind thread issues identified in mt c client code
[
https://issues.apache.org/jira/browse/ZOOKEEPER-333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12679629#action_12679629
]
Chris Darroch commented on ZOOKEEPER-333:
-
Some of these look to be innocuous on a first glance, such as wait called with
un-held mutex in wait_sync_completion(). I suppose the issue here might be
that wait_sync_completion() doesn't check the return value from
pthread_mutex_lock() before proceeding, but in practice I believe one can
reasonably assume it always succeeds.
One thing that might help a bit would be to have a reference to the C code
which produced this output because it refers to line numbers in the code. Was
the run against 3.1.0 or against a snapshot from trunk? If the latter, do you
know the SVN revision number?
helgrind thread issues identified in mt c client code
-
Key: ZOOKEEPER-333
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-333
Project: Zookeeper
Issue Type: Bug
Components: c client
Reporter: Patrick Hunt
Assignee: Mahadev konar
Priority: Critical
Fix For: 3.1.1, 3.2.0
Attachments: helgrind_mt.out
helgrind generated a number of issues, I pulled a bunch of them. Most are
related to the test, some are really issues with the mt zk client code though:
valgrind --tool=helgrind --log-file=helgrind_mt.out ./zktest-mt
==31294== Thread #2: pthread_cond_{timed}wait called with un-held mutex
==31294==at 0x4027F8F: pthread_cond_w...@* (hg_intercepts.c:560)
==31294==by 0x404D881: pthread_cond_w...@glibc_2.0 (in
/lib/tls/i686/cmov/libpthread-2.8.90.so)
==31294==by 0x4028037: pthread_cond_w...@* (hg_intercepts.c:574)
==31294==by 0x809EBB7: pthread_cond_wait (PthreadMocks.cc:54)
==31294==by 0x80ABCF6: notify_thread_ready (mt_adaptor.c:136)
==31294==by 0x80ABE90: do_io (mt_adaptor.c:277)
==31294== Possible data race during write of size 4 at 0x42E9A58
==31294==at 0x8050D83: terminateZookeeperThreads(_zhandle*)
(ZKMocks.cc:518)
==31294==by 0x805543B: DeliverWatchersWrapper::call(_zhandle*, int, int,
char const*, watcher_object_list**) (ZKMocks.cc:261)
==31294==by 0x80520F7: __wrap_deliverWatchers (ZKMocks.cc:220)
==31294==by 0x80A287B: process_completions (zookeeper.c:1393)
==31294==by 0x80ABDAA: do_completion (mt_adaptor.c:332)
==31294== Possible data race during write of size 4 at 0xBEFF5F30
==31294==at 0x80589AF:
Zookeeper_watchers::ConnectionWatcher::~ConnectionWatcher()
(TestWatchers.cc:54)
==31294==by 0x805D062: Zookeeper_watchers::testDefaultSessionWatcher1()
(TestWatchers.cc:438)
==31294==by 0x805608C: CppUnit::TestCallerZookeeper_watchers::runTest()
(TestCaller.h:166)
==31294== Possible data race during write of size 4 at 0x42EB104
==31294==at 0x80A03EE: queue_completion (zookeeper.c:1776)
==31294==by 0x80A3A44: zookeeper_process (zookeeper.c:1598)
==31294==by 0x80AC00B: do_io (mt_adaptor.c:309)
==31294== Thread #29: pthread_cond_{timed}wait called with un-held mutex
==31294==at 0x4027F8F: pthread_cond_w...@* (hg_intercepts.c:560)
==31294==by 0x404D881: pthread_cond_w...@glibc_2.0 (in
/lib/tls/i686/cmov/libpthread-2.8.90.so)
==31294==by 0x4028037: pthread_cond_w...@* (hg_intercepts.c:574)
==31294==by 0x809EBB7: pthread_cond_wait (PthreadMocks.cc:54)
==31294==by 0x80AB9B3: wait_sync_completion (mt_adaptor.c:82)
==31294==by 0x80A1E82: zoo_wget (zookeeper.c:2517)
==31294==by 0x80A1F13: zoo_get (zookeeper.c:2497)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-332) c client issues (memory leaks) reported by valgrind
[ https://issues.apache.org/jira/browse/ZOOKEEPER-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12678428#action_12678428 ] Chris Darroch commented on ZOOKEEPER-332: - These look OK to me, on a first glance. There's a call to freeaddrinfo() in getaddrs() to match getaddrinfo(). There's also a free(zh) in zookeeper_close() to match the allocation in zookeeper_init(). I think these are just spurious reports from valgrind, but I'm not absolutely 100% on that. c client issues (memory leaks) reported by valgrind --- Key: ZOOKEEPER-332 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-332 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.1.0 Reporter: Patrick Hunt Priority: Blocker Fix For: 3.1.1, 3.2.0 Attachments: valgrind_mt.out, valgrind_st.out Attaching valgrind log files. 1) getpwuid_r doesn't seem like it's due to us 2) the rest seem to be valid -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-330) zookeeper standalone server does not startup with just a port and datadir.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-330: Attachment: ZOOKEEPER-330.patch Sorry I didn't catch the problem with src/c/tests/zkServer.sh invoking ZooKeeperMain directly. The only issue I had with your proposed patch here is that the original problem I encountered in ZOOKEEPER-326 returns -- if you start a standalone server using QuorumPeerMain, it ignores tickTime settings there again. This contradicts the information here: http://hadoop.apache.org/zookeeper/docs/r3.1.0/zookeeperStarted.html#sc_InstallingSingleMode and it also just would be nice, I think, for the standard zkServer.sh (which uses QuorumPeerMain) and standard config file to work fully for standalone installations out of the box. My revisions in the attached patch allow ZooKeeperMain to take a single config file as an alternate set of arguments, in which case it works like QuorumPeerMain. This seems to resolve both my original issue, support the addition of a command-line tickTime argument as per your patch, and allow the src/c/tests/zkServer.sh script to work as-is. zookeeper standalone server does not startup with just a port and datadir. -- Key: ZOOKEEPER-330 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-330 Project: Zookeeper Issue Type: Bug Affects Versions: 3.1.1, 3.2.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-330.patch, ZOOKEEPER-330.patch ZOOKEEPER-326 made a change to zookeeperservermain.java that broke the starting of zookeeperserver with just the port and datadir. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-319) add locking around auth info in zhandle_t
[ https://issues.apache.org/jira/browse/ZOOKEEPER-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-319: Attachment: ZOOKEEPER-319.patch Good points -- see if this suits. add locking around auth info in zhandle_t - Key: ZOOKEEPER-319 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-319 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-319.patch, ZOOKEEPER-319.patch, ZOOKEEPER-319.patch Looking over the zookeeper.c code it appears to me that the zoo_add_auth() function may be called at any time by the user in their main thread. This function alters the elements of the auth_info structure in the zhandle_t structure. Meanwhile, the IO thread may read those elements at any time in such functions as send_auth_info() and auth_completion_func(). It seems important, then, to add a lock which prevents data being read by the IO thread while only partially changed by the user's thread. The attached patch add such a lock. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-318) remove locking in zk_hashtable.c or add locking in collect_keys()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12677147#action_12677147 ] Chris Darroch commented on ZOOKEEPER-318: - Well, my own take on things like this and ZOOKEEPER-262 is that it's always good to keep things clean and simple -- a good day of programming in my book is one that removes more lines of code than are created and yet keeps the same or better functionality. Aside from any incremental performance gains, I think the big win with both of these patches is that they make the purpose of the code that much more apparent. A significant part of programming, I believe, is psychology. A programmer who comes across a package laced with pthread_mutex_lock() statements immediately makes two pretty reasonable assumptions: the code is used in a multi-threaded context, and it's MT-safe. In this case, both assumptions are incorrect; the code isn't used in an MT context and if it were to be, collect_keys() appears to be lacking the necessary locks and I suspect it would be MT-unsafe. There could always be other subtle MT-related bugs which haven't been shaken out too, should one start using it in MT code. Thus my own feeling is that it's better to simplify and remove these locks for a variety of reasons: it makes the code more self-documenting; easier to read, understand, and revise; and marginally faster. Should the hashtables need to be used in an MT context in the future, the existing code can always be recovered quickly from SVN. If there's an explanatory note in the SVN log that mentions the collect_keys() issue, all the better; then whoever might need to do this work will be prompted to think that aspect through as well. That's just my two cents, of course. :-) remove locking in zk_hashtable.c or add locking in collect_keys() - Key: ZOOKEEPER-318 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-318 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.2.0, 4.0.0 Attachments: ZOOKEEPER-318.patch From a review of zk_hashtable.c it appears to me that all functions which manipulate the hashtables are called from the IO thread, and therefore any need for locking is obviated. If I'm wrong about that, then I think at a minimum collect_keys() should acquire a lock in the same manner as collect_session_watchers(). Both iterate over hashtable contents (in the latter case using copy_table()). However, from what I can see, the only function (besides the init/destroy functions used when creating a zhandle_t) called from the completion thread is deliverWatchers(), which simply iterates over a delivery list created from the hashtables by collectWatchers(). The activateWatcher() function contains comments which describe it being called by the completion thread, but in fact it is called by the IO thread in zookeeper_process(). I believe all calls to collectWatchers(), activateWatcher(), and collect_keys() are made by the IO thread in zookeeper_interest(), zookeeper_process(), check_events(), send_set_watches(), and handle_error(). Note that queue_session_event() is aliased as PROCESS_SESSION_EVENT, but appears only in handle_error() and check_events(). Also note that handle_error() is called only in zookeeper_process() and handle_socket_error_msg(), which is used only by the IO thread, so far as I can see. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (ZOOKEEPER-320) call auth completion in free_completions()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12677143#action_12677143 ] cdarroch edited comment on ZOOKEEPER-320 at 2/26/09 2:45 PM: -- Updated with a NULL initialization as per the comment on [ZOOKEEPER-319#action_12676824]. Out of interest --- what compiler gives these errors? My gcc 4.1.2 with -Wall doesn't report any troubles. was (Author: cdarroch): Updated with a NULL initialization as per the comment on ZOOKEEPER-319#action_12676824 Out of interest --- what compiler gives these errors? My gcc 4.1.2 with -Wall doesn't report any troubles. call auth completion in free_completions() -- Key: ZOOKEEPER-320 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-320 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Assignee: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-320-319.patch, ZOOKEEPER-320-319.patch, ZOOKEEPER-320.patch If a client calls zoo_add_auth() with an invalid scheme (e.g., foo) the ZooKeeper server will mark their session expired and close the connection. However, the C client has returned immediately after queuing the new auth data to be sent with a ZOK return code. If the client then waits for their auth completion function to be called, they can wait forever, as no session event is ever delivered to that completion function. All other completion functions are notified of session events by free_completions(), which is called by cleanup_bufs() in handle_error() in handle_socket_error_msg(). In actual fact, what can happen (about 50% of the time, for me) is that the next call by the IO thread to flush_send_queue() calls send() from within send_buffer(), and receives a SIGPIPE signal during this send() call. Because the ZooKeeper C API is a library, it properly does not catch that signal. If the user's code is not catching that signal either, they experience an abort caused by an untrapped signal. If they are ignoring the signal -- which is common in context I'm working in, the Apache httpd server -- then flush_send_queue()'s error return code is EPIPE, which is logged by handle_socket_error_msg(), and all non-auth completion functions are notified of a session event. However, if the caller is waiting for their auth completion function, they wait forever while the IO thread tries repeatedly to reconnect and is rejected by the server as having an expired session. So, first of all, it would be useful to document in the C API portion of the programmer's guide that trapping or ignoring SIGPIPE is important, as this signal may be generated by the C API. Next, the two attached patches call the auth completion function, if any, in free_completions(), which fixes this problem for me. The second attached patch includes auth lock/unlock function, as per ZOOKEEPER-319. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-326) standalone server ignores tickTime configuration
[ https://issues.apache.org/jira/browse/ZOOKEEPER-326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-326: Attachment: ZOOKEEPER-326.patch standalone server ignores tickTime configuration Key: ZOOKEEPER-326 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-326 Project: Zookeeper Issue Type: Bug Components: server Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-326.patch When using the ZooKeeper server in standalone mode, it ignores the tickTime setting in the configuration file and uses the DEFAULT_TICK_TIME of 3000 coded into ZooKeeperServer.java. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-326) standalone server ignores tickTime configuration
[ https://issues.apache.org/jira/browse/ZOOKEEPER-326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-326: Status: Patch Available (was: Open) standalone server ignores tickTime configuration Key: ZOOKEEPER-326 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-326 Project: Zookeeper Issue Type: Bug Components: server Affects Versions: 3.1.0, 3.0.1, 3.0.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-326.patch When using the ZooKeeper server in standalone mode, it ignores the tickTime setting in the configuration file and uses the DEFAULT_TICK_TIME of 3000 coded into ZooKeeperServer.java. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-311) handle small path lengths in zoo_create()
[
https://issues.apache.org/jira/browse/ZOOKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Darroch updated ZOOKEEPER-311:
Attachment: ZOOKEEPER-311.patch
Changes parameter names as suggested and revises C API documentation.
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.0.0, 3.0.1, 3.1.0
Reporter: Chris Darroch
Priority: Minor
Fix For: 4.0.0
Attachments: ZOOKEEPER-311.patch, ZOOKEEPER-311.patch
The synchronous completion for zoo_create() contains the following code:
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but
again, nothing will happen (even if strlen(res.path) is 0, which is unlikely
since new node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is
its description (the maximum length of real path you would want) in
zookeeper.h, and the example usage in the Programmer's Guide:
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL,
buffer, sizeof(buffer)-1);
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last
byte, leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to
zoo_set(). The path_buffer_len would be treated as the full length of the
buffer (as the code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-311) handle small path lengths in zoo_create()
[
https://issues.apache.org/jira/browse/ZOOKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Darroch updated ZOOKEEPER-311:
Release Note: The zoo_create() function is revised to take path_buffer and
path_buffer_len arguments. The path_buffer will be filled with as much of the
actual path of the new node fits into the buffer, including a null terminator.
Unlike 3.x behaviour, if path_buffer_len (formerly max_realpath_len) is 1, the
buffer will be filled with a single null terminator.
Hadoop Flags: [Incompatible change]
Status: Patch Available (was: Open)
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.1.0, 3.0.1, 3.0.0
Reporter: Chris Darroch
Priority: Minor
Fix For: 4.0.0
Attachments: ZOOKEEPER-311.patch, ZOOKEEPER-311.patch
The synchronous completion for zoo_create() contains the following code:
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but
again, nothing will happen (even if strlen(res.path) is 0, which is unlikely
since new node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is
its description (the maximum length of real path you would want) in
zookeeper.h, and the example usage in the Programmer's Guide:
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL,
buffer, sizeof(buffer)-1);
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last
byte, leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to
zoo_set(). The path_buffer_len would be treated as the full length of the
buffer (as the code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-318) remove locking in zk_hashtable.c or add locking in collect_keys()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-318: Release Note: Marks some hashtable-related functions static and removes them from the public C API. Status: Patch Available (was: Open) remove locking in zk_hashtable.c or add locking in collect_keys() - Key: ZOOKEEPER-318 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-318 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.1.0, 3.0.1, 3.0.0 Reporter: Chris Darroch Fix For: 3.2.0, 4.0.0 Attachments: ZOOKEEPER-318.patch From a review of zk_hashtable.c it appears to me that all functions which manipulate the hashtables are called from the IO thread, and therefore any need for locking is obviated. If I'm wrong about that, then I think at a minimum collect_keys() should acquire a lock in the same manner as collect_session_watchers(). Both iterate over hashtable contents (in the latter case using copy_table()). However, from what I can see, the only function (besides the init/destroy functions used when creating a zhandle_t) called from the completion thread is deliverWatchers(), which simply iterates over a delivery list created from the hashtables by collectWatchers(). The activateWatcher() function contains comments which describe it being called by the completion thread, but in fact it is called by the IO thread in zookeeper_process(). I believe all calls to collectWatchers(), activateWatcher(), and collect_keys() are made by the IO thread in zookeeper_interest(), zookeeper_process(), check_events(), send_set_watches(), and handle_error(). Note that queue_session_event() is aliased as PROCESS_SESSION_EVENT, but appears only in handle_error() and check_events(). Also note that handle_error() is called only in zookeeper_process() and handle_socket_error_msg(), which is used only by the IO thread, so far as I can see. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-318) remove locking in zk_hashtable.c or add locking in collect_keys()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12675468#action_12675468 ] Chris Darroch commented on ZOOKEEPER-318: - Note that this patch does not comprehensively deal with ZOOKEEPER-317; there remain others functions that might be marked static and excluded from the exported symbols besides the ones dealt with in this patch. remove locking in zk_hashtable.c or add locking in collect_keys() - Key: ZOOKEEPER-318 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-318 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.2.0, 4.0.0 Attachments: ZOOKEEPER-318.patch From a review of zk_hashtable.c it appears to me that all functions which manipulate the hashtables are called from the IO thread, and therefore any need for locking is obviated. If I'm wrong about that, then I think at a minimum collect_keys() should acquire a lock in the same manner as collect_session_watchers(). Both iterate over hashtable contents (in the latter case using copy_table()). However, from what I can see, the only function (besides the init/destroy functions used when creating a zhandle_t) called from the completion thread is deliverWatchers(), which simply iterates over a delivery list created from the hashtables by collectWatchers(). The activateWatcher() function contains comments which describe it being called by the completion thread, but in fact it is called by the IO thread in zookeeper_process(). I believe all calls to collectWatchers(), activateWatcher(), and collect_keys() are made by the IO thread in zookeeper_interest(), zookeeper_process(), check_events(), send_set_watches(), and handle_error(). Note that queue_session_event() is aliased as PROCESS_SESSION_EVENT, but appears only in handle_error() and check_events(). Also note that handle_error() is called only in zookeeper_process() and handle_socket_error_msg(), which is used only by the IO thread, so far as I can see. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-319) add locking around auth info in zhandle_t
[ https://issues.apache.org/jira/browse/ZOOKEEPER-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-319: Status: Patch Available (was: Open) add locking around auth info in zhandle_t - Key: ZOOKEEPER-319 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-319 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.1.0, 3.0.1, 3.0.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-319.patch Looking over the zookeeper.c code it appears to me that the zoo_add_auth() function may be called at any time by the user in their main thread. This function alters the elements of the auth_info structure in the zhandle_t structure. Meanwhile, the IO thread may read those elements at any time in such functions as send_auth_info() and auth_completion_func(). It seems important, then, to add a lock which prevents data being read by the IO thread while only partially changed by the user's thread. The attached patch add such a lock. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-320) call auth completion in free_completions()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-320: Status: Patch Available (was: Open) call auth completion in free_completions() -- Key: ZOOKEEPER-320 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-320 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.1.0, 3.0.1, 3.0.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-320-319.patch, ZOOKEEPER-320.patch If a client calls zoo_add_auth() with an invalid scheme (e.g., foo) the ZooKeeper server will mark their session expired and close the connection. However, the C client has returned immediately after queuing the new auth data to be sent with a ZOK return code. If the client then waits for their auth completion function to be called, they can wait forever, as no session event is ever delivered to that completion function. All other completion functions are notified of session events by free_completions(), which is called by cleanup_bufs() in handle_error() in handle_socket_error_msg(). In actual fact, what can happen (about 50% of the time, for me) is that the next call by the IO thread to flush_send_queue() calls send() from within send_buffer(), and receives a SIGPIPE signal during this send() call. Because the ZooKeeper C API is a library, it properly does not catch that signal. If the user's code is not catching that signal either, they experience an abort caused by an untrapped signal. If they are ignoring the signal -- which is common in context I'm working in, the Apache httpd server -- then flush_send_queue()'s error return code is EPIPE, which is logged by handle_socket_error_msg(), and all non-auth completion functions are notified of a session event. However, if the caller is waiting for their auth completion function, they wait forever while the IO thread tries repeatedly to reconnect and is rejected by the server as having an expired session. So, first of all, it would be useful to document in the C API portion of the programmer's guide that trapping or ignoring SIGPIPE is important, as this signal may be generated by the C API. Next, the two attached patches call the auth completion function, if any, in free_completions(), which fixes this problem for me. The second attached patch includes auth lock/unlock function, as per ZOOKEEPER-319. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-318) remove locking in zk_hashtable.c or add locking in collect_keys()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-318: Attachment: ZOOKEEPER-318.patch Renamed patch file with issue key. remove locking in zk_hashtable.c or add locking in collect_keys() - Key: ZOOKEEPER-318 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-318 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.2.0, 4.0.0 Attachments: ZOOKEEPER-318.patch From a review of zk_hashtable.c it appears to me that all functions which manipulate the hashtables are called from the IO thread, and therefore any need for locking is obviated. If I'm wrong about that, then I think at a minimum collect_keys() should acquire a lock in the same manner as collect_session_watchers(). Both iterate over hashtable contents (in the latter case using copy_table()). However, from what I can see, the only function (besides the init/destroy functions used when creating a zhandle_t) called from the completion thread is deliverWatchers(), which simply iterates over a delivery list created from the hashtables by collectWatchers(). The activateWatcher() function contains comments which describe it being called by the completion thread, but in fact it is called by the IO thread in zookeeper_process(). I believe all calls to collectWatchers(), activateWatcher(), and collect_keys() are made by the IO thread in zookeeper_interest(), zookeeper_process(), check_events(), send_set_watches(), and handle_error(). Note that queue_session_event() is aliased as PROCESS_SESSION_EVENT, but appears only in handle_error() and check_events(). Also note that handle_error() is called only in zookeeper_process() and handle_socket_error_msg(), which is used only by the IO thread, so far as I can see. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-318) remove locking in zk_hashtable.c or add locking in collect_keys()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-318: Attachment: (was: zk_hashtable.patch) remove locking in zk_hashtable.c or add locking in collect_keys() - Key: ZOOKEEPER-318 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-318 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.2.0, 4.0.0 Attachments: ZOOKEEPER-318.patch From a review of zk_hashtable.c it appears to me that all functions which manipulate the hashtables are called from the IO thread, and therefore any need for locking is obviated. If I'm wrong about that, then I think at a minimum collect_keys() should acquire a lock in the same manner as collect_session_watchers(). Both iterate over hashtable contents (in the latter case using copy_table()). However, from what I can see, the only function (besides the init/destroy functions used when creating a zhandle_t) called from the completion thread is deliverWatchers(), which simply iterates over a delivery list created from the hashtables by collectWatchers(). The activateWatcher() function contains comments which describe it being called by the completion thread, but in fact it is called by the IO thread in zookeeper_process(). I believe all calls to collectWatchers(), activateWatcher(), and collect_keys() are made by the IO thread in zookeeper_interest(), zookeeper_process(), check_events(), send_set_watches(), and handle_error(). Note that queue_session_event() is aliased as PROCESS_SESSION_EVENT, but appears only in handle_error() and check_events(). Also note that handle_error() is called only in zookeeper_process() and handle_socket_error_msg(), which is used only by the IO thread, so far as I can see. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-309) core dump using zoo_get_acl()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-309: Attachment: ZOOKEEPER-309.patch Renamed patch file with issue key. core dump using zoo_get_acl() -- Key: ZOOKEEPER-309 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-309 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-309.patch The zookeeper_process() function incorrectly calls the c.acl_result member of the completion_list_t structure when handling the completion from a synchronous zoo_get_acl() request. The c.acl_result member is set to SYNCHRONOUS_MARKER, which is a null pointer. The attached patch removes this call. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-319) add locking around auth info in zhandle_t
[ https://issues.apache.org/jira/browse/ZOOKEEPER-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-319: Attachment: ZOOKEEPER-319.patch This version avoids holding the auth lock while calling the user's auth completion function (which may run for a long time; we don't know). add locking around auth info in zhandle_t - Key: ZOOKEEPER-319 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-319 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-319.patch, ZOOKEEPER-319.patch Looking over the zookeeper.c code it appears to me that the zoo_add_auth() function may be called at any time by the user in their main thread. This function alters the elements of the auth_info structure in the zhandle_t structure. Meanwhile, the IO thread may read those elements at any time in such functions as send_auth_info() and auth_completion_func(). It seems important, then, to add a lock which prevents data being read by the IO thread while only partially changed by the user's thread. The attached patch add such a lock. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-319) add locking around auth info in zhandle_t
[ https://issues.apache.org/jira/browse/ZOOKEEPER-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-319: Attachment: (was: ZOOKEEPER-319.patch) add locking around auth info in zhandle_t - Key: ZOOKEEPER-319 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-319 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-319.patch Looking over the zookeeper.c code it appears to me that the zoo_add_auth() function may be called at any time by the user in their main thread. This function alters the elements of the auth_info structure in the zhandle_t structure. Meanwhile, the IO thread may read those elements at any time in such functions as send_auth_info() and auth_completion_func(). It seems important, then, to add a lock which prevents data being read by the IO thread while only partially changed by the user's thread. The attached patch add such a lock. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-320) call auth completion in free_completions()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-320: Attachment: (was: ZOOKEEPER-320-319.patch) call auth completion in free_completions() -- Key: ZOOKEEPER-320 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-320 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-320-319.patch, ZOOKEEPER-320.patch If a client calls zoo_add_auth() with an invalid scheme (e.g., foo) the ZooKeeper server will mark their session expired and close the connection. However, the C client has returned immediately after queuing the new auth data to be sent with a ZOK return code. If the client then waits for their auth completion function to be called, they can wait forever, as no session event is ever delivered to that completion function. All other completion functions are notified of session events by free_completions(), which is called by cleanup_bufs() in handle_error() in handle_socket_error_msg(). In actual fact, what can happen (about 50% of the time, for me) is that the next call by the IO thread to flush_send_queue() calls send() from within send_buffer(), and receives a SIGPIPE signal during this send() call. Because the ZooKeeper C API is a library, it properly does not catch that signal. If the user's code is not catching that signal either, they experience an abort caused by an untrapped signal. If they are ignoring the signal -- which is common in context I'm working in, the Apache httpd server -- then flush_send_queue()'s error return code is EPIPE, which is logged by handle_socket_error_msg(), and all non-auth completion functions are notified of a session event. However, if the caller is waiting for their auth completion function, they wait forever while the IO thread tries repeatedly to reconnect and is rejected by the server as having an expired session. So, first of all, it would be useful to document in the C API portion of the programmer's guide that trapping or ignoring SIGPIPE is important, as this signal may be generated by the C API. Next, the two attached patches call the auth completion function, if any, in free_completions(), which fixes this problem for me. The second attached patch includes auth lock/unlock function, as per ZOOKEEPER-319. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-320) call auth completion in free_completions()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-320: Attachment: ZOOKEEPER-320-319.patch This version avoids holding the auth lock while calling the user's auth completion function (which may run for a long time; we don't know). call auth completion in free_completions() -- Key: ZOOKEEPER-320 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-320 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-320-319.patch, ZOOKEEPER-320.patch If a client calls zoo_add_auth() with an invalid scheme (e.g., foo) the ZooKeeper server will mark their session expired and close the connection. However, the C client has returned immediately after queuing the new auth data to be sent with a ZOK return code. If the client then waits for their auth completion function to be called, they can wait forever, as no session event is ever delivered to that completion function. All other completion functions are notified of session events by free_completions(), which is called by cleanup_bufs() in handle_error() in handle_socket_error_msg(). In actual fact, what can happen (about 50% of the time, for me) is that the next call by the IO thread to flush_send_queue() calls send() from within send_buffer(), and receives a SIGPIPE signal during this send() call. Because the ZooKeeper C API is a library, it properly does not catch that signal. If the user's code is not catching that signal either, they experience an abort caused by an untrapped signal. If they are ignoring the signal -- which is common in context I'm working in, the Apache httpd server -- then flush_send_queue()'s error return code is EPIPE, which is logged by handle_socket_error_msg(), and all non-auth completion functions are notified of a session event. However, if the caller is waiting for their auth completion function, they wait forever while the IO thread tries repeatedly to reconnect and is rejected by the server as having an expired session. So, first of all, it would be useful to document in the C API portion of the programmer's guide that trapping or ignoring SIGPIPE is important, as this signal may be generated by the C API. Next, the two attached patches call the auth completion function, if any, in free_completions(), which fixes this problem for me. The second attached patch includes auth lock/unlock function, as per ZOOKEEPER-319. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-319) add locking around auth info in zhandle_t
add locking around auth info in zhandle_t - Key: ZOOKEEPER-319 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-319 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.1.0, 3.0.1, 3.0.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Looking over the zookeeper.c code it appears to me that the zoo_add_auth() function may be called at any time by the user in their main thread. This function alters the elements of the auth_info structure in the zhandle_t structure. Meanwhile, the IO thread may read those elements at any time in such functions as send_auth_info() and auth_completion_func(). It seems important, then, to add a lock which prevents data being read by the IO thread while only partially changed by the user's thread. The attached patch add such a lock. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-320) call auth completion in free_completions()
call auth completion in free_completions() -- Key: ZOOKEEPER-320 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-320 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.1.0, 3.0.1, 3.0.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 If a client calls zoo_add_auth() with an invalid scheme (e.g., foo) the ZooKeeper server will mark their session expired and close the connection. However, the C client has returned immediately after queuing the new auth data to be sent with a ZOK return code. If the client then waits for their auth completion function to be called, they can wait forever, as no session event is ever delivered to that completion function. All other completion functions are notified of session events by free_completions(), which is called by cleanup_bufs() in handle_error() in handle_socket_error_msg(). In actual fact, what can happen (about 50% of the time, for me) is that the next call by the IO thread to flush_send_queue() calls send() from within send_buffer(), and receives a SIGPIPE signal during this send() call. Because the ZooKeeper C API is a library, it properly does not catch that signal. If the user's code is not catching that signal either, they experience an abort caused by an untrapped signal. If they are ignoring the signal -- which is common in context I'm working in, the Apache httpd server -- then flush_send_queue()'s error return code is EPIPE, which is logged by handle_socket_error_msg(), and all non-auth completion functions are notified of a session event. However, if the caller is waiting for their auth completion function, they wait forever while the IO thread tries repeatedly to reconnect and is rejected by the server as having an expired session. So, first of all, it would be useful to document in the C API portion of the programmer's guide that trapping or ignoring SIGPIPE is important, as this signal may be generated by the C API. Next, the two attached patches call the auth completion function, if any, in free_completions(), which fixes this problem for me. The second attached patch includes auth lock/unlock function, as per ZOOKEEPER-319. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-320) call auth completion in free_completions()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-320: Attachment: ZOOKEEPER-320.patch This patch does not include locking for the auth data, as per ZOOKEEPER-319. call auth completion in free_completions() -- Key: ZOOKEEPER-320 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-320 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-320.patch If a client calls zoo_add_auth() with an invalid scheme (e.g., foo) the ZooKeeper server will mark their session expired and close the connection. However, the C client has returned immediately after queuing the new auth data to be sent with a ZOK return code. If the client then waits for their auth completion function to be called, they can wait forever, as no session event is ever delivered to that completion function. All other completion functions are notified of session events by free_completions(), which is called by cleanup_bufs() in handle_error() in handle_socket_error_msg(). In actual fact, what can happen (about 50% of the time, for me) is that the next call by the IO thread to flush_send_queue() calls send() from within send_buffer(), and receives a SIGPIPE signal during this send() call. Because the ZooKeeper C API is a library, it properly does not catch that signal. If the user's code is not catching that signal either, they experience an abort caused by an untrapped signal. If they are ignoring the signal -- which is common in context I'm working in, the Apache httpd server -- then flush_send_queue()'s error return code is EPIPE, which is logged by handle_socket_error_msg(), and all non-auth completion functions are notified of a session event. However, if the caller is waiting for their auth completion function, they wait forever while the IO thread tries repeatedly to reconnect and is rejected by the server as having an expired session. So, first of all, it would be useful to document in the C API portion of the programmer's guide that trapping or ignoring SIGPIPE is important, as this signal may be generated by the C API. Next, the two attached patches call the auth completion function, if any, in free_completions(), which fixes this problem for me. The second attached patch includes auth lock/unlock function, as per ZOOKEEPER-319. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-320) call auth completion in free_completions()
[ https://issues.apache.org/jira/browse/ZOOKEEPER-320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-320: Attachment: ZOOKEEPER-320-319.patch This patch includes auth data locking as per ZOOKEEPER-319. call auth completion in free_completions() -- Key: ZOOKEEPER-320 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-320 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0, 3.0.1, 3.1.0 Reporter: Chris Darroch Fix For: 3.1.1, 3.2.0 Attachments: ZOOKEEPER-320-319.patch, ZOOKEEPER-320.patch If a client calls zoo_add_auth() with an invalid scheme (e.g., foo) the ZooKeeper server will mark their session expired and close the connection. However, the C client has returned immediately after queuing the new auth data to be sent with a ZOK return code. If the client then waits for their auth completion function to be called, they can wait forever, as no session event is ever delivered to that completion function. All other completion functions are notified of session events by free_completions(), which is called by cleanup_bufs() in handle_error() in handle_socket_error_msg(). In actual fact, what can happen (about 50% of the time, for me) is that the next call by the IO thread to flush_send_queue() calls send() from within send_buffer(), and receives a SIGPIPE signal during this send() call. Because the ZooKeeper C API is a library, it properly does not catch that signal. If the user's code is not catching that signal either, they experience an abort caused by an untrapped signal. If they are ignoring the signal -- which is common in context I'm working in, the Apache httpd server -- then flush_send_queue()'s error return code is EPIPE, which is logged by handle_socket_error_msg(), and all non-auth completion functions are notified of a session event. However, if the caller is waiting for their auth completion function, they wait forever while the IO thread tries repeatedly to reconnect and is rejected by the server as having an expired session. So, first of all, it would be useful to document in the C API portion of the programmer's guide that trapping or ignoring SIGPIPE is important, as this signal may be generated by the C API. Next, the two attached patches call the auth completion function, if any, in free_completions(), which fixes this problem for me. The second attached patch includes auth lock/unlock function, as per ZOOKEEPER-319. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-273) Zookeeper c client build should not depend on CPPUNIT
[ https://issues.apache.org/jira/browse/ZOOKEEPER-273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-273: Description: One should be able to build Zookeeper C client libs on a machine without CPPUNIT installation. A simple fix is to remove from configure.ac the following line: M_PATH_CPPUNIT(1.10.2) was: One should be able to build Zookeeper C client libs on a machine without CPPUNIT installation. A simple fix is to remove from configure.ac the following line: M_PATH_CPPUNIT(1.10.2) Affects Version/s: 3.1.0 Fix Version/s: (was: 3.1.0) 3.2.0 As of 3.1.0 release, this doesn't seem to work as advertised: ./configure --without-cppunit make ... aclocal:configure.ac:36: warning: macro `AM_PATH_CPPUNIT' not found in library ... Makefile.am:68: wildcard tests/*.cc: non-POSIX variable name Makefile.am:68: (probably a GNU make extension) Makefile.am:68: wildcard tests/*.h: non-POSIX variable name Makefile.am:68: (probably a GNU make extension) Makefile.am:77: shell cat tests/wrappers.opt: non-POSIX variable name Makefile.am:77: (probably a GNU make extension) Makefile.am:90: shell cat tests/wrappers-mt.opt: non-POSIX variable name Makefile.am:90: (probably a GNU make extension) ... configure.ac:36: error: possibly undefined macro: AM_PATH_CPPUNIT If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. make: *** [configure] Error 1 Zookeeper c client build should not depend on CPPUNIT - Key: ZOOKEEPER-273 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-273 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.1.0 Reporter: Runping Qi Assignee: Patrick Hunt Fix For: 3.2.0 Attachments: patch_zookeeper_273.txt, ZOOKEEPER-273.patch, ZOOKEEPER-273.patch One should be able to build Zookeeper C client libs on a machine without CPPUNIT installation. A simple fix is to remove from configure.ac the following line: M_PATH_CPPUNIT(1.10.2) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-311) handle small path lengths in zoo_create()
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.0.1, 3.0.0, 3.1.0
Reporter: Chris Darroch
Priority: Minor
Fix For: 4.0.0
Attachments: zookeeper.c.patch
The synchronous completion for zoo_create() contains the following code:
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but again,
nothing will happen (even if strlen(res.path) is 0, which is unlikely since new
node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is its
description (the maximum length of real path you would want) in zookeeper.h,
and the example usage in the Programmer's Guide:
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL, buffer,
sizeof(buffer)-1);
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last byte,
leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to zoo_set().
The path_buffer_len would be treated as the full length of the buffer (as the
code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-311) handle small path lengths in zoo_create()
[
https://issues.apache.org/jira/browse/ZOOKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Darroch updated ZOOKEEPER-311:
Attachment: zookeeper.c.patch
handle small path lengths in zoo_create()
-
Key: ZOOKEEPER-311
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-311
Project: Zookeeper
Issue Type: Improvement
Components: c client
Affects Versions: 3.0.0, 3.0.1, 3.1.0
Reporter: Chris Darroch
Priority: Minor
Fix For: 4.0.0
Attachments: zookeeper.c.patch
The synchronous completion for zoo_create() contains the following code:
if (sc-u.str.str_len strlen(res.path)) {
len = strlen(res.path);
} else {
len = sc-u.str.str_len-1;
}
if (len 0) {
memcpy(sc-u.str.str, res.path, len);
sc-u.str.str[len] = '\0';
}
In the case where the max_realpath_len argument to zoo_create() is 0, none of
this code executes, which is OK. In the case where max_realpath_len is 1, a
user might expect their buffer to be filled with a null terminator, but
again, nothing will happen (even if strlen(res.path) is 0, which is unlikely
since new node's will have paths longer than /).
The name of the argument to zoo_create() is also a little misleading, as is
its description (the maximum length of real path you would want) in
zookeeper.h, and the example usage in the Programmer's Guide:
int rc = zoo_create(zh,/xyz,value, 5, CREATE_ONLY, ZOO_EPHEMERAL,
buffer, sizeof(buffer)-1);
In fact this value should be the actual length of the buffer, including space
for the null terminator. If the user supplies a max_realpath_len of 10 and a
buffer of 11 bytes, and strlen(res.path) is 10, the code will truncate the
returned value to 9 bytes and put the null terminator in the second-last
byte, leaving the final byte of the buffer unused.
It would be better, I think, to rename the realpath and max_realpath_len
arguments to something like path_buffer and path_buffer_len, akin to
zoo_set(). The path_buffer_len would be treated as the full length of the
buffer (as the code does now, in fact, but the docs suggest otherwise).
The code in the synchronous completion could then be changed as per the
attached patch.
Since this would change, slightly, the behaviour or contract of the API, I
would be inclined to suggest waiting until 4.0.0 to implement this change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-216) Improve logging in C client
[ https://issues.apache.org/jira/browse/ZOOKEEPER-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12670859#action_12670859 ] Chris Darroch commented on ZOOKEEPER-216: - A few specific ideas and areas of concern. The threads spawned by a single connection handle may write to the log file at the same time. Personally, this doesn't concern me overmuch; I'd prefer not to have the logging subsystem doing a lot of mutex locking just to ensure the occasional log entries don't collide. More seriously, the logLevel and logStream variables in zk_log.c are global resources. That means if a program opens several ZooKeeper connections, they all share the file descriptor, and so all connection handles are affected if any part of the program changes the log file descriptor. There may also be a small possibility of collision that if one thread changes the log file descriptor while another is reading it (although it's just a pointer, so it depends on whether the pointer is updated atomically). At any rate, it would be excellent to have per-connection-handle logging attributes (level and stream). Improve logging in C client --- Key: ZOOKEEPER-216 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-216 Project: Zookeeper Issue Type: Improvement Components: c client Affects Versions: 3.0.0 Reporter: Patrick Hunt Fix For: 3.2.0 While debugging a client issue we saw multiple areas for improvement in C logging. It might be a good idea to also move to log4c. Anyone have experience with this? multithread an issue? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-295) prevent multiple namespace pollution by C API
prevent multiple namespace pollution by C API - Key: ZOOKEEPER-295 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-295 Project: Zookeeper Issue Type: Improvement Components: c client Affects Versions: 3.0.1, 3.0.0, 3.1.0 Reporter: Chris Darroch The ZOOKEEPER-6 issue touched on the problem of namespace pollution by the ZooKeeper C API; this report was closed but I don't think the problem has actually been substantially resolved. There are multiple namespaces to consider. First, the names of the C include files should ideally have a common prefix, e.g., zoo_recordio.h, or else be concatenated into a single zookeeper.h file. The zookeeper.jute.h include file has a reasonably good name in this regard. Second, all macros should ideally have a common prefix, e.g., ZOO_ or ZK_ or ZOOKEEPER_. Currently many exported constants (not macros) have the ZOO_ prefix, such as ZOO_PERM_READ, but error codes have a Z prefix, e.g., ZOK, ZNOTEMPTY. Third, all functions should ideally have a common prefix, e.g., zoo_ or zk_ or zookeeper_. Many do already, but there is some variation, such as zookeeper_init(), zookeeper_process(), and there are also a large number of functions which have no prefix. These include many of the functions defined in recordio.h and zookeeper.jute.h, such as get_buffer() and serialize_Id(). Many others are simply used internally within the ZooKeeper C library and not declared in an external include file, but still pollute the caller's namespace, e.g., get_xid(), process_completions(), adaptor_init(), etc. All external symbols in the libraries should have a common prefix. Fourth, all structure and type definitions should also have a common prefix, again, zoo_ or zk_ or zookeeper_. This is especially true of structures which currently have very generic names such as Id, Stat, and ACL from zookeeper.jute.h; buffer, iarchive, and oarchive from recordio.h; and clientid_t and watcher_fn from zookeeper.h. The zhandle_t structure should also be renamed to have the same prefix, e.g., zoo_handle_t. The ZOOKEEPER-6 report includes the comment that the names in zookeeper.jute.h will be difficult to change because they affect the Java code and that there should be limited exposure since jute naming starts with caps. It would be nice to think so, but I fear that a structure named Id or Stat is going to be pretty darn commonplace in other people's code. I would strongly recommend revising the entire set file, function, macro, type and structure names for 4.0.0. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-227) gcc warning from recordio.h
[ https://issues.apache.org/jira/browse/ZOOKEEPER-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-227: Status: Open (was: Patch Available) gcc warning from recordio.h --- Key: ZOOKEEPER-227 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-227 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.0.0 Recent gcc compilers issue warnings when function declarations for functions with no arguments don't specific void. The attached patch fixes one such warning for create_buffer_oarchive() in recordio.h. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (ZOOKEEPER-227) gcc warning from recordio.h
gcc warning from recordio.h --- Key: ZOOKEEPER-227 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-227 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.0.0 Recent gcc compilers issue warnings when function declarations for functions with no arguments don't specific void. The attached patch fixes one such warning for create_buffer_oarchive() in recordio.h. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-227) gcc warning from recordio.h
[ https://issues.apache.org/jira/browse/ZOOKEEPER-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-227: Status: Patch Available (was: Open) gcc warning from recordio.h --- Key: ZOOKEEPER-227 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-227 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.0.0 Recent gcc compilers issue warnings when function declarations for functions with no arguments don't specific void. The attached patch fixes one such warning for create_buffer_oarchive() in recordio.h. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-227) gcc warning from recordio.h
[ https://issues.apache.org/jira/browse/ZOOKEEPER-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-227: Attachment: zookeeper-recordio-trunk.patch FWIW, I'm an Apache committer with a contributor agreement on file. gcc warning from recordio.h --- Key: ZOOKEEPER-227 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-227 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.0.0 Attachments: zookeeper-recordio-trunk.patch Recent gcc compilers issue warnings when function declarations for functions with no arguments don't specific void. The attached patch fixes one such warning for create_buffer_oarchive() in recordio.h. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-227) gcc warning from recordio.h
[ https://issues.apache.org/jira/browse/ZOOKEEPER-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Darroch updated ZOOKEEPER-227: Status: Patch Available (was: Open) gcc warning from recordio.h --- Key: ZOOKEEPER-227 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-227 Project: Zookeeper Issue Type: Bug Components: c client Affects Versions: 3.0.0 Reporter: Chris Darroch Priority: Minor Fix For: 3.0.0 Attachments: zookeeper-recordio-trunk.patch Recent gcc compilers issue warnings when function declarations for functions with no arguments don't specific void. The attached patch fixes one such warning for create_buffer_oarchive() in recordio.h. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-208) Zookeeper C client uses API that are not thread safe, causing crashes when multiple instances are active
[
https://issues.apache.org/jira/browse/ZOOKEEPER-208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Darroch updated ZOOKEEPER-208:
Attachment: zookeeper-strtok_getaddrinfo-trunk.patch
I've reformatted the original submitter's normal patch as a unified diff file
to use against 3.0.0 and SVN trunk (at the moment).
I won't grant a license for it, though, since I didn't do the original work.
Hopefully this saves the submitter a little work. If they can just sign off on
this file, they shouldn't need to deal with any other details.
Zookeeper C client uses API that are not thread safe, causing crashes when
multiple instances are active
Key: ZOOKEEPER-208
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-208
Project: Zookeeper
Issue Type: Bug
Components: c client
Affects Versions: 3.0.0
Environment: Linux
Reporter: Austin Shoemaker
Assignee: Austin Shoemaker
Priority: Critical
Fix For: 3.1.0
Attachments: zookeeper-strtok_getaddrinfo-trunk.patch
The Zookeeper C client library uses gethostbyname and strtok, both of which
are not safe to use from multiple threads. Below is the original patch we
made which fixes the problem.
The problem is resolved by using getaddrinfo and strtok_r in place of the
older API.
Patch for zookeeper-c-client-2.2.1/src/zookeeper.c (2008-06-09 on SF.net)
241c241
struct hostent *he;
---
struct addrinfo hints, *res, *res0;
243,245d242
struct sockaddr_in *addr4;
struct sockaddr_in6 *addr6;
char **ptr;
247a245
char *strtok_last;
263c261
host=strtok(hosts, ,);
---
host=strtok_r(hosts, ,, strtok_last);
283,294c281,297
he = gethostbyname(host);
if (!he) {
LOG_ERROR((could not resolve %s, host));
errno=EINVAL;
rc=ZBADARGUMENTS;
goto fail;
}
/* Setup the address array */
for(ptr = he-h_addr_list;*ptr != 0; ptr++) {
if (zh-addrs_count == alen) {
void *tmpaddr;
---
memset(hints, 0, sizeof(hints));
hints.ai_flags = AI_ADDRCONFIG;
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
hints.ai_protocol = IPPROTO_TCP;
if (getaddrinfo(host, port_spec, hints, res0) != 0) {
LOG_ERROR((getaddrinfo: %s\n, strerror(errno)));
rc=ZSYSTEMERROR;
goto fail;
}
for (res = res0; res; res = res-ai_next) {
// Expand address list if needed
if (zh-addrs_count == alen) {
void *tmpaddr;
304,313c307,312
}
addr = zh-addrs[zh-addrs_count];
addr4 = (struct sockaddr_in*)addr;
addr6 = (struct sockaddr_in6*)addr;
addr-sa_family = he-h_addrtype;
if (addr-sa_family == AF_INET) {
addr4-sin_port = htons(port);
memset(addr4-sin_zero, 0, sizeof(addr4-sin_zero));
memcpy(addr4-sin_addr, *ptr, he-h_length);
zh-addrs_count++;
---
}
// Copy addrinfo into address list
addr = zh-addrs[zh-addrs_count];
switch (res-ai_family) {
case AF_INET:
315,320c314
} else if (addr-sa_family == AF_INET6) {
addr6-sin6_port = htons(port);
addr6-sin6_scope_id = 0;
addr6-sin6_flowinfo = 0;
memcpy(addr6-sin6_addr, *ptr, he-h_length);
zh-addrs_count++;
---
case AF_INET6:
322,327c316,328
} else {
LOG_WARN((skipping unknown address family %x for %s,
addr-sa_family, zh-hostname));
}
}
host = strtok(0, ,);
---
memcpy(addr, res-ai_addr, res-ai_addrlen);
++zh-addrs_count;
break;
default:
LOG_WARN((skipping unknown address family %x
for %s,
res-ai_family, zh-hostname));
break;
}
}
freeaddrinfo(res0);
host = strtok_r(0, ,, strtok_last);
329a331
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue
