[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13821286#comment-13821286 ] Flavio Junqueira commented on ZOOKEEPER-1519: - What should we do with this issue? It sounds like we haven't converged. Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3, 3.3.6 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius Assignee: Daniel Lescohier Fix For: 3.4.6, 3.5.0 Attachments: zookeeper-1519.patch zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13822117#comment-13822117 ] Jacky007 commented on ZOOKEEPER-1519: - I don't think this is a problem. The caller should hold the json.dumps(value), and free it in callback if needed. Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3, 3.3.6 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius Assignee: Daniel Lescohier Fix For: 3.4.6, 3.5.0 Attachments: zookeeper-1519.patch zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13785884#comment-13785884 ] Marshall McMullen commented on ZOOKEEPER-1519: -- I agree with Flavio... All the work I did in the C client made the same contract that the caller owns the memory not the C client. This is typical of all async interfaces I've used (e.g. asio, etc). Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3, 3.3.6 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius Assignee: Daniel Lescohier Fix For: 3.4.6, 3.5.0 Attachments: zookeeper-1519.patch zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13783752#comment-13783752 ] Flavio Junqueira commented on ZOOKEEPER-1519: - Hi there, I believe the contract with the C client is that the application manages the memory allocation of the data it passes to async calls, so I'm not really happy about copying the data. I'm fine with improving documentation, though. A different issue seems to be with the python client. I don't know enough about it to determine if this could be fixed on its side instead of applying the fix proposed here. Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3, 3.3.6 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius Assignee: Daniel Lescohier Fix For: 3.4.6, 3.5.0 Attachments: zookeeper-1519.patch zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13784159#comment-13784159 ] Patrick Hunt commented on ZOOKEEPER-1519: - [~henryr] any insight here? Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3, 3.3.6 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius Assignee: Daniel Lescohier Fix For: 3.4.6, 3.5.0 Attachments: zookeeper-1519.patch zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587340#comment-13587340 ] Raul Gutierrez Segales commented on ZOOKEEPER-1519: --- Does sizeof *(void *) work? Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3, 3.3.6 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius Attachments: zookeeper-1519.patch zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587351#comment-13587351 ] Raul Gutierrez Segales commented on ZOOKEEPER-1519: --- Don't think so: http://fpaste.org/iwjf/ Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3, 3.3.6 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius Attachments: zookeeper-1519.patch zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587520#comment-13587520 ] Daniel Lescohier commented on ZOOKEEPER-1519: - You're correct. I didn't have time to test the patch, but I wanted to get it out there for discussion. I looked further, and the callers get void* data from the public API with no length parameter. So, the public API does not allow us to copy the data. In order to fix it, it looks like a public API change is required. Either: 1. Document in the API that the caller cannot free that memory until the zookeeper library is done with it (which also means it can't be a pointer to memory on the stack). I assume that the library is done with it once it calls the completion callback? So the program can free it once it gets the same pointer back in a callback (or when the zookeeper connection is closed). I think this would make it hard to integrate with scripting languages like Python, because the scripting language C interface would have to copy the memory, account for it in some global structure, and free it once it sees that pointer again in a callback or when the zookeeper connection is closed. 2. Document in the API that the void * must be malloc'ed memory, and the ownership is passed to the library (which means the caller copies it, and the library frees it). That's also a difficult API. 3. Add a data length parameter to the API, so the library can copy it. 4. Don't use a void * for the 'data' parameter, use something else. Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3, 3.3.6 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius Attachments: zookeeper-1519.patch zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587562#comment-13587562 ] Daniel Lescohier commented on ZOOKEEPER-1519: - Never mind my comments about 'data'. I was confusing 'value' with 'data.' 'value' is copied when it is serialized. So, I don't see where the submitter's original problem can occur. 'data' is all right, because it should be passed by reference. In the zkpython bindings, it's used to hold a reference to the Python callable object that should be called on the callback. That's a python object, so it's refcounted. 'completion' is the zkpython C function that is called, and calls the python callable object that is cast to a Python object and called with the Python API. Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3, 3.3.6 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius Attachments: zookeeper-1519.patch zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422447#comment-13422447
]
Stephen Tyree commented on ZOOKEEPER-1519:
--
Duplicating the data wouldn't help in circumstances where the data is a
structure contain malloc'd memory, i.e.:
struct Foo {
const char* bar;
};
Zookeeper Async calls can reference free()'d memory
---
Key: ZOOKEEPER-1519
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519
Project: ZooKeeper
Issue Type: Bug
Components: c client
Affects Versions: 3.3.3
Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some
backported fixes.
Reporter: Mark Gius
zoo_acreate() and zoo_aset() take a char * argument for data and prepare a
call to zookeeper. This char * doesn't seem to be duplicated at any point,
making it possible that the caller of the asynchronous function might
potentially free() the char * argument before the zookeeper library completes
its request. This is unlikely to present a real problem unless the freed
memory is re-used before zookeeper consumes it. I've been unable to
reproduce this issue using pure C as a result.
However, ZKPython is a whole different story. Consider this snippet:
ok = zookeeper.acreate(handle, path, json.dumps(value),
acl, flags, callback)
assert ok == zookeeper.OK
In this snippet, json.dumps() allocates a string which is passed into the
acreate(). When acreate() returns, the zookeeper request has been
constructed with a pointer to the string allocated by json.dumps(). Also
when acreate() returns, that string is now referenced by 0 things (ZKPython
doesn't bump the refcount) and the string is eligible for garbage collection
and re-use. The Zookeeper request now has a pointer to dangerous freed
memory.
I've been seeing odd behavior in our development environments for some time
now, where it appeared as though two separate JSON payloads had been joined
together. Python has been allocating a new JSON string in the middle of the
old string that an incomplete zookeeper async call had not yet processed.
I am not sure if this is a behavior that should be documented, or if the C
binding implementation needs to be updated to create copies of the data
payload provided for aset and acreate.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422461#comment-13422461 ] Mark Gius commented on ZOOKEEPER-1519: -- Then perhaps the solution is to document a contract between aysnc calls and library consumers that the data pointer needs to be alive until after the async call has complete (or been submitted, or whatever). Or maybe async calls need another argument for a cleanup function that frees the data pointer. Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1519) Zookeeper Async calls can reference free()'d memory
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13422538#comment-13422538 ] Mark Gius commented on ZOOKEEPER-1519: -- well, I don't think zookeeper would handle such a struct correctly anyway, because zookeeper treats whatever you throw into that argument like a byte array. So zookeeper will persist the memory location of the inner-malloc()'d memory, but not the memory itself. Zookeeper Async calls can reference free()'d memory --- Key: ZOOKEEPER-1519 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1519 Project: ZooKeeper Issue Type: Bug Components: c client Affects Versions: 3.3.3 Environment: Ubuntu 11.10, Ubuntu packaged Zookeeper 3.3.3 with some backported fixes. Reporter: Mark Gius zoo_acreate() and zoo_aset() take a char * argument for data and prepare a call to zookeeper. This char * doesn't seem to be duplicated at any point, making it possible that the caller of the asynchronous function might potentially free() the char * argument before the zookeeper library completes its request. This is unlikely to present a real problem unless the freed memory is re-used before zookeeper consumes it. I've been unable to reproduce this issue using pure C as a result. However, ZKPython is a whole different story. Consider this snippet: ok = zookeeper.acreate(handle, path, json.dumps(value), acl, flags, callback) assert ok == zookeeper.OK In this snippet, json.dumps() allocates a string which is passed into the acreate(). When acreate() returns, the zookeeper request has been constructed with a pointer to the string allocated by json.dumps(). Also when acreate() returns, that string is now referenced by 0 things (ZKPython doesn't bump the refcount) and the string is eligible for garbage collection and re-use. The Zookeeper request now has a pointer to dangerous freed memory. I've been seeing odd behavior in our development environments for some time now, where it appeared as though two separate JSON payloads had been joined together. Python has been allocating a new JSON string in the middle of the old string that an incomplete zookeeper async call had not yet processed. I am not sure if this is a behavior that should be documented, or if the C binding implementation needs to be updated to create copies of the data payload provided for aset and acreate. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
