Re: Retrying sequential znode creation
On Wed, Oct 20, 2010 at 3:27 PM, Ted Dunning ted.dunn...@gmail.com wrote: These corner cases are relatively rare, I would think (I personally keep logs around for days or longer). A concern I would have is that it does add complexity, would be hard to debug... Would it be possible to get a partial solution in place that invokes the current behavior if logs aren't available? Seems like it's possible. The issue of finding a viable solution (one where, for example, the memory overhead is limited) is still an issue though. In the end it wouldn't really help the end user, given they would still have to code for this corner case. Patrick On Wed, Oct 20, 2010 at 10:42 AM, Patrick Hunt phu...@gmail.com wrote: Hi Ted, Mahadev is in the best position to comment (he looked at it last) but iirc when we started looking into implementing this we immediately ran into so big questions. One was what to do if the logs had been cleaned up and the individual transactions no longer available. This could be overcome by changes wrt cleanup, log rotation, etc... There was another more bulletproof option, essentially to keep all the changes in memory that might be necessary to implement 22, however this might mean a significant increase in mem requirements and general bookkeeping. It turned out (again correct me if I'm wrong) that more thought was going to be necessary, esp around ensuring correct operation in any/all special cases. Patrick On Wed, Oct 13, 2010 at 12:49 PM, Ted Dunning ted.dunn...@gmail.com wrote: Patrick, What are these hurdles? The last comment on ZK-22 was last winter. Back then, it didn't sound like it was going to be that hard. On Wed, Oct 13, 2010 at 12:08 PM, Patrick Hunt ph...@apache.org wrote: 22 would help with this issue https://issues.apache.org/jira/browse/ZOOKEEPER-22 however there are some real hurdles to implementing 22 successfully.
Re: Reading znodes directly from snapshot and log files
Sounds like a useful utility, the closest that I know of is this: http://hadoop.apache.org/zookeeper/docs/current/api/org/apache/zookeeper/server/LogFormatter.html but it just dumps the txn log. Seems like it would be cool to be able to open a shell on the datadir and query it (separate from running a server). Another option is to just copy the datadir and start a standalone zk instance on it. You can then use the std zk shell to query it. Patrick ps. I had worked on something similar in python a while back: http://github.com/phunt/zk-txnlog-tools/blob/master/parse_txnlog.py On Thu, Oct 21, 2010 at 2:31 PM, Vishal K vishalm...@gmail.com wrote: Hi, Is it possible to read znodes directly from snapshot and log files instead of usign ZooKeeper API. In case a ZK ensemble is not available, can I login to all available nodes and run a utility that will dump all znodes? Thanks. -Vishal
Re: Stale value for read request
On Sat, Oct 23, 2010 at 9:03 PM, jingguo yao yaojing...@gmail.com wrote: Read requests are handled locally at each Zookeeper server. So it is possible for a read request to return a stale value even though a more recent update to the same znode has been committed. Does this statement still hold if the Zookeeper follower serving the read request is the one which has just served the recent update request? It's probably good to start with the explicit guarantees: http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkGuarantees Yes (it could still get stale data from quorum perspective). The leader may have committed a new change that has not yet been seen by the follower (ie two changes in quick succession) For example, client A connects to follower X. And client A issues a request to update znode /a from 0 to 1. After receiving this request, follower X forwards this request to the leader. Then the leader broadcasts this update proposal to all the Zookeeper servers. After a quorum of the followers commit the update request, the update succeeds. Then client A issues a read request to get the value of znode /a. And follower X receives this read request. So if follower X is not among the quorum and follower X has not committed the update to catch up with the leader, it is still possible for client A to get a stale value of znode /a. In this case, the return value is 0. Is my understanding correct? That's correct. See the the NOTE in the section (link) I provided above. Patrick
