zookeeper utils

2010-03-02 Thread David Rosenstrauch
Was reading through the zookeeper docs on the web - specifically the recipes and solutions page (as well as comments elsewhere inviting additional such contributions from the community) and was wondering: Is there a library of higher-level zookeeper utilities that people have contributed,

Re: zookeeper utils

2010-03-02 Thread David Rosenstrauch
Thanks, I'll take a look. DR On 03/02/2010 05:37 PM, Mahadev Konar wrote: Hi David, There is an implementation for locks and queues in src/recipes. The documentation residres in src/recipes/{lock/queue}/README.txt. Thanks mahadev On 3/2/10 1:04 PM, David Rosenstrauchdar...@darose.net

Re: zookeeper utils

2010-03-03 Thread David Rosenstrauch
On 03/03/2010 01:43 AM, Henry Robinson wrote: Just to illustrate one of the primitives you're looking for: an AtomicInteger equivalent would be fairly easy to construct, with nearly identical semantics to the Java version. Let's say a given znode has four bytes of data that represent an integer

Re: zookeeper utils

2010-03-03 Thread David Rosenstrauch
On 03/03/2010 12:05 PM, Patrick Hunt wrote: This is def. something we should add to the recipes (docs code lib), Henry/David can you create a jira for this? Patrick Done. https://issues.apache.org/jira/browse/ZOOKEEPER-686 Thanks, DR

Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread David Rosenstrauch
I'm contemplating an upcoming zookeeper rollout and was wondering what the zookeeper brain trust here thought about a network deployment question: Is it generally considered bad practice to just deploy zookeeper on our existing hdfs/MR nodes? Or is it better to run zookeeper instances on

Re: Ok to share ZK nodes with Hadoop nodes?

2010-03-08 Thread David Rosenstrauch
On 03/08/2010 02:21 PM, Patrick Hunt wrote: See the troubleshooting page, some apropos detail there (esp relative to virtual env). http://wiki.apache.org/hadoop/ZooKeeper/Troubleshooting ZK servers are sensitive to IO (disk/network) latency. As long as you aren't very sensitive latency

Zookeeper unit tester?

2010-03-09 Thread David Rosenstrauch
Just wondering if there was a mock/fake version of org.apache.zookeeper.Zookeeper that could be used for unit testing? What I'm envisioning would be a single instance Zookeeper that operates completely in memory, with no network or disk I/O. This would make it possible to pass one of the

zkCli.sh missing from zookeeper package in Cloudera CDH contrib repo?

2010-04-26 Thread David Rosenstrauch
I installed the zookeeper package (hbase-0.20-zookeeper) from the Cloudera CDH contrib repo. But neither it nor the hbase-0.20 package that it's dependent on seem to supply the zkCli command line utility. Anyone know if this is intentional and/or how to fix? Thanks, DR

Re: zkCli.sh missing from zookeeper package in Cloudera CDH contrib repo?

2010-04-27 Thread David Rosenstrauch
On 04/27/2010 04:10 AM, Thomas Koch wrote: Zookeeper is also in the Debian unstable archive: http://packages.debian.org/sid/zookeeper I'm planning to push an update as soon as ZK 3.3.1 is out. Regards, Thomas Koch, http://www.koch.ro Thanks for the heads up. Unfortunately doesn't help me,

Re: Sequence Number Generation With Zookeeper

2010-08-05 Thread David Rosenstrauch
On 08/05/2010 06:31 PM, Jonathan Holloway wrote: Hi all, I'm looking at using Zookeeper for distributed sequence number generation. What's the best way to do this currently? Is there a particular recipe available for this? My so far involve: a) Creating a node with PERSISTENT_SEQUENTIAL

Re: Sequence Number Generation With Zookeeper

2010-08-06 Thread David Rosenstrauch
Perhaps. I'd have to ask my boss for permission to release the code. Is this something that would be interesting/useful to other people? If so, I can ask about it. DR On 08/05/2010 11:02 PM, Jonathan Holloway wrote: Hi David, We did discuss potentially doing this as well. It would be

Re: Sequence Number Generation With Zookeeper

2010-08-06 Thread David Rosenstrauch
I'll run it by my boss next week. DR On 08/06/2010 07:30 PM, Mahadev Konar wrote: Hi David, I think it would be really useful. It would be very helpful for someone looking for geenrating unique tokens/generations ids ( I can think of plenty of applications for this). Please do consider

Re: Sequence Number Generation With Zookeeper

2010-08-10 Thread David Rosenstrauch
need to tweak in the code to make it release-ready. (e.g., Change package names? Slap an Apache license on it? etc.) Thanks, DR On 08/06/2010 10:39 PM, David Rosenstrauch wrote: I'll run it by my boss next week. DR On 08/06/2010 07:30 PM, Mahadev Konar wrote: Hi David, I think it would

Re: Sequence Number Generation With Zookeeper

2010-08-10 Thread David Rosenstrauch
, David Rosenstrauch wrote: Good news! I got approval to release this code! (Man, I love working for a startup!!!) :-) So anyone know: what's the next step? Do I need to obtain commit privileges? Or do I deliver the code to someone who has commit privs who shepherds this for me? Also, what

Create node with ancestors?

2010-10-05 Thread David Rosenstrauch
The ZK create method explicitly states in the documentation If the parent node does not exist in the ZooKeeper, a KeeperException with error code KeeperException.NoNode will be thrown. (

Re: Create node with ancestors?

2010-10-07 Thread David Rosenstrauch
On 10/05/2010 06:20 PM, David Rosenstrauch wrote: The ZK create method explicitly states in the documentation If the parent node does not exist in the ZooKeeper, a KeeperException with error code KeeperException.NoNode will be thrown. ( (http://hadoop.apache.org/zookeeper/docs/current/api/org

Re: Create node with ancestors?

2010-10-07 Thread David Rosenstrauch
if you're doing it frequently (which we don't do), but it works for us. ~Jared On Thu, Oct 7, 2010 at 11:35 AM, David Rosenstrauchdar...@darose.net wrote: On 10/05/2010 06:20 PM, David Rosenstrauch wrote: The ZK create method explicitly states in the documentation If the parent node does

Re: Testing zookeeper outside the source distribution?

2010-10-18 Thread David Rosenstrauch
On 10/18/2010 08:17 AM, Anthony Urso wrote: Anyone have any pointers on how to test against ZK outside of the source distribution? All the fun classes (e.g. ClientBase) do not make it into the ZK release jar. Right now I am manually running a ZK node for the unit tests to connect to prior to