Re: Reconnecting to another host on failure but before session expires...

2009-01-06 Thread Kevin Burton
Crazy, I don't know how I missed that... Wouldn't it be cleaner to specify this as a List of host:port names? Kevin On Mon, Jan 5, 2009 at 10:36 AM, Thomas Vinod Johnson < thomas.john...@sun.com> wrote: > Kevin Burton wrote: > >> The ZooKeeper constructor only takes a host and port... not a list

group messaging, empheral nodes on zookeeper

2009-01-06 Thread Nuthan Ashkore
I've 2 separate questions regarding zookeeper: 1. Group Messaging: Is it possible to do group messaging with zookeeper 3.0.1 ? If not, are there any plans to add group messaging (say w/ casual ordering or even better total ordering) to future releases ? What would be the best approach if I need

Simpler ZooKeeper event interface....

2009-01-06 Thread Kevin Burton
Hey guys. I think I'm finally in the position to push ZK into production for a while to test it out. My biggest feedback (other than the small bugs I found) was that the API could be a bit simpler. I codified my thoughts here: http://pastebin.com/f2ecea8c7 http://pastebin.com/f62a01e9 Basicall

Re: group messaging, empheral nodes on zookeeper

2009-01-06 Thread Kevin Burton
> > 1. Group Messaging: > Is it possible to do group messaging with zookeeper 3.0.1 ? > If not, are there any plans to add group messaging (say w/ casual ordering > or even better total ordering) to future releases ? > What would be the best approach if I need to do group messaging using > zookeepe

Re: group messaging, empheral nodes on zookeeper

2009-01-06 Thread Flavio Junqueira
If I understand it correctly, you propose two mechanisms: 1- Have one single node, and modify the data of that znode; 2- Have a znode, say "/broadcast", and have clients creating a new child znode under "/broadcast" for every new message they want to broadcast. In case 1), if you are propos

Re: group messaging, empheral nodes on zookeeper

2009-01-06 Thread Kevin Burton
> > In case 1), if you are proposing to overwrite the content of the znode, > then you would need first to make sure that all receivers have already > received the previous message. This doesn't seem a good solution to me > because a client that wants to broadcast a message would have to wait until

Re: group messaging, empheral nodes on zookeeper

2009-01-06 Thread Flavio Junqueira
One potential problem with solution 2 is that a naive implementation may cause what we call a "herd effect": once there is a new message, zookeeper generates a large number of notifications and all these clients generate a request to receive the message. Depending on the requirements of you

Re: group messaging, empheral nodes on zookeeper

2009-01-06 Thread Flavio Junqueira
On Jan 6, 2009, at 6:55 PM, Kevin Burton wrote: In case 1), if you are proposing to overwrite the content of the znode, then you would need first to make sure that all receivers have already received the previous message. This doesn't seem a good solution to me because a client that want

RE: group messaging, empheral nodes on zookeeper

2009-01-06 Thread Benjamin Reed
Your first case only works if it is okay to miss messages as long as you see the latest. For your second case (using a directory with a child node for each message) I agree with flavio that you do not want to use ephemeral nodes. you can implement a ttl by having a daemon process that periodica

RE: group messaging, empheral nodes on zookeeper

2009-01-06 Thread Benjamin Reed
With respect to your 2nd question (in case it didn't get answered) nothing happens to the ephemeral nodes. as long as the client is able to talk to any active server before the session timeout the ephemeral nodes will stay. ben From: Nuthan Ashkore [nutha

Re: group messaging, empheral nodes on zookeeper

2009-01-06 Thread Mahadev Konar
I think ben already responded to your second question. Just to make sure all of the questions in 2 are answered -- - > 2. What happens to Empheral Nodes when a zookeeper server (not client) dies or > is separated from the group ? > > Supposing there are 5 zookeeper servers: server_1, ..., server

Re: Simpler ZooKeeper event interface....

2009-01-06 Thread Patrick Hunt
Hi Kevin, a couple of issues I noticed while looking at the pastebin: 1) you are ignoring the result codes in the callbacks, this could get you into trouble (say you do a getData on a node that has been deleted ie someone changes then immed. deletes the node) 2) I'm confused by one of your co

Re: Simpler ZooKeeper event interface....

2009-01-06 Thread Patrick Hunt
I should have been more clear on 3c - in this case you will get notified in the callback of CONNECTIONLOSS for any pending async requests, but as you are ignoring rc it may cause problems. Patrick Patrick Hunt wrote: Hi Kevin, a couple of issues I noticed while looking at the pastebin: 1) yo

Re: Simpler ZooKeeper event interface....

2009-01-06 Thread Kevin Burton
> 1) you are ignoring the result codes in the callbacks, this could get you > into trouble (say you do a getData on a node that has been deleted ie > someone changes then immed. deletes the node) > Actually I think I removed that FIXME. I'll try to fix this now.. Another issue was that the A

Re: Simpler ZooKeeper event interface....

2009-01-06 Thread Kevin Burton
> > > zk.getData( event.getPath(), true, this, null ); > >> >> Also, why not rename this getDataAsync I can't tell the difference just by looking at the method and the different number of arguments. Should make things a bit more straight forward. Kevin -- Founder/CEO Spinn3r.com Location:

Sending data during NodeDataChanged or NodeCreated

2009-01-06 Thread Kevin Burton
So if I understand this correctly, if I receive a NodeDataChanged event, and then attempt do do a read of that node, there's a race condition where the server could crash and I would be disconnected and my read would hit an Exception Or, the ACL could change and I no longer have permission to read

Re: Simpler ZooKeeper event interface....

2009-01-06 Thread Kevin Burton
> > 3) it's possible for your code to get notified of a change, but never > process the change. This might happen if: > a) a node changed watch fires > b) your client code runs an async getData > c) you are disconnected from the server > Also, this seems very confusing... If I run an async req

Re: Simpler ZooKeeper event interface....

2009-01-06 Thread Kevin Burton
This could be simplified if the semantics for reconnect were simplified. Is there any reason why I should know about a disconnect if ZK is just going to reconnect me to another server in 1ms? Why not hide *all* of this form the user and have the client re-issue watches on reconnect and hold off on

event re-issue on reconnect?

2009-01-06 Thread Kevin Burton
I have an event watching a file... and if I restart the server I get this: onConnect onData path: /foo, version: 4, data: '2333' onDisconnect onConnect onData path: /foo, version: 4, data: '2333' It re-issues the same version of the file. I can of course watch for this in my code but it seems lik

InterruptedException

2009-01-06 Thread Kevin Burton
Why does ZK throw InterruptedException? Shouldn't this be a KeeperException instead of a java system exception when interrupt() is called? The javadoc just says: "If the server transaction is interrupted" If this is a ZK related it should be KeeperException... -- Founder/CEO Spinn3r.com Loca

Re: InterruptedException

2009-01-06 Thread Mahadev Konar
Hi Kevin, The interrupt exception would be thrown in case any other thread tries to interrupt zookeeper threads during a client call (its not really interrupting the server but interrupting the client threads). Its like any synchronous operation that waits throwing an interrupted exception if inte

Re: event re-issue on reconnect?

2009-01-06 Thread Mahadev Konar
Does onData mean a datawatch? onConnect > onData path: /foo, version: 4, data: '2333' > onDisconnect > onConnect > onData path: /foo, version: 4, data: '2333' Are these the sequence of events that you get on the client? mahadev On 1/6/09 5:03 PM, "Kevin Burton" wrote: > I have an event watch

Re: Simpler ZooKeeper event interface....

2009-01-06 Thread Mahadev Konar
http://issues.apache.org/jira/browse/ZOOKEEPER-23 This has been fixed in zookeeper-3.0 release. Are you using a release from sourceforge? mahadev On 1/6/09 4:57 PM, "Kevin Burton" wrote: > This could be simplified if the semantics for reconnect were simplified. > Is there any reason why I sh

Re: Sending data during NodeDataChanged or NodeCreated

2009-01-06 Thread Mahadev Konar
> So if I understand this correctly, if I receive a NodeDataChanged event, and > then attempt do do a read of that node, there's a race condition where the > server could crash and I would be disconnected and my read would hit an > Exception > Or, the ACL could change and I no longer have permiss

Re: Simpler ZooKeeper event interface....

2009-01-06 Thread Mahadev Konar
Does javadoc help? :) Mahadev On 1/6/09 4:10 PM, "Kevin Burton" wrote: >> >> >> zk.getData( event.getPath(), true, this, null ); >> >>> >>> > Also, why not rename this getDataAsync I can't tell the difference just > by looking at the method and the different number of arguments. >

ouch, zookeeper infinite loop

2009-01-06 Thread Kevin Burton
Creating this node with this ACL: Created /foo setAcl /foo world:anyone:w Causes the exception included below. It's an infinite loop so it's just called over and over again filling my console. I'm just doing an exists( path, true ); ... setting a watch still causes the problem. java.lang.Null

Can ConnectionLossException be thrown when using multiple hosts?

2009-01-06 Thread Kevin Burton
Can this be thrown when using multiple servers as long as > 1 of them is online? Trying to figure out of I should try some type of reconnect if a single machine fails instead of failing altogether. Kevin -- Founder/CEO Spinn3r.com Location: San Francisco, CA AIM/YIM: sfburtonator Skype: burtonat

Does session expiration only happen during total ensemble failure or network split?

2009-01-06 Thread Kevin Burton
>From what I can tell, a session will only expire if it can't communicate with the ensemble due to the entire servers failing or the network splitting preventing a ZK node from seeing the servers. Correct? Then in this case, an application should in theory only care about session expiration and ZK

Re: event re-issue on reconnect?

2009-01-06 Thread Kevin Burton
On Tue, Jan 6, 2009 at 7:09 PM, Mahadev Konar wrote: > Does onData mean a datawatch? > SorryI should have recorded the actual events. It's a NodeCreated event being thrown when reconnecting this is the result of event.toString on the events as I see them. WatchedEvent: Server state ch

Re: Sending data during NodeDataChanged or NodeCreated

2009-01-06 Thread Kevin Burton
> > > It's almost certainly what the user wants anyway. > Its just that the watches are pretty lightweight and sending bytes around > is > just more work to do at the server. Though we should experiment with how > much more load it generates and how useful would it be to send out the > bytes > with

Re: Simpler ZooKeeper event interface....

2009-01-06 Thread Kevin Burton
3.0.1. my watches get recreated on the new server but I'm still too aware of connections. In fact, shouldn't disconnect be removed entirely? Or is this just advice telling the client that something bad might have happened? Kevin On Tue, Jan 6, 2009 at 7:12 PM, Mahadev Konar wrote: > http: