Re: Concurrent reads and writes on BookKeeper
Well Flavio, it is a extremely simple prototype where a primary broadcast updates on a single integer to backups. So we gonna have (n-1) reads for every write in a cluster of size n. I think sequential nodes in Zookeeper are fine for now, But I don't know if I am going to review that if things begin to get more complex. Tks a lot, André Oriani > Hi Andre, To guarantee that two clients that read from a ledger will > read the same sequence of entries, we need to make sure that there is > agreement on the end of the sequence. A client is still able to read > from an open ledger, though. We have an open jira about informing > clients of the progress of an open ledger (ZOOKEEPER-462), but we > haven't reached agreement on it yet. Some folks think that it is best > that each application use the mechanism it finds best. One option is > to have the writer writing periodically to a ZooKeeper znode to inform > of its progress. > > I would need to know more detail of your application before > recommending you to stick with BookKeeper or switch to ZooKeeper. If > your workload is dominated by writes, then BookKeeper might be a > better option. > > -Flavio > > On May 19, 2010, at 1:29 AM, André Oriani wrote: > >> Sorry, I forgot the subject on my last message :| >> >> Hi all, >> I was considering BookKeeper to implement some server replicated >> application having one primary server as writer and many backup >> servers >> reading from BookKeeper concurrently. The last documentation a I had >> access says "This writer has to execute a close ledger operation >> before >> any other client can read from it." So readers cannot ready any >> entry on >> the ledger, even the already committed ones until writer stops >> writing to >> the ledger,i.e, closes it. Is my understanding right ? Should I >> then use >> Zookeeper directly to achieve what I want ? >> >> >> Thanks for the attention, >> André Oriani >> >> >> >> >> >> > >
Re: Concurrent reads and writes on BookKeeper
On 05/20/2010 08:42 AM, Flavio Junqueira wrote: We have such a mechanism already, as Utkarsh mentions in the jira. The question is if we need more sophisticated mechanisms implemented, or if we should leave to the application to implement it. For now, we haven't felt the need for such extra mechanisms implemented along with BK, but I'd certainly be happy to hear a different perspective. Ok, was just saying that we shouldn't be too strict about it (impls available "out of the box"). Otw we run into situations similar to zk recipes where multiple users were re-implementing common patterns. Having said that, we have interesting projects to get folks involved with BK, but I don't have it clear that this is one of them. It would be great if you could enter JIRAs on this (projects), perhaps also a wiki 'interesting projects around bk (or hedwig, etc...)' page that catalogs those JIRAs. Thanks! Patrick -Flavio On May 20, 2010, at 1:36 AM, Patrick Hunt wrote: On 05/19/2010 01:23 PM, Flavio Junqueira wrote: Hi Andre, To guarantee that two clients that read from a ledger will read the same sequence of entries, we need to make sure that there is agreement on the end of the sequence. A client is still able to read from an open ledger, though. We have an open jira about informing clients of the progress of an open ledger (ZOOKEEPER-462), but we haven't reached agreement on it yet. Some folks think that it is best that each application use the mechanism it finds best. One option is to have the writer writing periodically to a ZooKeeper znode to inform of its progress. Hi Flavio. Seems like wrapping up a couple/few of these options in the client library (or "a" client library) would be useful for users -- reuse rather than everyone reinvent. Similar to how we now provide recipes in zk source base rather than everyone rewriting the basic locks/queues... Would be a great project I would think for someone interested in getting started with bk (and to some extent zk) development. Patrick I would need to know more detail of your application before recommending you to stick with BookKeeper or switch to ZooKeeper. If your workload is dominated by writes, then BookKeeper might be a better option. -Flavio On May 19, 2010, at 1:29 AM, André Oriani wrote: Sorry, I forgot the subject on my last message :| Hi all, I was considering BookKeeper to implement some server replicated application having one primary server as writer and many backup servers reading from BookKeeper concurrently. The last documentation a I had access says "This writer has to execute a close ledger operation before any other client can read from it." So readers cannot ready any entry on the ledger, even the already committed ones until writer stops writing to the ledger,i.e, closes it. Is my understanding right ? Should I then use Zookeeper directly to achieve what I want ? Thanks for the attention, André Oriani
Re: Zookeeper EventThread and SendThread
Seems like a likely possibility. We definetly don't have as many application threads as there are of these. Possibly we are keeping references to handles somehow though. We will look into that possibility. Thanks! Nick Bailey Rackspace Hosting Software Developer, Email & Apps nicholas.bai...@rackspace.com On Thursday, May 20, 2010 3:11pm, "Mahadev Konar" said: > > Hi Nick, > These threads are spawned with each zookeeper client handle. As soon as you > create a zookeeper client object these threads are spawned. > > Are yu creating too many zookeeper client objects in your application? > > Htanks > mahadev > > On 5/20/10 11:30 AM, "Nick Bailey" wrote: > >> Hey guys, >> >> Question regarding zookeeper's EventThread and SendThread. I'm not quite sure >> what these are used for but a stacktrace of our client application contains >> lines similar to >> >> "pool-2-thread-20-EventThread" daemon prio=10 tid=0x2aac3cb29c00 >> nid=0x75d >> waiting on condition [0x6b08..0x6b080b10] >> java.lang.Thread.State: WAITING (parking) >> at sun.misc.Unsafe.park(Native Method) >> - parking to wait for <0x2aab1f577250> (a >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) >> at >> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Ab >> stractQueuedSynchronizer.java:1925) >> at >> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358) >> at >> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:414) >> >> "pool-2-thread-20-SendThread" daemon prio=10 tid=0x2aac3c35d400 nid=0x75c >> runnable [0x70ede000..0x70edeb90] >> java.lang.Thread.State: RUNNABLE >> at sun.nio.ch.EPollArrayWrapper.epollWait(Native >> Method) >> at >> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215) >> at >> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65) >> at >> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) >> - locked <0x2aab1f571d08> (a sun.nio.ch.Util$1) >> - locked <0x2aab1f571cf0> (a >> java.util.Collections$UnmodifiableSet) >> - locked <0x2aab1f5715b8> (a >> sun.nio.ch.EPollSelectorImpl) >> at >> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) >> at >> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:921) >> >> There are pairs of threads ranging from thread-1 to thread-50 and also >> multiple pairs of these threads. As in pool-2-thread-20-SendThread is the >> name of multiple threads in the trace. I'm debugging some load issues with >> our system and am suspicious that the large amount of zookeeper threads is >> contributing. Would anyone be able to elaborate on the purpose of these >> threads and how they are spawned? >> >> Thanks, >> >> Nick Bailey >> Rackspace Hosting >> Software Developer, Email & Apps >> nicholas.bai...@rackspace.com >> > >
Re: Zookeeper EventThread and SendThread
Hi Nick, These threads are spawned with each zookeeper client handle. As soon as you create a zookeeper client object these threads are spawned. Are yu creating too many zookeeper client objects in your application? Htanks mahadev On 5/20/10 11:30 AM, "Nick Bailey" wrote: > Hey guys, > > Question regarding zookeeper's EventThread and SendThread. I'm not quite sure > what these are used for but a stacktrace of our client application contains > lines similar to > > "pool-2-thread-20-EventThread" daemon prio=10 tid=0x2aac3cb29c00 nid=0x75d > waiting on condition [0x6b08..0x6b080b10] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x2aab1f577250> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Ab > stractQueuedSynchronizer.java:1925) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358) > at > org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:414) > > "pool-2-thread-20-SendThread" daemon prio=10 tid=0x2aac3c35d400 nid=0x75c > runnable [0x70ede000..0x70edeb90] > java.lang.Thread.State: RUNNABLE > at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) > at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215) > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65) > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) > - locked <0x2aab1f571d08> (a sun.nio.ch.Util$1) > - locked <0x2aab1f571cf0> (a > java.util.Collections$UnmodifiableSet) > - locked <0x2aab1f5715b8> (a sun.nio.ch.EPollSelectorImpl) > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:921) > > There are pairs of threads ranging from thread-1 to thread-50 and also > multiple pairs of these threads. As in pool-2-thread-20-SendThread is the > name of multiple threads in the trace. I'm debugging some load issues with > our system and am suspicious that the large amount of zookeeper threads is > contributing. Would anyone be able to elaborate on the purpose of these > threads and how they are spawned? > > Thanks, > > Nick Bailey > Rackspace Hosting > Software Developer, Email & Apps > nicholas.bai...@rackspace.com >
Zookeeper EventThread and SendThread
Hey guys, Question regarding zookeeper's EventThread and SendThread. I'm not quite sure what these are used for but a stacktrace of our client application contains lines similar to "pool-2-thread-20-EventThread" daemon prio=10 tid=0x2aac3cb29c00 nid=0x75d waiting on condition [0x6b08..0x6b080b10] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x2aab1f577250> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:414) "pool-2-thread-20-SendThread" daemon prio=10 tid=0x2aac3c35d400 nid=0x75c runnable [0x70ede000..0x70edeb90] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x2aab1f571d08> (a sun.nio.ch.Util$1) - locked <0x2aab1f571cf0> (a java.util.Collections$UnmodifiableSet) - locked <0x2aab1f5715b8> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:921) There are pairs of threads ranging from thread-1 to thread-50 and also multiple pairs of these threads. As in pool-2-thread-20-SendThread is the name of multiple threads in the trace. I'm debugging some load issues with our system and am suspicious that the large amount of zookeeper threads is contributing. Would anyone be able to elaborate on the purpose of these threads and how they are spawned? Thanks, Nick Bailey Rackspace Hosting Software Developer, Email & Apps nicholas.bai...@rackspace.com
Re: problem connecting to zookeeper server
good catch lei! if this helps gregory, can you open a jira to throw an
exception in this situation. we should be throwing an invalid argument
exception or something in this case.
thanx
ben
On 05/20/2010 09:04 AM, Lei Zhang wrote:
Seems you are passing in wrong arguments:
Should have been:
public ZooKeeper(String connectString, int sessionTimeout, Watcher
watcher)
throws IOException
What you have in your client code is:
On Thu, May 20, 2010 at 5:21 AM, Gregory Haskins
wrote:
public App() throws Exception {
zk = new ZooKeeper("192.168.1.124:2181", 0, this);
}
Try use a sensible timeout value such as 2. The error you are getting
means the server has timed out the session.
Hope this unstucks you.
Re: problem connecting to zookeeper server
Seems you are passing in wrong arguments:
Should have been:
public ZooKeeper(String connectString, int sessionTimeout, Watcher
watcher)
throws IOException
What you have in your client code is:
On Thu, May 20, 2010 at 5:21 AM, Gregory Haskins
wrote:
>
>
>public App() throws Exception {
>zk = new ZooKeeper("192.168.1.124:2181", 0, this);
>}
>
>
Try use a sensible timeout value such as 2. The error you are getting
means the server has timed out the session.
Hope this unstucks you.
Re: Concurrent reads and writes on BookKeeper
We have such a mechanism already, as Utkarsh mentions in the jira. The question is if we need more sophisticated mechanisms implemented, or if we should leave to the application to implement it. For now, we haven't felt the need for such extra mechanisms implemented along with BK, but I'd certainly be happy to hear a different perspective. Having said that, we have interesting projects to get folks involved with BK, but I don't have it clear that this is one of them. -Flavio On May 20, 2010, at 1:36 AM, Patrick Hunt wrote: On 05/19/2010 01:23 PM, Flavio Junqueira wrote: Hi Andre, To guarantee that two clients that read from a ledger will read the same sequence of entries, we need to make sure that there is agreement on the end of the sequence. A client is still able to read from an open ledger, though. We have an open jira about informing clients of the progress of an open ledger (ZOOKEEPER-462), but we haven't reached agreement on it yet. Some folks think that it is best that each application use the mechanism it finds best. One option is to have the writer writing periodically to a ZooKeeper znode to inform of its progress. Hi Flavio. Seems like wrapping up a couple/few of these options in the client library (or "a" client library) would be useful for users -- reuse rather than everyone reinvent. Similar to how we now provide recipes in zk source base rather than everyone rewriting the basic locks/queues... Would be a great project I would think for someone interested in getting started with bk (and to some extent zk) development. Patrick I would need to know more detail of your application before recommending you to stick with BookKeeper or switch to ZooKeeper. If your workload is dominated by writes, then BookKeeper might be a better option. -Flavio On May 19, 2010, at 1:29 AM, André Oriani wrote: Sorry, I forgot the subject on my last message :| Hi all, I was considering BookKeeper to implement some server replicated application having one primary server as writer and many backup servers reading from BookKeeper concurrently. The last documentation a I had access says "This writer has to execute a close ledger operation before any other client can read from it." So readers cannot ready any entry on the ledger, even the already committed ones until writer stops writing to the ledger,i.e, closes it. Is my understanding right ? Should I then use Zookeeper directly to achieve what I want ? Thanks for the attention, André Oriani
problem connecting to zookeeper server
Hi,
ZooKeeper newbie here, so probably simple 'pilot error', but yet I am
stuck so any help appreciated.
I am trying to stand up a zookeeper environment so I can check things
out. I have an opensuse 11.2 virtual machine that I run the server in,
and then I try to write a java client to connect to it from an OSX
10.6.3 machine. No matter what combination of v3.2.2 or v.3.3.1 I try
(e.g. both client and server running v3.2.2, v3.3.1, or mixed) I get an
error similar to the following:
Client side:
Exception in thread "main"
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /myapp
at
org.apache.zookeeper.KeeperException.create(zookeeper:KeeperException.java):90)
at
org.apache.zookeeper.KeeperException.create(zookeeper:KeeperException.java):42)
at
org.apache.zookeeper.ZooKeeper.create(zookeeper:ZooKeeper.java):608)
at com.novell.lsg.zookeeperclient.App.run(App.java:32)
at com.novell.lsg.zookeeperclient.App.main(App.java:50)
Server side:
2010-05-20 07:59:31,517 - WARN
[NIOServerCxn.Factory:2181:nioserverc...@518] - Exception causing close
of session 0x0 due to java.io.IOException: Read error
2010-05-20 07:59:31,518 - INFO
[NIOServerCxn.Factory:2181:nioserverc...@857] - closing session:0x0
NIOServerCnxn: java.nio.channels.SocketChannel[connected
local=/192.168.1.124:2181 remote=/192.168.1.141:50135]
The server side error varies slightly depending on the versions in use
at the time, but its always related to problems reading the client
stream. In this particular run, I have v3.2.2 running on both sides.
Any help appreciated!
In case it matters, here is my client
package com.novell.lsg.zookeeperclient;
import java.util.List;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.WatchedEvent;
import org.apache.zookeeper.Watcher;
import org.apache.zookeeper.ZooKeeper;
/**
* Hello world!
*
*/
public class App implements Watcher {
ZooKeeper zk;
boolean dead = false;
public App() throws Exception {
zk = new ZooKeeper("192.168.1.124:2181", 0, this);
}
private void getChildren() throws Exception {
List children = zk.getChildren("/", true);
System.out.println("Current child list:");
for (String child : children) {
System.out.println(child);
}
}
private void run() throws Exception {
String znode = zk.create("/myapp", null, null,
CreateMode.EPHEMERAL_SEQUENTIAL);
System.out.println("Created " + znode);
getChildren();
try {
synchronized (this) {
while (!this.dead) {
wait();
}
}
} catch (InterruptedException e) {
}
System.out.println("Exiting");
}
public static void main(String[] args) throws Exception {
new App().run();
}
public void process(WatchedEvent event) {
System.out.println("Event: " + event.getType().toString());
}
}
signature.asc
Description: OpenPGP digital signature
