Zookeeper stops
Hi, I have a zookeeper server running that can sometimes run for days and then quits: Is there somebody with a clue to the problem? I am running 64 bit Ubuntu with java version 1.6.0_18 OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1) OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode) Zookeeper 3.3.0 The log below has some context before it shows the fatal error. Our component.id=40676 indicates that it is the 40676th time that I ask ZK to publish this information. It has been seen to go up to half a million before stopping. Regards, Wim ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:28 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice, osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@68a1e081, component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server, component.id=40676, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5b9a6ad1 }]] ZooDiscovery Service Published: Aug 18, 2010 11:17:29 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice, osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]] [log;+0200 2010.08.18 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No async remote service interface found with name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]] 2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:zookeeperser...@262] - Severe unrecoverable error, exiting java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e (No such file or directory) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:209) at java.io.FileOutputStream.init(FileOutputStream.java:160) at org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224) at org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211) at org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260) at org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120) ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:37 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice, osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]]
Re: Zookeeper stops
Hi Wim, It mostly looks like that zookeeper is not able to create files on the /tmp filesystem. Is there is a space shortage or is it possible the file is being deleted as its being written to? Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem. Thanks mahadev On 8/19/10 1:15 AM, Wim Jongman wim.jong...@gmail.com wrote: Hi, I have a zookeeper server running that can sometimes run for days and then quits: Is there somebody with a clue to the problem? I am running 64 bit Ubuntu with java version 1.6.0_18 OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1) OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode) Zookeeper 3.3.0 The log below has some context before it shows the fatal error. Our component.id=40676 indicates that it is the 40676th time that I ask ZK to publish this information. It has been seen to go up to half a million before stopping. Regards, Wim ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:28 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice, osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@68a1e081, component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server, component.id=40676, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5b9a6ad1 }]] ZooDiscovery Service Published: Aug 18, 2010 11:17:29 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice, osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]] [log;+0200 2010.08.18 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No async remote service interface found with name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]] 2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:zookeeperser...@262] - Severe unrecoverable error, exiting java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e (No such file or directory) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:209) at java.io.FileOutputStream.init(FileOutputStream.java:160) at org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224) at org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211) at org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260) at org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120) ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:37 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice, osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]]
Re: Zookeeper stops
Also, /tmp is not a great place to keep things that are intended for persistence. On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar maha...@yahoo-inc.comwrote: Hi Wim, It mostly looks like that zookeeper is not able to create files on the /tmp filesystem. Is there is a space shortage or is it possible the file is being deleted as its being written to? Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem. Thanks mahadev On 8/19/10 1:15 AM, Wim Jongman wim.jong...@gmail.com wrote: Hi, I have a zookeeper server running that can sometimes run for days and then quits: Is there somebody with a clue to the problem? I am running 64 bit Ubuntu with java version 1.6.0_18 OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1) OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode) Zookeeper 3.3.0 The log below has some context before it shows the fatal error. Our component.id=40676 indicates that it is the 40676th time that I ask ZK to publish this information. It has been seen to go up to half a million before stopping. Regards, Wim ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:28 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@68a1e081, component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server, component.id=40676, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5b9a6ad1 }]] ZooDiscovery Service Published: Aug 18, 2010 11:17:29 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]] [log;+0200 2010.08.18 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No async remote service interface found with name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]] 2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:zookeeperser...@262] - Severe unrecoverable error, exiting java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e (No such file or directory) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:209) at java.io.FileOutputStream.init(FileOutputStream.java:160) at org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224) at org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211) at org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260) at org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120) ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:37 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]]
Re: Zookeeper stops
Ah, thanks guys! I did not realize that this was a user setting. Will try. Best regards, Wim On Thu, Aug 19, 2010 at 4:43 PM, Ted Dunning ted.dunn...@gmail.com wrote: Also, /tmp is not a great place to keep things that are intended for persistence. On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konar maha...@yahoo-inc.com wrote: Hi Wim, It mostly looks like that zookeeper is not able to create files on the /tmp filesystem. Is there is a space shortage or is it possible the file is being deleted as its being written to? Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem. Thanks mahadev On 8/19/10 1:15 AM, Wim Jongman wim.jong...@gmail.com wrote: Hi, I have a zookeeper server running that can sometimes run for days and then quits: Is there somebody with a clue to the problem? I am running 64 bit Ubuntu with java version 1.6.0_18 OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1) OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode) Zookeeper 3.3.0 The log below has some context before it shows the fatal error. Our component.id=40676 indicates that it is the 40676th time that I ask ZK to publish this information. It has been seen to go up to half a million before stopping. Regards, Wim ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:28 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@68a1e081, component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server, component.id=40676, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5b9a6ad1 }]] ZooDiscovery Service Published: Aug 18, 2010 11:17:29 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]] [log;+0200 2010.08.18 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No async remote service interface found with name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]] 2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:zookeeperser...@262] - Severe unrecoverable error, exiting java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e (No such file or directory) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:209) at java.io.FileOutputStream.init(FileOutputStream.java:160) at org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224) at org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211) at org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260) at org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120) ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:37 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id
Re: Zookeeper stops
+1 on that Ted. I frequently see this issue crop up as I just rebooted my server and lost all my data ... -- many os's will cleanup tmp on reboot. :-) Patrick On 08/19/2010 07:43 AM, Ted Dunning wrote: Also, /tmp is not a great place to keep things that are intended for persistence. On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konarmaha...@yahoo-inc.comwrote: Hi Wim, It mostly looks like that zookeeper is not able to create files on the /tmp filesystem. Is there is a space shortage or is it possible the file is being deleted as its being written to? Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem. Thanks mahadev On 8/19/10 1:15 AM, Wim Jongmanwim.jong...@gmail.com wrote: Hi, I have a zookeeper server running that can sometimes run for days and then quits: Is there somebody with a clue to the problem? I am running 64 bit Ubuntu with java version 1.6.0_18 OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1) OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode) Zookeeper 3.3.0 The log below has some context before it shows the fatal error. Our component.id=40676 indicates that it is the 40676th time that I ask ZK to publish this information. It has been seen to go up to half a million before stopping. Regards, Wim ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:28 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@68a1e081, component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server, component.id=40676, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5b9a6ad1 }]] ZooDiscovery Service Published: Aug 18, 2010 11:17:29 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]] [log;+0200 2010.08.18 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remoteservice;code=0;message=No async remote service interface found with name=org.eclipse.ecf.services.quotes.QuoteServiceAsync for proxy service class=org.eclipse.ecf.services.quotes.QuoteService;severity2;exception=null;children=[]]] 2010-08-18 23:17:37,057 - FATAL [Snapshot Thread:zookeeperser...@262] - Severe unrecoverable error, exiting java.io.FileNotFoundException: /tmp/zookeeperData/version-2/snapshot.13e2e (No such file or directory) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:209) at java.io.FileOutputStream.init(FileOutputStream.java:160) at org.apache.zookeeper.server.persistence.FileSnap.serialize(FileSnap.java:224) at org.apache.zookeeper.server.persistence.FileTxnSnapLog.save(FileTxnSnapLog.java:211) at org.apache.zookeeper.server.ZooKeeperServer.takeSnapshot(ZooKeeperServer.java:260) at org.apache.zookeeper.server.SyncRequestProcessor$1.run(SyncRequestProcessor.java:120) ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:37 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677,
Re: Zookeeper stops
Hi, But zk does default to /tmp? Regards, Wim On Thursday, August 19, 2010, Patrick Hunt ph...@apache.org wrote: +1 on that Ted. I frequently see this issue crop up as I just rebooted my server and lost all my data ... -- many os's will cleanup tmp on reboot. :-) Patrick On 08/19/2010 07:43 AM, Ted Dunning wrote: Also, /tmp is not a great place to keep things that are intended for persistence. On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konarmaha...@yahoo-inc.comwrote: Hi Wim, It mostly looks like that zookeeper is not able to create files on the /tmp filesystem. Is there is a space shortage or is it possible the file is being deleted as its being written to? Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem. Thanks mahadev On 8/19/10 1:15 AM, Wim Jongmanwim.jong...@gmail.com wrote: Hi, I have a zookeeper server running that can sometimes run for days and then quits: Is there somebody with a clue to the problem? I am running 64 bit Ubuntu with java version 1.6.0_18 OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1) OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode) Zookeeper 3.3.0 The log below has some context before it shows the fatal error. Our component.id=40676 indicates that it is the 40676th time that I ask ZK to publish this information. It has been seen to go up to half a million before stopping. Regards, Wim ZooDiscovery Service Unpublished: Aug 18, 2010 11:17:28 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@68a1e081, component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server, component.id=40676, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5b9a6ad1 }]] ZooDiscovery Service Published: Aug 18, 2010 11:17:29 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]] [log;+0200 2010.08.18 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remo
Re: Zookeeper stops
No. You configure it in the server configuration file. Patrick On 08/19/2010 01:19 PM, Wim Jongman wrote: Hi, But zk does default to /tmp? Regards, Wim On Thursday, August 19, 2010, Patrick Huntph...@apache.org wrote: +1 on that Ted. I frequently see this issue crop up as I just rebooted my server and lost all my data ... -- many os's will cleanup tmp on reboot. :-) Patrick On 08/19/2010 07:43 AM, Ted Dunning wrote: Also, /tmp is not a great place to keep things that are intended for persistence. On Thu, Aug 19, 2010 at 7:34 AM, Mahadev Konarmaha...@yahoo-inc.comwrote: Hi Wim, It mostly looks like that zookeeper is not able to create files on the /tmp filesystem. Is there is a space shortage or is it possible the file is being deleted as its being written to? Sometimes admins have a crontab on /tmp that cleans up the /tmp filesystem. Thanks mahadev On 8/19/10 1:15 AM, Wim Jongmanwim.jong...@gmail.comwrote: Hi, I have a zookeeper server running that can sometimes run for days and then quits: Is there somebody with a clue to the problem? I am running 64 bit Ubuntu with java version 1.6.0_18 OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1) OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode) Zookeeper 3.3.0 The log below has some context before it shows the fatal error. Our component.id=40676 indicates that it is the 40676th time that I ask ZK to publish this information. It has been seen to go up to half a million before stopping. Regards, Wim ZooDiscoveryService Unpublished: Aug 18, 2010 11:17:28 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_19q0FmlQF0wEwjSl6SpUTJRlV5g=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@68a1e081, component.name=Star Wars Quotes Service, ecf.sp.ect=ecf.generic.server, component.id=40676, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5b9a6ad1 }]] ZooDiscoveryService Published: Aug 18, 2010 11:17:29 PM. ServiceInfo[uri=osgiservices:// 188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;id=ServiceID[type=ServiceTypeID[typeName=_osgiservices._tcp.default._iana];location=osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=;full=_osgiservices._tcp.default._i...@osgiservices://188.40.116.87:3282/svc_u2GpWmF3YKSlTauWcwOMsDgiBxs=];priority=0;weight=0;props=ServiceProperties[{ecf.rsvc.ns=ecf.namespace.generic.remoteservice , osgi.remote.service.interfaces=org.eclipse.ecf.services.quotes.QuoteService, ecf.sp.cns=org.eclipse.ecf.core.identity.StringID, ecf.rsvc.id =org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@71bfa0a4, component.name=Eclipse Twitter, ecf.sp.ect=ecf.generic.server, component.id=40677, ecf.sp.cid=org.eclipse.ecf.discovery.serviceproperties$bytearraywrap...@5bcba953 }]] [log;+0200 2010.08.18 23:17:29:545;INFO;org.eclipse.ecf.remoteservice;org.eclipse.core.runtime.Status[plugin=org.eclipse.ecf.remo