I feel that a "metaformat" has always been needed in BookKeeper (at least since 4.5) , here the problem is about using a non first level path
Anyway, from 4.7 to 4.9 we did a lot of refactors, and we finished the abstraction of the Metadata Service, giving the ability to support pluggable Metadata Services, namely we are supporting now ZooKeeper and Etcd (patially). Maybe there was not test or active user that shown up for this kind of "regression". Beside note: We are going to release 4.10 soon, maybe it is worth to check compatibility with the current master and/or setup periodic compatibility tests against current "master", we are deploying -SNAPSHOT version to Apache Maven Repositories for this purpose. This will help the community in preventing regressions For instance in 4.10 we invested my time in the new CLI, named 'bkctl' and if major OSS projects (like Pravega) migrate to the new CLI we will be more confident in deprecating the old CLI Il giorno gio 10 ott 2019 alle ore 11:01 <prajakta.belgu...@dell.com> ha scritto: > Thanks Enrico. > > > > This was not needed with Bookkeeper 4.7.3 and without any init/metaformat > we were able to bring up the bookie. > > What has changed in 4.9.1 because of which this is needed now? > > > > -Thanks, > > Prajakta > > > > *From:* Enrico Olivelli <eolive...@gmail.com> > *Sent:* Thursday, October 10, 2019 2:06 PM > *To:* user > *Cc:* Flavio Junqueira > *Subject:* Re: Error starting bookie using Apache Bookkeeper 4.9.1 > > > > [EXTERNAL EMAIL] > > Looking at the code I think that the short answer is "yes" > > > https://github.com/apache/bookkeeper/blob/2f996dcf0159f945f7ec97ce7402e5d293009444/bookkeeper-server/src/main/java/org/apache/bookkeeper/discover/ZKRegistrationManager.java#L403 > > > > we are using ZooKeeper#create, that does not recursively create all the > hierarchy. > > I feel this is due to the fact that usually a BooKeeper cluster stores > metadata in /ledgers, as a direct child of '/' > > > > IIRC in some product of my company we had the same problem and we simply > pre-created the znode structure before '/ledgers' > > > > Maybe you can give it a try. > > I also think it is worth to submit a simply patch in order to change the > behaviour and create the full hierarchy (the change is really simple) > > > > Hope that helprs > > Enrico > > > > > > > > Il giorno gio 10 ott 2019 alle ore 10:28 <prajakta.belgu...@dell.com> ha > scritto: > > Does that need to be created manually? > > > > *From:* Enrico Olivelli <eolive...@gmail.com> > *Sent:* Thursday, October 10, 2019 12:25 PM > *To:* user > *Cc:* Flavio Junqueira > *Subject:* Re: Error starting bookie using Apache Bookkeeper 4.9.1 > > > > [EXTERNAL EMAIL] > > I see your base path is /pravega/pravega/bookkeeper/ledgers > > > > Maybe the root path does not yet exist, I mean /pravega/pravega/bookkeeper > > > > Enrico > > > > Il giorno gio 10 ott 2019 alle ore 08:20 <prajakta.belgu...@dell.com> ha > scritto: > > Thanks for your response, Enrico. > > > > I tried using both `metaformat` and `initnewcluster` shell commands to > initialize zookeeper before starting the bookie. > > But both failed with similar exceptions…. > > > > Using `bin/bookkeeper shell metaformat -nonInteractive -force` I see … > > “ > > Exception in thread "main" java.util.concurrent.ExecutionException: > KeeperErrorCode = NoNode for /pravega/pravega/bookkeeper/ledgers > at > org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithMetadataBookieDriver(MetadataDrivers.java:378) > at > org.apache.bookkeeper.client.BookKeeperAdmin.format(BookKeeperAdmin.java:1150) > at > org.apache.bookkeeper.bookie.BookieShell$MetaFormatCmd.runCmd(BookieShell.java:328) > at > org.apache.bookkeeper.bookie.BookieShell$MyCommand.runCmd(BookieShell.java:277) > at org.apache.bookkeeper.bookie.BookieShell.run(BookieShell.java:3081) > at org.apache.bookkeeper.bookie.BookieShell.main(BookieShell.java:3172) > Caused by: org.apache.zookeeper.KeeperException$NoNodeException: > KeeperErrorCode = NoNode for /pravega/pravega/bookkeeper/ledgers > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:114) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient.access$1901(ZooKeeperClient.java:70) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient$9.call(ZooKeeperClient.java:711) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient$9.call(ZooKeeperClient.java:705) > at > org.apache.bookkeeper.zookeeper.ZooWorker.syncCallWithRetries(ZooWorker.java:140) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient.create(ZooKeeperClient.java:705) > at > org.apache.bookkeeper.discover.ZKRegistrationManager.prepareFormat(ZKRegistrationManager.java:403) > at > org.apache.bookkeeper.client.BookKeeperAdmin.lambda$format$2(BookKeeperAdmin.java:1152) > at > org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithMetadataBookieDriver(MetadataDrivers.java:373) > > > > “ > > Using ` bin/bookkeeper shell initnewcluster` I see: > > “ > > Exception in thread "main" java.util.concurrent.ExecutionException: > KeeperErrorCode = NoNode > at > org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithMetadataBookieDriver(MetadataDrivers.java:378) > at > org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithRegistrationManager(MetadataDrivers.java:398) > at > org.apache.bookkeeper.client.BookKeeperAdmin.initNewCluster(BookKeeperAdmin.java:1197) > at > org.apache.bookkeeper.bookie.BookieShell$InitNewCluster.runCmd(BookieShell.java:368) > at > org.apache.bookkeeper.bookie.BookieShell$MyCommand.runCmd(BookieShell.java:277) > at org.apache.bookkeeper.bookie.BookieShell.run(BookieShell.java:3081) > at org.apache.bookkeeper.bookie.BookieShell.main(BookieShell.java:3172) > Caused by: org.apache.zookeeper.KeeperException$NoNodeException: > KeeperErrorCode = NoNode > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:114) > at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1015) > at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:919) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient.access$801(ZooKeeperClient.java:70) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient$2.call(ZooKeeperClient.java:466) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient$2.call(ZooKeeperClient.java:455) > at > org.apache.bookkeeper.zookeeper.ZooWorker.syncCallWithRetries(ZooWorker.java:140) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient.multi(ZooKeeperClient.java:455) > at > org.apache.bookkeeper.discover.ZKRegistrationManager.initNewCluster(ZKRegistrationManager.java:453) > at > org.apache.bookkeeper.client.BookKeeperAdmin.lambda$initNewCluster$3(BookKeeperAdmin.java:1199) > at > org.apache.bookkeeper.meta.MetadataDrivers.lambda$runFunctionWithRegistrationManager$1(MetadataDrivers.java:398) > at > org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithMetadataBookieDriver(MetadataDrivers.java:373) > > “ > > What initialization step am I missing that needs to be done before > invoking: ` /opt/bookkeeper/scripts/entrypoint.sh bookie` ?? > > > > -Thanks, > > Prajakta > > > > *From:* Enrico Olivelli <eolive...@gmail.com> > *Sent:* Friday, October 4, 2019 9:09 PM > *To:* user@bookkeeper.apache.org > *Cc:* f...@apache.org > *Subject:* Re: Error starting bookie using Apache Bookkeeper 4.9.1 > > > > [EXTERNAL EMAIL] > > Did you format/init the cluster? > > > > Enrico > > > > Il ven 4 ott 2019, 16:33 <prajakta.belgu...@dell.com> ha scritto: > > Hi Bookkeeper Team, > > > > I’m a Pravega developer and I’m currently trying to use the recent > Bookkeeper version `4.9.1` with Pravega (docker image > apache/bookkeeper:4.9.1 <https://hub.docker.com/r/apache/bookkeeper/tags>) > > However, when starting a new bookie, I see this exception: > > “ > > 2019-10-03 14:53:43,836 - INFO - > [main-EventThread:ZooKeeperWatcherBase@130] - ZooKeeper client is > connected now. > 2019-10-03 14:53:43,860 - ERROR - [main:ZKRegistrationManager@379] - > BookKeeper metadata doesn't exist in zookeeper. Has the cluster been > initialized? Try running bin/bookkeeper shell metaformat > 2019-10-03 14:53:43,861 - INFO - [main:BookieNettyServer@424] - Shutting > down BookieNettyServer > 2019-10-03 14:53:43,875 - ERROR - [main:Main@223] - Failed to build > bookie server > org.apache.bookkeeper.bookie.BookieException$MetadataStoreException: > Failed to get cluster instance id > at > org.apache.bookkeeper.discover.ZKRegistrationManager.getClusterInstanceId(ZKRegistrationManager.java:392) > at > org.apache.bookkeeper.bookie.Bookie.checkEnvironmentWithStorageExpansion(Bookie.java:408) > at org.apache.bookkeeper.bookie.Bookie.checkEnvironment(Bookie.java:252) > at org.apache.bookkeeper.bookie.Bookie.<init>(Bookie.java:691) > at > org.apache.bookkeeper.proto.BookieServer.newBookie(BookieServer.java:137) > at > org.apache.bookkeeper.proto.BookieServer.<init>(BookieServer.java:106) > at > org.apache.bookkeeper.server.service.BookieService.<init>(BookieService.java:43) > at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:301) > at org.apache.bookkeeper.server.Main.doMain(Main.java:221) > at org.apache.bookkeeper.server.Main.main(Main.java:203) > Caused by: org.apache.zookeeper.KeeperException$NoNodeException: > KeeperErrorCode = NoNode for BookKeeper metadata > at > org.apache.bookkeeper.discover.ZKRegistrationManager.getClusterInstanceId(ZKRegistrationManager.java:382) > > “ > > > > If I try to run `metaformat` or `initnewCluster` using the bookkeeper > shell, before starting the new bookie, that too fails because the expected > znode is not present… > > Error with `metaformat`: > > “ > > Exception in thread "main" java.util.concurrent.ExecutionException: > KeeperErrorCode = NoNode for /pravega/pravega/bookkeeper/ledgers > at > org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithMetadataBookieDriver(MetadataDrivers.java:378) > at > org.apache.bookkeeper.client.BookKeeperAdmin.format(BookKeeperAdmin.java:1150) > at > org.apache.bookkeeper.bookie.BookieShell$MetaFormatCmd.runCmd(BookieShell.java:328) > at > org.apache.bookkeeper.bookie.BookieShell$MyCommand.runCmd(BookieShell.java:277) > at org.apache.bookkeeper.bookie.BookieShell.run(BookieShell.java:3081) > at org.apache.bookkeeper.bookie.BookieShell.main(BookieShell.java:3172) > Caused by: org.apache.zookeeper.KeeperException$NoNodeException: > KeeperErrorCode = NoNode for /pravega/pravega/bookkeeper/ledgers > at org.apache.zookeeper.KeeperException.create(KeeperException.java:114) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient.access$1901(ZooKeeperClient.java:70) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient$9.call(ZooKeeperClient.java:711) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient$9.call(ZooKeeperClient.java:705) > at > org.apache.bookkeeper.zookeeper.ZooWorker.syncCallWithRetries(ZooWorker.java:140) > at > org.apache.bookkeeper.zookeeper.ZooKeeperClient.create(ZooKeeperClient.java:705) > at > org.apache.bookkeeper.discover.ZKRegistrationManager.prepareFormat(ZKRegistrationManager.java:403) > at > org.apache.bookkeeper.client.BookKeeperAdmin.lambda$format$2(BookKeeperAdmin.java:1152) > at > org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithMetadataBookieDriver(MetadataDrivers.java:373 > > ” > > > > Could you please help me understand what could be causing this? > > > > -Thanks, > > Prajakta > > > > > > > > > >
