Thank you for the reference, Edward! It looks like Guano mostly accomplished what I want out of it. I spent a little time today digging in the guts of ZK and found a few ways to make taking a backup more efficient by delegating all the work to the server. Description of these ways does not fit the boundaries of a user mailing list and I intend to send a little write-up into the "dev" mailing list. Please let me know if you are interested in the subject and I can CC you.
On Fri, Jul 5, 2013 at 12:43 PM, Edward Ribeiro <[email protected]>wrote: > Sergey, > > Have you looked into the Guano Project (https://github.com/d2fn/guano) ? > It > seems to do exactly what you want, or at least as close a dump/restore so > that you can start from there. > > Edward > > > On Fri, Jul 5, 2013 at 1:40 PM, Sergey Maslyakov <[email protected]> > wrote: > > > The nature of the system that I am working with is mostly read-heavy. > > Writes happen rarely. In he event of disaster recovery to some point in > > time in the past, the system should sustain minor inconsistencies. > > > > I am also considering to have Zookeeper server create a copy of the > > DataTree, and then serialize it into a file, which will later be picked > up > > by the import client. This could be the most efficient way of taking a > > backup. > > > > > > > > On Fri, Jul 5, 2013 at 11:33 AM, Flavio Junqueira <[email protected] > > >wrote: > > > > > In your approach, would you "lock" the zookeeper state and read the > data > > > tree using getData/getChildren? If you have concurrent updates, then > you > > > may > > > end up having an inconsistent snapshot. > > > > > > -Flavio > > > > > > -----Original Message----- > > > From: Sergey Maslyakov [mailto:[email protected]] > > > Sent: 05 July 2013 17:12 > > > To: [email protected] > > > Subject: Re: Backup/restore of an emsemble > > > > > > Yes, Flavio, I looked at Exhibitor, but I need a pretty granular > control > > > over a cluster of ZK servers. This is why I'm inclined to build > something > > > by > > > hand. So far, a pair of external export and import clients seems like a > > > promising approach. > > > > > > Export would connect to the ensemble and dump out the data into a file > on > > > disk. Import would connect, wipe out the namespace, and then reload the > > > data > > > from the file that was earlier created by the export client. > > > > > > > > > > > > On Fri, Jul 5, 2013 at 8:31 AM, Flavio Junqueira > > > <[email protected]>wrote: > > > > > > > Sergey, > > > > > > > > Have you had a look at Exhibitor? > > > > > > > > https://github.com/Netflix/exhibitor > > > > > > > > -Flavio > > > > > > > > -----Original Message----- > > > > From: Sergey Maslyakov [mailto:[email protected]] > > > > Sent: 05 July 2013 04:39 > > > > To: [email protected] > > > > Subject: Backup/restore of an emsemble > > > > > > > > A while ago, Jack Ma asked this question: > > > > > > > > > http://mail-archives.apache.org/mod_mbox/zookeeper-user/201306.mbox/%3 > > > > CCAB%2 > > > > BcfdyPDpbUh5FyDT%3D9mU%3DFCHEA1AZpkF6X0nN1t4mjwqu2tA% > 40mail.gmail.com% > > > > 3E > > > > > > > > I wonder if there were any helpful suggestions that did not go into > > > > the mailing list. > > > > > > > > I am mostly concerned about restoring data in a Zookeeper ensemble. > > > > > > > > There is no document at the project web-site that would explain the > > > > restore procedure for a distributed deployment. The home-grown > > > > solution that involves stopping the whole cluster, wiping out > > > > databases on all but one server, restoring the database on one > server, > > > > and then bring up the cluster and pray that the populated server > > > > becomes the leader and populates the cluster. Such solution seems to > be > > > too error-prone. > > > > > > > > Does anyone have recommendations on how to make it robust? > > > > > > > > Maybe there is a way to force-populate the ensemble remotely? > > > > > > > > > > > > Thanks, > > > > /Sergey > > > > > > > > > > > > > > > > >
