On Thu, Jan 23, 2014 at 10:25 AM, Joe Caswell <[email protected]> wrote:
> Backing up LevelDB data files can be accomplished while the node is > running if the sst_x directories are backed up in numerical order. The > undesirable side effects of that could be duplicated data, inconsistent > manifest, or incomplete writes, which necessitates running the leveldb > repair process upon restoration for any vnode backed up while the node was > running. Since the data is initially written to the recovery log before > being appended to level 0, and any compaction operation fully writes the > data to its new location before removing it from its old location, if any > of these operations are interrupted, the data can be completely recovered > by leveldb repair. > > The only incomplete write that won't be recovered by the LevelDB repair > process is the initial write to the recovery log, limiting exposure to the > key being actively written at the time of the snapshot/backup. As long as > 2 vnodes in the same preflist are not backed up while simultaneously > writing the same key to the recovery log (i.e. rolling backups are good), > this key will be recovered by AAE/read repair after restoration. > Joe, thanks for the information. Will Riak detect a LevelDB that was not cleanly shutdown when it starts up and start repair on its own? If not, will Basho add a command to riak-admin to make it easy to repair LevelDB backends while restoring them? Any comments on the backup strategy for Yokozuna? Will it make use of Solr's Replication Handler, or something more lower level? Will the node need to be offline to backup it up? Ryan Zezeski any thoughts?
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
