[
https://issues.apache.org/jira/browse/BLUR-132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13737847#comment-13737847
]
Rahul Challapalli commented on BLUR-132:
----------------------------------------
We can provide the backup in multiple ways :
1. Backup all shards(all tables) on a shard server onto local
filesystem(cluster/table/shard/files). While restoring from backup, every shard
server
reads from its local filesystem and copies the shards onto HDFS.
2. Bakup all shards on all shard servers onto a common HDFS location.
While restoring we would partition the shards onto shard servers.
We should also have some mechanism to restore from a backup. For us to restore
the index from a backup, we might as well need a
point-in-time copy of all the table descriptors.
How are we planning to expose this snapshot functionality (Shell, API, BOTH)?
Where are we even using LocalIndexServer?
I was able to take a backup by wrapping IndexDeletionPolicy with
SnapshotDeletionPolicy and then take a snapshot
and copy all the files to a local file system. This technically works even if
the index is being actively updated, but the way in
which the code is structured (DistributeIndexServer.openShard), we would only
get a BlurIndexReader when the shard is being updated.
but the sample code I have below is using the writer to take the snapshot. May
be there is a different way?
/*
* As of now we cannot use the snapshotDeletionPolicy
instance which we sent to IndexWriter
* as it is getting cloned internally by IndexWriter.
So we always get it from the LiveIndexWriterConfig
*/
SnapshotDeletionPolicy snapshotter =
(SnapshotDeletionPolicy)writer.getConfig().getIndexDeletionPolicy();
String commitIdentifier = generateCommitIdentifier();
IndexCommit indexCommit =
snapshotter.snapshot(commitIdentifier);
for (String filename : indexCommit.getFileNames()) {
if (!filename.equals("write.lock")) {
// take backup
}
}
Also What happens when multiple sources try to add documents to the same shard
simultaneously(using the same IndexWriter)?
Would really love to know your thoughts and appreciate it if someone can fill
in gaps in my understanding. Thank You.
> Create Index Snapshots
> ----------------------
>
> Key: BLUR-132
> URL: https://issues.apache.org/jira/browse/BLUR-132
> Project: Apache Blur
> Issue Type: New Feature
> Affects Versions: 0.3.0
> Reporter: Aaron McCurry
> Fix For: 0.3.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
