Re: Data Import Handler / Backup indexes

2015-11-23 Thread Jeff Wartes
The backup/restore approach in SOLR-5750 and in solrcloud_manager is really just that - copying the index files. On backup, it saves your index directories, and on restore, it puts them in the data dir, moves a pointer for the current index dir, and opens a new searcher. Both are mostly just

Re: Data Import Handler / Backup indexes

2015-11-22 Thread Erick Erickson
These are just Lucene indexes. There's the Cloud backup and restore that is being worked on. But if the index is static (i.e. not being indexed to), simply copying the data/index (well, actually the whole data index and subdirs) directory will backup and restore it. Copying the index directory

Re: Data Import Handler / Backup indexes

2015-11-21 Thread Brian Narsi
What are the caveats regarding the copy of a collection? At this time DIH takes only about 10 minutes. So in case of accidental delete we can just re-run the DIH. The reason I am thinking about backup is just in case records are deleted accidentally and the DIH cannot be run because the database

Data Import Handler / Backup indexes

2015-11-17 Thread Brian Narsi
I am using Data Import Handler to retrieve data from a database with full-import, clean = true, commit = true and optimize = true This has always worked correctly without any errors. But just to be on the safe side, I am thinking that we should do a backup before initiating Data Import Handler.

Re: Data Import Handler / Backup indexes

2015-11-17 Thread Brian Narsi
Sorry I forgot to mention that we are using SolrCloud 5.1.0. On Tue, Nov 17, 2015 at 12:09 PM, KNitin wrote: > afaik Data import handler does not offer backups. You can try using the > replication handler to backup data as you wish to any custom end point. > > You can

Re: Data Import Handler / Backup indexes

2015-11-17 Thread KNitin
afaik Data import handler does not offer backups. You can try using the replication handler to backup data as you wish to any custom end point. You can also try out : https://github.com/bloomreach/solrcloud-haft. This helps backup solr indices across clusters. On Tue, Nov 17, 2015 at 7:08 AM,

Re: Data Import Handler / Backup indexes

2015-11-17 Thread Jeff Wartes
https://github.com/whitepages/solrcloud_manager supports 5.x, and I added some backup/restore functionality similar to SOLR-5750 in the last release. Like SOLR-5750, this backup strategy requires a shared filesystem, but note that unlike SOLR-5750, I haven’t yet added any backup functionality