Roy Perkins created SOLR-13216:
----------------------------------
Summary: Trouble restoring a collection
Key: SOLR-13216
URL: https://issues.apache.org/jira/browse/SOLR-13216
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Backup/Restore
Affects Versions: 6.6.5
Reporter: Roy Perkins
I'm having a weird issue when attempting to restore a collection from our prod
cluster to our staging cluster. The restore seems to be moving along normally,
and then right at the end, the data gets dumped altogether. Below are relevant
messages in the logs:
{{2019-02-04 12:51:57.465 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is2_Lucene50_0.tip to restore directory}}
{{2019-02-04 12:51:57.524 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is3.fdt to restore directory}}
{{2019-02-04 12:51:57.590 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is3.fdx to restore directory}}
{{2019-02-04 12:51:57.642 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is3.fnm to restore directory}}
{{2019-02-04 12:51:57.707 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is3.si to restore directory}}
{{2019-02-04 12:51:57.760 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is3_Lucene50_0.doc to restore directory}}
{{2019-02-04 12:51:57.812 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is3_Lucene50_0.tim to restore directory}}
{{2019-02-04 12:51:57.878 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is3_Lucene50_0.tip to restore directory}}
{{2019-02-04 12:51:57.936 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4.fdt to restore directory}}
{{2019-02-04 12:51:58.003 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4.fdx to restore directory}}
{{2019-02-04 12:51:58.057 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4.fnm to restore directory}}
{{2019-02-04 12:51:58.124 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4.nvd to restore directory}}
{{2019-02-04 12:51:58.191 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4.nvm to restore directory}}
{{2019-02-04 12:51:58.244 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4.si to restore directory}}
{{2019-02-04 12:51:58.298 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4_Lucene50_0.doc to restore directory}}
{{2019-02-04 12:51:58.350 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4_Lucene50_0.pos to restore directory}}
{{2019-02-04 12:51:58.402 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4_Lucene50_0.tim to restore directory}}
{{2019-02-04 12:51:58.467 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
_406is4_Lucene50_0.tip to restore directory}}
{{2019-02-04 12:51:58.520 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Copying file
segments_3x02 to restore directory}}
{{2019-02-04 12:51:58.573 INFO
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.c.SolrCore Updating index
properties... index=restore.20190204124745615}}
{{2019-02-04 12:51:58.599 WARN
(parallelCoreAdminExecutor-5-thread-2-processing-n:solrmcstg11.dc3.homes.com:8983_solr
100019616170409755642 RESTORECORE) [ ] o.a.s.h.RestoreCore Could not switch to
restored index. Rolling back to the current index}}
{{org.apache.lucene.index.CorruptIndexException: Unexpected file read error
while reading index.
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/data/solr/slprod-02-04-2019_shard3_replica0/data/restore.20190204124745615/segments_3x02")))}}
{{ at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:290)}}
{{ at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:930)}}
{{ at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:118)}}
{{ at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:93)}}
{{ at
org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:257)}}
{{ at
org.apache.solr.update.DefaultSolrCoreState.changeWriter(DefaultSolrCoreState.java:220)}}
{{ at
org.apache.solr.update.DefaultSolrCoreState.newIndexWriter(DefaultSolrCoreState.java:229)}}
{{ at
org.apache.solr.update.DirectUpdateHandler2.newIndexWriter(DirectUpdateHandler2.java:726)}}
{{ at org.apache.solr.handler.RestoreCore.doRestore(RestoreCore.java:108)}}
{{ at
org.apache.solr.handler.admin.RestoreCoreOp.execute(RestoreCoreOp.java:65)}}
{{ at
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)}}
{{ at
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:388)}}
{{ at
org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:182)}}
{{ at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)}}
{{ at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)}}
{{ at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)}}
{{ at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)}}
{{ at java.lang.Thread.run(Thread.java:745)}}
{{Caused by: java.nio.file.NoSuchFileException:
/data/solr/slprod-02-04-2019_shard3_replica0/data/restore.20190204124745615/_406hsh.si}}
{{ at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)}}
{{ at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)}}
{{ at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)}}
{{ at
sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)}}
{{ at java.nio.channels.FileChannel.open(FileChannel.java:287)}}
{{ at java.nio.channels.FileChannel.open(FileChannel.java:335)}}
{{ at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238)}}
{{ at
org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)}}
{{ at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:137)}}
{{ at
org.apache.lucene.codecs.lucene62.Lucene62SegmentInfoFormat.read(Lucene62SegmentInfoFormat.java:89)}}
{{ at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:357)}}
{{ at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
