[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889704#comment-16889704 ] ASF subversion and git services commented on SOLR-11556: Commit 4e6a6ab46befaeceb330489be9a5889d3ce246b0 in lucene-solr's branch refs/heads/branch_8x from Mikhail Khludnev [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4e6a6ab ] SOLR-11556: fixing multiple backup repository support. > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-11556.patch, SOLR-11556.patch, SOLR-11556.patch, > SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889702#comment-16889702 ] ASF subversion and git services commented on SOLR-11556: Commit c173c2e8635edfe393cb7ca4491f8c43eb2d70d6 in lucene-solr's branch refs/heads/master from Mikhail Khludnev [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=c173c2e ] SOLR-11556: fixing multiple backup repository support. > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-11556.patch, SOLR-11556.patch, SOLR-11556.patch, > SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889273#comment-16889273 ] Lucene/Solr QA commented on SOLR-11556: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 41s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Release audit (RAT) {color} | {color:green} 3m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Check forbidden APIs {color} | {color:green} 3m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Validate source patterns {color} | {color:green} 3m 41s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 48m 55s{color} | {color:green} core in the patch passed. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 57m 13s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | SOLR-11556 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12975156/SOLR-11556.patch | | Optional Tests | compile javac unit ratsources checkforbiddenapis validatesourcepatterns | | uname | Linux lucene2-us-west.apache.org 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | ant | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh | | git revision | master / c5cabf1 | | ant | version: Apache Ant(TM) version 1.9.6 compiled on July 20 2018 | | Default Java | LTS | | Test Results | https://builds.apache.org/job/PreCommit-SOLR-Build/500/testReport/ | | modules | C: solr solr/core U: solr | | Console output | https://builds.apache.org/job/PreCommit-SOLR-Build/500/console | | Powered by | Apache Yetus 0.7.0 http://yetus.apache.org | This message was automatically generated. > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-11556.patch, SOLR-11556.patch, SOLR-11556.patch, > SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > j
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887962#comment-16887962 ] Mikhail Khludnev commented on SOLR-11556: - I'm going to commit it soon. Let me know if there is something which might be a concern. > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-11556.patch, SOLR-11556.patch, SOLR-11556.patch, > SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887845#comment-16887845 ] Lucene/Solr QA commented on SOLR-11556: --- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 4 new or modified test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 0s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Release audit (RAT) {color} | {color:green} 1m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Check forbidden APIs {color} | {color:green} 1m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} Validate source patterns {color} | {color:green} 1m 54s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 28m 6s{color} | {color:green} core in the patch passed. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 34m 54s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | SOLR-11556 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12975104/SOLR-11556.patch | | Optional Tests | compile javac unit ratsources checkforbiddenapis validatesourcepatterns | | uname | Linux lucene1-us-west 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon Sep 24 17:14:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | ant | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh | | git revision | master / 1fc4164 | | ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 | | Default Java | LTS | | Test Results | https://builds.apache.org/job/PreCommit-SOLR-Build/497/testReport/ | | modules | C: solr/core U: solr/core | | Console output | https://builds.apache.org/job/PreCommit-SOLR-Build/497/console | | Powered by | Apache Yetus 0.7.0 http://yetus.apache.org | This message was automatically generated. > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Mikhail Khludnev >Priority: Major > Attachments: SOLR-11556.patch, SOLR-11556.patch, SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.Thre
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886277#comment-16886277 ] Richard Goodman commented on SOLR-11556: Hey, With my current use case, we backup our indexes to a remote HDFS cluster. But we are slowly going to move this into AWS. We wanted to have more than 1 backup repository defined so that whilst we were testing backing up to AWS we could still backup to our current pipeline. I came across the exact same error as you _(currently running v7.4)_, and I applied the patch you submitted, and it solved the problem for me. So it looks like this didn't get chased up in newer versions. > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Timothy Potter >Priority: Major > Attachments: SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630869#comment-16630869 ] Ryan Rockenbaugh commented on SOLR-11556: - Thanks Varun! > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Timothy Potter >Priority: Major > Attachments: SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630835#comment-16630835 ] Varun Thacker commented on SOLR-11556: -- Hi Ryan, No promises but I'll plan on tackling this within the next couple of weeks. > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Timothy Potter >Priority: Major > Attachments: SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630832#comment-16630832 ] Ryan Rockenbaugh commented on SOLR-11556: - It looks like this patch has not be integrated into the 7.x branch yet. Any way to submit a request? > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Timothy Potter >Priority: Major > Attachments: SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221770#comment-16221770 ] Varun Thacker commented on SOLR-11556: -- Hi Tim, Patch looks good to me. TestBackupRepositoryFactory tests if the repository gets read properly but no AbstractCloudBackupRestoreTestCase actually defined two repositories to trigger it. Today TestHdfsCloudBackupRestore defines a custom solr.xml. I think we can do something similar in TestLocalFSCloudBackupRestore - define two repositories in the solr.xml string and {{getBackupRepoName}} could return the "local" repo name. > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Timothy Potter > Attachments: SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221582#comment-16221582 ] Hrishikesh Gadre commented on SOLR-11556: - Looks good. Core admin APIs may have similar issue? > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Timothy Potter > Attachments: SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11556) Backup/Restore with multiple BackupRepository objects defined results in the wrong repo being used.
[ https://issues.apache.org/jira/browse/SOLR-11556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221384#comment-16221384 ] Timothy Potter commented on SOLR-11556: --- Fix is easy enough, but why not just copy all incoming params into the Map in the CollectionsHandler? I don't get the selective copying being done here. > Backup/Restore with multiple BackupRepository objects defined results in the > wrong repo being used. > --- > > Key: SOLR-11556 > URL: https://issues.apache.org/jira/browse/SOLR-11556 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: Backup/Restore >Affects Versions: 6.3 >Reporter: Timothy Potter >Assignee: Timothy Potter > Attachments: SOLR-11556.patch > > > I defined two repos for backup/restore, one local and one remote on GCS, e.g. > {code} > > class="org.apache.solr.core.backup.repository.HdfsBackupRepository" > default="false"> > ... > > class="org.apache.solr.core.backup.repository.LocalFileSystemRepository" > default="false"> > /tmp/solr-backups > > > {code} > Since the CollectionHandler does not pass the "repository" param along, once > the BackupCmd gets the ZkNodeProps, it selects the wrong repo! > The error I'm seeing is: > {code} > 2017-10-26 17:07:27.326 ERROR > (OverseerThreadFactory-19-thread-1-processing-n:host:8983_solr) [ ] > o.a.s.c.OverseerCollectionMessageHandler Collection: product operation: > backup failed:java.nio.file.FileSystemNotFoundException: Provider "gs" not > installed > at java.nio.file.Paths.get(Paths.java:147) > at > org.apache.solr.core.backup.repository.LocalFileSystemRepository.resolve(LocalFileSystemRepository.java:82) > at org.apache.solr.cloud.BackupCmd.call(BackupCmd.java:99) > at > org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224) > at > org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > {code} > Notice the Local backup repo is being selected in the BackupCmd even though I > passed repository=hdfs in my backup command, e.g. > {code} > curl > "http://localhost:8983/solr/admin/collections?action=BACKUP&name=foo&collection=foo&location=gs://tjp-solr-test/backups&repository=hdfs"; > {code} > I think the fix here is to include the repository param, see patch. I'll fix > for the next 7.x release and those on 6 can just apply the patch here. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org