Sorry for the back and forth everyone, I'm trying to understand the issue, but I absolutely cannot replicate this locally.
In the S3 console, I create a directory named "temp1/" *(Please note, this is not the same as "/temp1/", the actual folder in the bucket CANNOT start with a "/")* under the bucket that I am using. When I use the following Solr APIs, they ALL work for me: http://localhost:8983/solr/admin/collections?action=BACKUP&name=test-s3-test&repository=default&collection=test&location=temp1 http://localhost:8983/solr/admin/collections?action=BACKUP&name=test-s3-test&repository=default&collection=test&location=/temp1 http://localhost:8983/solr/admin/collections?action=BACKUP&name=test-s3-test&repository=default&collection=test&location=s3://temp1 http://localhost:8983/solr/admin/collections?action=BACKUP&name=test-s3-test&repository=default&collection=test&location=s3:///temp1 The only possible reason I can think of for so many people to be seeing errors is that they are pre-creating directories in S3 that have "/" prepended to the name. This is already documented in the ref guide <https://solr.apache.org/guide/8_11/making-and-restoring-backups.html#s3backuprepository>, but we can think of verbiage to make it more clear. I understand that we can make it so that "location" is not a required property and default it to "/", but that doesn't make sense for all backup repository types, such as a fileSystem backup. - Houston On Mon, Feb 7, 2022 at 8:54 AM Michael Conrad <[email protected]> wrote: > The restore process allows restoring to a collection with a different > name than the original. So I would expect the requirement to have a > collection already existing to be legit. > > The documentation does say (or at least implies) that to restore the > number of shards in the destination collection has to match the backup. > > On 2/5/22 12:02, Eric Charles wrote: > > ... and thx Houston and other participants to this thread. > > > > I have just restored the backup, and it works fine. The only point is > that I had to create the collection before restoring (I had dropped it). I > wonder if I have to recreate the collection with the same number of > shards... ? > > > > On 2022/02/05 16:55:26 Eric Charles wrote: > >> I have faced similar questions to get it working. I have first opened > an issue on the solr-operatorhttps:// > github.com/apache/solr-operator/issues/404 and then was able to > replicate the issue on my laptop, so came to the conclusion the issue was > at solr level, and not solr-operator. > >> > >> Without Sergio reply, I would not have been able to make sense of the > parameters and get is working. Thx! > >> > >> On 2022/01/27 14:01:46 Sergio García Maroto wrote: > >>> Hi, > >>> > >>> Correct. Using "location=backupfolder" didn't work for me. > >>> > >>> The only way I made it work is with location=s3:/ > >>> Below my sample url which works well. > >>> > http://servername:8983/solr/admin/collections?action=BACKUP&name=personbackup&collection=person&repository=s3&location=s3:/ > >>> > >>> > >>> On Tue, 25 Jan 2022 at 17:23, Houston Putman<[email protected]> > wrote: > >>> > >>>> Thanks for all of the information everybody. I want to determine if > this is > >>>> actually a bug before we release 9.0 > >>>> > >>>> First, I want to clear up the usage of the "location" parameter: > >>>> > >>>> - It is required, but you can provide "/" as an "empty" > directory, much > >>>> like "s3:/". > >>>> - You don't have to include "s3:/" or "s3://". You can you "/dir", > >>>> "dir", "s3:/dir" or "s3://dir". All of these options will > eventually be > >>>> converted to the "dir/" directory in your bucket. > >>>> - The s3 repository does not allow for directory names starting > with > >>>> "/". In general this is to allow all of the above ^ examples to > compute > >>>> to > >>>> the same thing without users being confused how many '/'s they > need > >>>> after > >>>> "s3:". Now I see that this may have contributed to the confusion, > but we > >>>> can always improve the documentation in the ref-guide. > >>>> > >>>> Now with these points cleared up, you said that you were unable to > use the > >>>> collections API calls with "location=backupfolder" did not work, even > >>>> though you had a directory in your bucket named "backupfolder". Just > to be > >>>> clear, in S3 the folder did not have a "/" prefix correct? If so, > this is > >>>> indeed a bug and something that we should fix. I'll do more thorough > >>>> testing once you confirm that the directory is following the > expectation. > >>>> > >>>> - Houston > >>>> > >>>> On Sat, Jan 22, 2022 at 3:32 AM Shawn Heisey<[email protected]> > wrote: > >>>> > >>>>> On 1/22/2022 12:38 AM, Sergio García Maroto wrote: > >>>>>> The only difference I can see i am using collection API trying to > >>>> backup > >>>>>> solr cloud collection. > >>>>> That could do it. I have done very little with SolrCloud beyond > >>>>> occasionally firing up the cloud example. > >>>>> > >>>>> The Collections API backup is most likely a different code path than > the > >>>>> replication handler backup. I wonder if the behavior you're seeing > >>>>> would be considered a bug. > >>>>> > >>>>> Thanks, > >>>>> Shawn > >>>>> >
