Hi Jason, thanks for having a look at this and your confirmation that you can reproduce the issue. Sorry, I forgot to mention that I started the server with SOLR_OPTS set to -Dsolr.disableConfigSetsCreateAuthChecks=true -Dsolr.allowPaths=/var/solr_backup
I created https://issues.apache.org/jira/browse/SOLR-15478 for it. I attached your steps as a comment. Yes, we tried to use RELOAD the collection, but it did not help to make the schema changes visible. I'll try RESTORE with collection.configName Kind Regards Steffen > -----Original Message----- > From: Jason Gerlowski <[email protected]> > Sent: Freitag, 11. Juni 2021 16:00 > To: [email protected] > Subject: Re: Schema Changes are not Visible after Collection Restore - Solr > 8.8.2 > > Hey Steffen, > > I can reproduce the issue on 8.8.2 using the steps you laid out in your > original email. (With one deviation - I had to enable auth to get around an > error about "Can't create a configset with an unauthenticated request from > a trusted baseConfigSet") > > The steps you're using to trigger this are a bit odd. Do you really edit the > configset stored with the backup as a part of your production workflow, or is > this just a step you introduced to simplify the reproduction? The /schema > API is the recommended way to handle configuration changes, followed by > editing the configset files and uploading them to ZooKeeper. None of that > quirkiness makes this any less of a bug though - it seems very much like > something's going wrong here. I can't imagine any reason why Solr would > intentionally not pick up fields until a restart. > > To further the reproduction a bit, it's possible to reproduce this without > backup/restore in the picture at all. The key piece seems to be that the new > (post-schema-edit) collection and config set have the same name as their > now-deleted counterparts. > > 1. Start Solr: bin/solr start -c -p 7574 2. Turn on auth (this was necessary > for > me to create a configset that extended "_default"): bin/solr auth enable - > type basicAuth -credentials solr:solrRocks -z localhost:8574 3. Create a > config > based on "_default": curl -ilk -X GET -u solr:solrRocks > "http://localhost:7574/solr/admin/configs?action=CREATE&name=testconfig > set&baseConfigSet=_default" > 4. Create a collection using this configset: curl -ilk -X GET -u > solr:solrRocks > "http://localhost:7574/solr/admin/collections?action=CREATE&name=coll1&; > collection.configName=testconfigset&numShards=1" > 5. Download the configset locally: bin/solr zk downconfig -d > downloaded_test_config -n testconfig2 -z localhost:8574 6. Drop a new > <field> definition into downloaded_test_config/conf/managed-schema > 7. Delete the existing collection: curl -ilk -X GET -u solr:solrRocks > "http://localhost:7574/solr/admin/collections?action=DELETE&name=coll1"; > 8. Delete the configset from ZK: curl -ilk -X GET -u solr:solrRocks > "http://localhost:7574/solr/admin/configs?action=DELETE&name=testconfigs > et" > 9. Create the config from the local copy: bin/solr zk upconfig -d > downloaded_test_config -n testconfigset -z localhost:8574 10. Create a new > collection with the same name as the previous > collection: curl -ilk -X GET -u solr:solrRocks > "http://localhost:7574/solr/admin/collections?action=CREATE&name=coll1&; > collection.configName=testconfigset&numShards=1" > 11. List the fields and show that the added field is missing: curl -ilk -X > GET -u > solr:solrRocks "http://localhost:7574/solr/coll1/schema/fields"; > > My best guess is that Solr is caching the configset (or the schemas parsed out > of them) somewhere that sticks around even after the corresponding > configset/collection is deleted. But I'm not very familiar with our configset > code, so I can't dig in further than that with the time I have. Hopefully > someone else can pick it up from here. In the meantime, you might be able > to workaround this by specifying a "collection.configName" param on your > RESTORE call - this will cause the backed up configset to be uploaded to ZK > under a different name - hopefully avoiding whatever caching is causing the > trouble here. > > Best, > > Jason > > On Fri, Jun 11, 2021 at 8:38 AM Jason Gerlowski <[email protected]> > wrote: > > > > Hey Steffen, > > > > I took a quick look at the backup/restore codepath involved here - > > surprisingly the restore code itself hasn't changed between 8.6.3 and > > 8.8.2. In both 8.6.3 and 8.8.2, if the configset mentioned in the > > backup has the same name as a config currently in ZooKeeper, the > > version in ZK is used. So fields added to the schema after the backup > > _should_ be visible. Obviously per your report this isn't happening > > though. To me that says that there must be some change of behavior in > > collection-creation or in schema-reading. > > > > I'm hoping to do a bit more digging to get to the bottom of this. In > > the meantime, would you mind filing a JIRA ticket regarding this > > behavior (if you haven't already)? > > > > One last question: you mentioned above that the field becomes visible > > after a restart - does a reload > > (/admin/collections?action=RELOAD&name=testcollection) have the same > > effect? If so that may be a more palatable work-around for you in the > > short-term... > > > > Best, > > > > Jason > > > > On Thu, Jun 3, 2021 at 9:32 AM Steffen Moldenhauer > > <[email protected]> wrote: > > > > > > we tried to upgrade from 8.6.3 to 8.8 but had trouble with collection > restore and schema changes. > > > If there were changes to the schema - they are not visible in the Schema > API. > > > We are using the backup/restore to transfer pre-built collections to > > > production systems and this issue prevents us from using the latest > > > version 8.8.2 > > > > > > These are the basic steps to reproduce the issue on a "solr start -c" : > > > > > > For preparation create a config set, collection and backup: > > > http://localhost:8983/solr/admin/configs?action=CREATE&name=testcoll > > > ection&baseConfigSet=_default > > > http://localhost:8983/solr/admin/collections?action=CREATE&collectio > > > n.configName=testcollection&name=testcollection&numShards=1 > > > http://localhost:8983/solr/admin/collections?action=BACKUP&location= > > > /var/solr_backup&name=testcollection_backup&collection=testcollectio > > > n > > > > > > Check the list of fields with the schema API: > > > http://localhost:8983/solr/testcollection/schema/fields > > > Note the fields listed. > > > > > > Simulate a schema change - add a field "test1"to the managed-schema: > > > > /var/solr_backup/testcollection_backup/zk_backup/configs/testcollection/m > anaged-schema > > > <field name="test1" type="string" indexed="true" stored="true" > > > /> > > > > > > Delete the collection and config set and restore it from the backup: > > > http://localhost:8983/solr/admin/collections?action=DELETE&name=test > > > collection > > > http://localhost:8983/solr/admin/configs?action=DELETE&name=testcoll > > > ection > > > http://localhost:8983/solr/admin/collections?action=RESTORE&location > > > =/var/solr_backup&name=testcollection_backup&collection=testcollecti > > > on > > > > > > Check the list of fields with the schema API again: > > > http://localhost:8983/solr/testcollection/schema/fields > > > The added test1 field is missing. > > > > > > The field is immediately visible in 8.6.3 In 8.8.2 a server restart is > necessary to see the new field. > > > Is this a known issue? > > > > > > Regards > > > Steffen Moldenhauer
