Ah, I was wondering why I hit "trusted config-set" warnings that you didn't. Makes sense.
Thanks for filing a ticket - very curious whether the "different-configname" workaround helps out! Jason On Tue, Jun 15, 2021 at 8:38 AM Steffen Moldenhauer <[email protected]> wrote: > > Hi Jason, > > thanks for having a look at this and your confirmation that you can reproduce > the issue. > Sorry, I forgot to mention that I started the server with SOLR_OPTS set to > -Dsolr.disableConfigSetsCreateAuthChecks=true > -Dsolr.allowPaths=/var/solr_backup > > I created https://issues.apache.org/jira/browse/SOLR-15478 for it. I attached > your steps as a comment. > > Yes, we tried to use RELOAD the collection, but it did not help to make the > schema changes visible. > I'll try RESTORE with collection.configName > > Kind Regards > Steffen > > > -----Original Message----- > > From: Jason Gerlowski <[email protected]> > > Sent: Freitag, 11. Juni 2021 16:00 > > To: [email protected] > > Subject: Re: Schema Changes are not Visible after Collection Restore - Solr > > 8.8.2 > > > > Hey Steffen, > > > > I can reproduce the issue on 8.8.2 using the steps you laid out in your > > original email. (With one deviation - I had to enable auth to get around an > > error about "Can't create a configset with an unauthenticated request from > > a trusted baseConfigSet") > > > > The steps you're using to trigger this are a bit odd. Do you really edit > > the > > configset stored with the backup as a part of your production workflow, or > > is > > this just a step you introduced to simplify the reproduction? The /schema > > API is the recommended way to handle configuration changes, followed by > > editing the configset files and uploading them to ZooKeeper. None of that > > quirkiness makes this any less of a bug though - it seems very much like > > something's going wrong here. I can't imagine any reason why Solr would > > intentionally not pick up fields until a restart. > > > > To further the reproduction a bit, it's possible to reproduce this without > > backup/restore in the picture at all. The key piece seems to be that the > > new > > (post-schema-edit) collection and config set have the same name as their > > now-deleted counterparts. > > > > 1. Start Solr: bin/solr start -c -p 7574 2. Turn on auth (this was > > necessary for > > me to create a configset that extended "_default"): bin/solr auth enable - > > type basicAuth -credentials solr:solrRocks -z localhost:8574 3. Create a > > config > > based on "_default": curl -ilk -X GET -u solr:solrRocks > > "http://localhost:7574/solr/admin/configs?action=CREATE&name=testconfig > > set&baseConfigSet=_default" > > 4. Create a collection using this configset: curl -ilk -X GET -u > > solr:solrRocks > > "http://localhost:7574/solr/admin/collections?action=CREATE&name=coll1&; > > collection.configName=testconfigset&numShards=1" > > 5. Download the configset locally: bin/solr zk downconfig -d > > downloaded_test_config -n testconfig2 -z localhost:8574 6. Drop a new > > <field> definition into downloaded_test_config/conf/managed-schema > > 7. Delete the existing collection: curl -ilk -X GET -u solr:solrRocks > > "http://localhost:7574/solr/admin/collections?action=DELETE&name=coll1"; > > 8. Delete the configset from ZK: curl -ilk -X GET -u solr:solrRocks > > "http://localhost:7574/solr/admin/configs?action=DELETE&name=testconfigs > > et" > > 9. Create the config from the local copy: bin/solr zk upconfig -d > > downloaded_test_config -n testconfigset -z localhost:8574 10. Create a new > > collection with the same name as the previous > > collection: curl -ilk -X GET -u solr:solrRocks > > "http://localhost:7574/solr/admin/collections?action=CREATE&name=coll1&; > > collection.configName=testconfigset&numShards=1" > > 11. List the fields and show that the added field is missing: curl -ilk -X > > GET -u > > solr:solrRocks "http://localhost:7574/solr/coll1/schema/fields"; > > > > My best guess is that Solr is caching the configset (or the schemas parsed > > out > > of them) somewhere that sticks around even after the corresponding > > configset/collection is deleted. But I'm not very familiar with our > > configset > > code, so I can't dig in further than that with the time I have. Hopefully > > someone else can pick it up from here. In the meantime, you might be able > > to workaround this by specifying a "collection.configName" param on your > > RESTORE call - this will cause the backed up configset to be uploaded to ZK > > under a different name - hopefully avoiding whatever caching is causing the > > trouble here. > > > > Best, > > > > Jason > > > > On Fri, Jun 11, 2021 at 8:38 AM Jason Gerlowski <[email protected]> > > wrote: > > > > > > Hey Steffen, > > > > > > I took a quick look at the backup/restore codepath involved here - > > > surprisingly the restore code itself hasn't changed between 8.6.3 and > > > 8.8.2. In both 8.6.3 and 8.8.2, if the configset mentioned in the > > > backup has the same name as a config currently in ZooKeeper, the > > > version in ZK is used. So fields added to the schema after the backup > > > _should_ be visible. Obviously per your report this isn't happening > > > though. To me that says that there must be some change of behavior in > > > collection-creation or in schema-reading. > > > > > > I'm hoping to do a bit more digging to get to the bottom of this. In > > > the meantime, would you mind filing a JIRA ticket regarding this > > > behavior (if you haven't already)? > > > > > > One last question: you mentioned above that the field becomes visible > > > after a restart - does a reload > > > (/admin/collections?action=RELOAD&name=testcollection) have the same > > > effect? If so that may be a more palatable work-around for you in the > > > short-term... > > > > > > Best, > > > > > > Jason > > > > > > On Thu, Jun 3, 2021 at 9:32 AM Steffen Moldenhauer > > > <[email protected]> wrote: > > > > > > > > we tried to upgrade from 8.6.3 to 8.8 but had trouble with collection > > restore and schema changes. > > > > If there were changes to the schema - they are not visible in the Schema > > API. > > > > We are using the backup/restore to transfer pre-built collections to > > > > production systems and this issue prevents us from using the latest > > > > version 8.8.2 > > > > > > > > These are the basic steps to reproduce the issue on a "solr start -c" : > > > > > > > > For preparation create a config set, collection and backup: > > > > http://localhost:8983/solr/admin/configs?action=CREATE&name=testcoll > > > > ection&baseConfigSet=_default > > > > http://localhost:8983/solr/admin/collections?action=CREATE&collectio > > > > n.configName=testcollection&name=testcollection&numShards=1 > > > > http://localhost:8983/solr/admin/collections?action=BACKUP&location= > > > > /var/solr_backup&name=testcollection_backup&collection=testcollectio > > > > n > > > > > > > > Check the list of fields with the schema API: > > > > http://localhost:8983/solr/testcollection/schema/fields > > > > Note the fields listed. > > > > > > > > Simulate a schema change - add a field "test1"to the managed-schema: > > > > > > /var/solr_backup/testcollection_backup/zk_backup/configs/testcollection/m > > anaged-schema > > > > <field name="test1" type="string" indexed="true" stored="true" > > > > /> > > > > > > > > Delete the collection and config set and restore it from the backup: > > > > http://localhost:8983/solr/admin/collections?action=DELETE&name=test > > > > collection > > > > http://localhost:8983/solr/admin/configs?action=DELETE&name=testcoll > > > > ection > > > > http://localhost:8983/solr/admin/collections?action=RESTORE&location > > > > =/var/solr_backup&name=testcollection_backup&collection=testcollecti > > > > on > > > > > > > > Check the list of fields with the schema API again: > > > > http://localhost:8983/solr/testcollection/schema/fields > > > > The added test1 field is missing. > > > > > > > > The field is immediately visible in 8.6.3 In 8.8.2 a server restart is > > necessary to see the new field. > > > > Is this a known issue? > > > > > > > > Regards > > > > Steffen Moldenhauer
