On 2026-10-12 12:01 UTC Enlai Li wrote:
> Hi,
>
> Schema API add-field, add-dynamic-field, and add-copy-field currently
> persist the managed schema and reload every core using that configset.
> Reload calls newIndexWriter(), which closes the old IndexWriter and waits
> for running merges while holding the writer lock. Under continuous
> indexing this blocks updates until those merges finish. Vector fields make
> the merges expensive, so the stall can last for minutes.
>
> Schemaless mode already adds fields without a reload: persist the managed
> schema and install it with setLatestSchema(). Writes see the new field
> because the IndexWriter reads the core's latest schema. I would like the
> same for these three Schema API operations only. Delete and replace
> operations would still reload.
>
> One difference from schemaless: after installing the schema, open a new
> searcher without committing, so queries can use the new field immediately.
> The current searcher keeps the old schema.
>
> Any objection to this approach?
>
> Thanks,
> Li Enlai
>
> This message was drafted with AI assistance and reviewed by me.
>
Hi Li,
I am not an expert in this area, but wanted to chime in to give some feedback.
The idea is simpler to do in a single node setup, but if we are distributed
then an issue may be that a Schema API change propagates to other replicas via
a ZK watch on the configset, and that watch handler currently does an
unconditional full core.reload() whenever the schema version bumps — it doesn't
distinguish additive from destructive changes. So unless that path also gets
updated, this would only skip the reload on the node that received the API
call; every other replica in the collection would still reload and stall the
same way. Given the problem is specifically about continuous indexing across a
cluster, that seems like it'd need to be part of the change too.
Smaller thing: the IndexWriter's analyzer is re-fetched from the schema on
every lookup, so it naturally picks up new fields — but Similarity is captured
once at writer-construction time, not re-fetched. Probably fine for most
setups, but worth a second look if anyone's using custom per-field similarity.
Or something that might need to trigger the full .reload?
I like the idea of making Solr more robust and friendlier to changing schemas.
Lots of cool use cases where you are getting new fields added to your setup as
new data comes in!
Thanks,
Eric
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
