> For such older segments, after replication, are they at the same exact file path or does replication create a new path?
Replication only downloads the delta. If it already has the files, they are not downloaded On Wed, Nov 30, 2022 at 4:41 AM David Smiley <dsmi...@apache.org> wrote: > It could be interesting to explore optimizing "openNewSearcher" when there > is an existing searcher open over some of the same segments. > For such older segments, after replication, are they at the same exact file > path or does replication create a new path? I forget this detail; I recall > seeing some index data path shuffling going on. ReplicationHandler etc. is > overly complex. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Wed, Nov 23, 2022 at 1:50 PM Patson Luk <pat...@fullstory.com> wrote: > > > Hi all! > > > > We are testing multiple replica setup here (1 NRT + 1 PULL) and noticed > > that CPU consumption for replication is unreasonably high. Profiling > shows > > that `SolrCore#openNewSearcher` triggered from `IndexFetcher` takes much > > more CPU time than the same method triggered from regular commits. > > > > Debugging shows that when `SolrCore#openNewSearcher` is triggered from > > `IndexFetcher`, it opens a new `SegmentReader` for every single fragment > > for the updated collection. As a new `IndexWriter`, which keeps a > > `ReaderPool`, is instantiated for each replication. And such pool is not > > reused nor previous segment readers are carried over. > > > > Details in this ticket https://issues.apache.org/jira/browse/SOLR-16560. > > > > Since I'm pretty new to this area, I would love to get some thoughts from > > the community! > > > > Many thanks! > > Patsn > > > -- ----------------------------------------------------- Noble Paul
