Yeah, I'm pretty lost in all the ways we index here. But if we are passing maxBufferedDocs <low number> for this deterministic indexing, I think it would cause the issue? I have no idea what the IW config here is...
On Thu, Oct 21, 2021 at 8:48 AM Robert Muir <[email protected]> wrote: > > On Thu, Oct 21, 2021 at 8:36 AM Robert Muir <[email protected]> wrote: > > > > But also the internal reuse of IndexingChain.PerField (which houses > > the reused tokenstream) isn't just per-thread, it is > > per-thread-per-segment, right? So if Mike is indexing with 100 > > threads, and flushes 200 times, I'd expect 20k of these things to be > > made. There's a lot going on in the benchmark code for nightly and it > > is tricky for me to try to navigate the various cases (1KB, > > 1KB-with-vectors, 4KB, "deterministic indexing", etc) > > I think this might be the case with your link. If you look at the URL > of your actual link, you see it ends with #profiler_4kb_indexing_1_cpu > ? > This makes me think i'm looking at the profiler output of the > "deterministic indexing". > For this one, LogDocMergePolicy is used. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
