If you have sparse data, I would have expected index time to *decrease*, not increase.
Can you enable the IW info stream and share flush + merge times to see where indexing time goes? If you can run with a profiler, this might also give useful information. Le jeu. 18 janv. 2018 à 11:23, Rob Audenaerde <rob.audenae...@gmail.com> a écrit : > Hi all, > > We recently upgraded from Lucene 6.6 to 7.1. We see a significant drop in > indexing performace. > > We have a-typical use of Lucene, as we (also) index some database tables > and add all the values as AssociatedFacetFields as well. This allows us to > create pivot tables on search results really fast. > > These tables have some overlapping columns, but also disjoint ones. > > We anticipated a decrease in index size because of the sparse docvalues. We > see this happening, with decreases to ~50%-80% of the original index size. > But we did not expect an drop in indexing performance (client systems > indexing time increased with +50% to +250%). > > (Our indexing-speed used to be mainly bound by the speed the Taxonomy could > deliver new ordinals for new values, currently we are investigating if this > is still the case, will report later when a profiler run has been done) > > Does anyone know if this increase in indexing time is to be expected as > result of the sparse docvalues change? > > Kind regards, > > Rob Audenaerde >
