Slower fetch document after upgrade >=8.7

2021-04-07 Thread Никита Михайлов
Hi BEST_SPEED has been changed in LUCENE-9447 and LUCENE-9486. For this reason, retrieving data from elasticsearch has slowed down by 10-20%. When there is a lot of data, this is critical Can developers leave the choice of which codec to use: LZ4(16kB) (old BEST_SPEED) or LZ4 with preset dict(BEST_

Slower search after 8.5.x to >=8.6

2021-04-07 Thread Никита Михайлов
Hi. I noticed that after the upgrade from Lucene8.5.x to Lucene >=8.6, search became slower(example TopScoreDocCollector became 20-30% slower, from ElasticSearch - 50%). While testing, I realized that it happened after LUCENE-9257(commit e7a61ea). Bug or feature? Can add settings for isOffHeep? T

Re: Slower search after 8.5.x to >=8.6

2021-04-07 Thread Adrien Grand
Hello, Why are you forcing NIOFSDirectory instead of using Lucene's defaults via FSDirectory#open? I wonder if this might contribute to the slowdown you are seeing given that access to the terms index tends to be a bit random. It's very unlikely we'll add back a toggle for this as there is no poi

Re: Slower fetch document after upgrade >=8.7

2021-04-07 Thread Adrien Grand
Thanks for the feedback. We don't want to offer too many choices, as it complicates backward compatibility testing, and want to stick to two options at most. Since this is the second time I'm seeing this feedback, I'm inclined to reduce the block size for BEST_SPEED in order to trade a bit of com