Re: Slower fetch document after upgrade >=8.7
I just opened https://issues.apache.org/jira/browse/LUCENE-9917. On Thu, Apr 8, 2021 at 4:34 PM Никита Михайлов wrote: > Thank you. Understood the policy > > > I have some changes to stored fields on my plate, I'll include this > change as well. > > Is there a ticket for this change? > > чт, 8 апр. 2021 г. в 15:30, Adrien Grand : > > > > Actually, we don't plan to have flexible settings even for advanced > > developers. Our stance on these discussions is that we should be > > opinionated about the default codec and not offer any options. Rather > than > > exposing advanced settings for advanced users, these advanced users can > > build their own codec and take care of backward compatibility themselves. > > > > On Thu, Apr 8, 2021 at 10:11 AM Никита Михайлов < > mihaylovniki...@gmail.com> > > wrote: > > > > > Thanks for the reply. > > > > > > The problem of understanding. You can make flexible settings for > > > advanced developers, leaving two facets by default. In tests, check > > > these facets > > > Never change them so that the developers themselves explicitly set the > > > settings. IMHO, I think this will help to avoid such problems > > > > > > OK. Have a ticket? > > > > > > чт, 8 апр. 2021 г. в 13:52, Adrien Grand : > > > > > > > > Thanks for the feedback. > > > > > > > > We don't want to offer too many choices, as it complicates backward > > > > compatibility testing, and want to stick to two options at most. > > > > > > > > Since this is the second time I'm seeing this feedback, I'm inclined > to > > > > reduce the block size for BEST_SPEED in order to trade a bit of > > > compression > > > > ratio for better decompression speed. I have some changes to stored > > > fields > > > > on my plate, I'll include this change as well. > > > > > > > > On Thu, Apr 8, 2021 at 7:04 AM Никита Михайлов < > > > mihaylovniki...@gmail.com> > > > > wrote: > > > > > > > > > Hi > > > > > BEST_SPEED has been changed in LUCENE-9447 and LUCENE-9486. For > this > > > > > reason, retrieving data from elasticsearch has slowed down by > 10-20%. > > > When > > > > > there is a lot of data, this is critical > > > > > Can developers leave the choice of which codec to use: LZ4(16kB) > (old > > > > > BEST_SPEED) or LZ4 with preset dict(BEST_SPEED_SAVING_DISKSIZE)? Or > > > make > > > > > more flexible settings? > > > > > > > > > > Otherwise, such changes may be a blocker or will have to spend > money on > > > > > buying new hardware > > > > > > > > > > > > > > > > > -- > > > > Adrien > > > > > > - > > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > > > > > -- > > Adrien > > - > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > -- Adrien
Re: Slower fetch document after upgrade >=8.7
Thank you. Understood the policy > I have some changes to stored fields on my plate, I'll include this change as > well. Is there a ticket for this change? чт, 8 апр. 2021 г. в 15:30, Adrien Grand : > > Actually, we don't plan to have flexible settings even for advanced > developers. Our stance on these discussions is that we should be > opinionated about the default codec and not offer any options. Rather than > exposing advanced settings for advanced users, these advanced users can > build their own codec and take care of backward compatibility themselves. > > On Thu, Apr 8, 2021 at 10:11 AM Никита Михайлов > wrote: > > > Thanks for the reply. > > > > The problem of understanding. You can make flexible settings for > > advanced developers, leaving two facets by default. In tests, check > > these facets > > Never change them so that the developers themselves explicitly set the > > settings. IMHO, I think this will help to avoid such problems > > > > OK. Have a ticket? > > > > чт, 8 апр. 2021 г. в 13:52, Adrien Grand : > > > > > > Thanks for the feedback. > > > > > > We don't want to offer too many choices, as it complicates backward > > > compatibility testing, and want to stick to two options at most. > > > > > > Since this is the second time I'm seeing this feedback, I'm inclined to > > > reduce the block size for BEST_SPEED in order to trade a bit of > > compression > > > ratio for better decompression speed. I have some changes to stored > > fields > > > on my plate, I'll include this change as well. > > > > > > On Thu, Apr 8, 2021 at 7:04 AM Никита Михайлов < > > mihaylovniki...@gmail.com> > > > wrote: > > > > > > > Hi > > > > BEST_SPEED has been changed in LUCENE-9447 and LUCENE-9486. For this > > > > reason, retrieving data from elasticsearch has slowed down by 10-20%. > > When > > > > there is a lot of data, this is critical > > > > Can developers leave the choice of which codec to use: LZ4(16kB) (old > > > > BEST_SPEED) or LZ4 with preset dict(BEST_SPEED_SAVING_DISKSIZE)? Or > > make > > > > more flexible settings? > > > > > > > > Otherwise, such changes may be a blocker or will have to spend money on > > > > buying new hardware > > > > > > > > > > > > > -- > > > Adrien > > > > - > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > -- > Adrien - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: Slower fetch document after upgrade >=8.7
Actually, we don't plan to have flexible settings even for advanced developers. Our stance on these discussions is that we should be opinionated about the default codec and not offer any options. Rather than exposing advanced settings for advanced users, these advanced users can build their own codec and take care of backward compatibility themselves. On Thu, Apr 8, 2021 at 10:11 AM Никита Михайлов wrote: > Thanks for the reply. > > The problem of understanding. You can make flexible settings for > advanced developers, leaving two facets by default. In tests, check > these facets > Never change them so that the developers themselves explicitly set the > settings. IMHO, I think this will help to avoid such problems > > OK. Have a ticket? > > чт, 8 апр. 2021 г. в 13:52, Adrien Grand : > > > > Thanks for the feedback. > > > > We don't want to offer too many choices, as it complicates backward > > compatibility testing, and want to stick to two options at most. > > > > Since this is the second time I'm seeing this feedback, I'm inclined to > > reduce the block size for BEST_SPEED in order to trade a bit of > compression > > ratio for better decompression speed. I have some changes to stored > fields > > on my plate, I'll include this change as well. > > > > On Thu, Apr 8, 2021 at 7:04 AM Никита Михайлов < > mihaylovniki...@gmail.com> > > wrote: > > > > > Hi > > > BEST_SPEED has been changed in LUCENE-9447 and LUCENE-9486. For this > > > reason, retrieving data from elasticsearch has slowed down by 10-20%. > When > > > there is a lot of data, this is critical > > > Can developers leave the choice of which codec to use: LZ4(16kB) (old > > > BEST_SPEED) or LZ4 with preset dict(BEST_SPEED_SAVING_DISKSIZE)? Or > make > > > more flexible settings? > > > > > > Otherwise, such changes may be a blocker or will have to spend money on > > > buying new hardware > > > > > > > > > -- > > Adrien > > - > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > -- Adrien
Re: Slower fetch document after upgrade >=8.7
Thanks for the reply. The problem of understanding. You can make flexible settings for advanced developers, leaving two facets by default. In tests, check these facets Never change them so that the developers themselves explicitly set the settings. IMHO, I think this will help to avoid such problems OK. Have a ticket? чт, 8 апр. 2021 г. в 13:52, Adrien Grand : > > Thanks for the feedback. > > We don't want to offer too many choices, as it complicates backward > compatibility testing, and want to stick to two options at most. > > Since this is the second time I'm seeing this feedback, I'm inclined to > reduce the block size for BEST_SPEED in order to trade a bit of compression > ratio for better decompression speed. I have some changes to stored fields > on my plate, I'll include this change as well. > > On Thu, Apr 8, 2021 at 7:04 AM Никита Михайлов > wrote: > > > Hi > > BEST_SPEED has been changed in LUCENE-9447 and LUCENE-9486. For this > > reason, retrieving data from elasticsearch has slowed down by 10-20%. When > > there is a lot of data, this is critical > > Can developers leave the choice of which codec to use: LZ4(16kB) (old > > BEST_SPEED) or LZ4 with preset dict(BEST_SPEED_SAVING_DISKSIZE)? Or make > > more flexible settings? > > > > Otherwise, such changes may be a blocker or will have to spend money on > > buying new hardware > > > > > -- > Adrien - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: Slower fetch document after upgrade >=8.7
Thanks for the feedback. We don't want to offer too many choices, as it complicates backward compatibility testing, and want to stick to two options at most. Since this is the second time I'm seeing this feedback, I'm inclined to reduce the block size for BEST_SPEED in order to trade a bit of compression ratio for better decompression speed. I have some changes to stored fields on my plate, I'll include this change as well. On Thu, Apr 8, 2021 at 7:04 AM Никита Михайлов wrote: > Hi > BEST_SPEED has been changed in LUCENE-9447 and LUCENE-9486. For this > reason, retrieving data from elasticsearch has slowed down by 10-20%. When > there is a lot of data, this is critical > Can developers leave the choice of which codec to use: LZ4(16kB) (old > BEST_SPEED) or LZ4 with preset dict(BEST_SPEED_SAVING_DISKSIZE)? Or make > more flexible settings? > > Otherwise, such changes may be a blocker or will have to spend money on > buying new hardware > -- Adrien
Slower fetch document after upgrade >=8.7
Hi BEST_SPEED has been changed in LUCENE-9447 and LUCENE-9486. For this reason, retrieving data from elasticsearch has slowed down by 10-20%. When there is a lot of data, this is critical Can developers leave the choice of which codec to use: LZ4(16kB) (old BEST_SPEED) or LZ4 with preset dict(BEST_SPEED_SAVING_DISKSIZE)? Or make more flexible settings? Otherwise, such changes may be a blocker or will have to spend money on buying new hardware