[
https://issues.apache.org/jira/browse/LUCENE-5914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117759#comment-14117759
]
Robert Muir commented on LUCENE-5914:
-------------------------------------
{quote}
This doesn't mean though not to provide the option to users if a case for it
can be made
{quote}
Its ok to provide such options without hesitation in the codecs/ module,
however:
We have to be careful, this issue proposes supporting such options *in the
default codec*.
This is a completely different thing. This means we support such formats for
years and years and years. Currently as we speak we are trying to release 4.10,
and it must still be able to read the 3.0 index format from 5 years ago (and we
had to respin another release candidate because there were bugs in such
support).
So that's why i point out, if we want to add an option (and i expect we can add
at most 1 option here feasibly), then tradeoffs have to be made in our
backwards compatibility support such that we can maintain this stuff and it
does not spin out of control.
> More options for stored fields compression
> ------------------------------------------
>
> Key: LUCENE-5914
> URL: https://issues.apache.org/jira/browse/LUCENE-5914
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Adrien Grand
> Assignee: Adrien Grand
> Fix For: 4.11
>
> Attachments: LUCENE-5914.patch
>
>
> Since we added codec-level compression in Lucene 4.1 I think I got about the
> same amount of users complaining that compression was too aggressive and that
> compression was too light.
> I think it is due to the fact that we have users that are doing very
> different things with Lucene. For example if you have a small index that fits
> in the filesystem cache (or is close to), then you might never pay for actual
> disk seeks and in such a case the fact that the current stored fields format
> needs to over-decompress data can sensibly slow search down on cheap queries.
> On the other hand, it is more and more common to use Lucene for things like
> log analytics, and in that case you have huge amounts of data for which you
> don't care much about stored fields performance. However it is very
> frustrating to notice that the data that you store takes several times less
> space when you gzip it compared to your index although Lucene claims to
> compress stored fields.
> For that reason, I think it would be nice to have some kind of options that
> would allow to trade speed for compression in the default codec.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
