[jira] [Commented] (LUCENE-4792) Smaller doc maps
[ https://issues.apache.org/jira/browse/LUCENE-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585490#comment-13585490 ] Adrien Grand commented on LUCENE-4792: -- In case someone would like to use this class, I'd add that: - the encoded sequence does not strictly need to be monotonic: it can encode any sequence of values but it compresses best when the stream contains monotonic sub-sequences of 1024 longs at least (for example it would have a good compression ratio if there are first 1 increasing values and then 5000 decreasing values), - it can address up to 2^42 values, - there are writer/reader equivalents called MonotonicBlockPackedWriter and MonotonicBlockPackedReader (which can either load values in memory or read from disk). > Smaller doc maps > > > Key: LUCENE-4792 > URL: https://issues.apache.org/jira/browse/LUCENE-4792 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Assignee: Adrien Grand >Priority: Minor > Fix For: 4.2 > > Attachments: LUCENE-4792.patch > > > MergeState.DocMap could leverage MonotonicAppendingLongBuffer to save memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4792) Smaller doc maps
[ https://issues.apache.org/jira/browse/LUCENE-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585474#comment-13585474 ] Robert Muir commented on LUCENE-4792: - We are using the same compression for (as far as i know): * stored fields, term vectors, docvalues "disk" addresses * multidocvalues ordinal maps We could consider trying it out for fieldcache and other places for example, im not sure what the perf hit would be. (I'm not very interested in optimizing fieldcache myself) > Smaller doc maps > > > Key: LUCENE-4792 > URL: https://issues.apache.org/jira/browse/LUCENE-4792 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Assignee: Adrien Grand >Priority: Minor > Fix For: 4.2 > > Attachments: LUCENE-4792.patch > > > MergeState.DocMap could leverage MonotonicAppendingLongBuffer to save memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4792) Smaller doc maps
[ https://issues.apache.org/jira/browse/LUCENE-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585471#comment-13585471 ] Michael McCandless commented on LUCENE-4792: These RAM savings are AWESOME! Where else can we use MonotonicAppendingLongBuffer! > Smaller doc maps > > > Key: LUCENE-4792 > URL: https://issues.apache.org/jira/browse/LUCENE-4792 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Assignee: Adrien Grand >Priority: Minor > Fix For: 4.2 > > Attachments: LUCENE-4792.patch > > > MergeState.DocMap could leverage MonotonicAppendingLongBuffer to save memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4792) Smaller doc maps
[ https://issues.apache.org/jira/browse/LUCENE-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583720#comment-13583720 ] Commit Tag Bot commented on LUCENE-4792: [branch_4x commit] Adrien Grand http://svn.apache.org/viewvc?view=revision&revision=1448861 LUCENE-4792: Reduction of the memory required to build the doc ID maps used when merging segments (merged from r1448853). > Smaller doc maps > > > Key: LUCENE-4792 > URL: https://issues.apache.org/jira/browse/LUCENE-4792 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Assignee: Adrien Grand >Priority: Minor > Fix For: 4.2 > > Attachments: LUCENE-4792.patch > > > MergeState.DocMap could leverage MonotonicAppendingLongBuffer to save memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4792) Smaller doc maps
[ https://issues.apache.org/jira/browse/LUCENE-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583703#comment-13583703 ] Commit Tag Bot commented on LUCENE-4792: [trunk commit] Adrien Grand http://svn.apache.org/viewvc?view=revision&revision=1448853 LUCENE-4792: Reduction of the memory required to build the doc ID maps used when merging segments. > Smaller doc maps > > > Key: LUCENE-4792 > URL: https://issues.apache.org/jira/browse/LUCENE-4792 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Assignee: Adrien Grand >Priority: Minor > Fix For: 4.2 > > Attachments: LUCENE-4792.patch > > > MergeState.DocMap could leverage MonotonicAppendingLongBuffer to save memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4792) Smaller doc maps
[ https://issues.apache.org/jira/browse/LUCENE-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13583609#comment-13583609 ] Robert Muir commented on LUCENE-4792: - +1, this is great. > Smaller doc maps > > > Key: LUCENE-4792 > URL: https://issues.apache.org/jira/browse/LUCENE-4792 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Adrien Grand >Assignee: Adrien Grand >Priority: Minor > Fix For: 4.2 > > Attachments: LUCENE-4792.patch > > > MergeState.DocMap could leverage MonotonicAppendingLongBuffer to save memory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
