Re: SortingAtomicReader alternate to Tim-Sort...

2015-05-06 Thread Adrien Grand
Sorry for the delay, I opened https://issues.apache.org/jira/browse/LUCENE-6469. It can go to trunk and 5.x (the value of x depending on when it's ready :)). On Thu, Apr 30, 2015 at 9:02 AM, Ravikumar Govindarajan ravikumar.govindara...@gmail.com wrote: Would you like to submit a patch that

Re: SortingAtomicReader alternate to Tim-Sort...

2015-04-30 Thread Ravikumar Govindarajan
Would you like to submit a patch that changes SortingMergePolicy to use the approach that you are proposing using bitsets instead of sorting int[] arrays? Sure can do that. Can you open a ticket for this, as I don't know what versions this can go in? -- Ravi On Tue, Apr 28, 2015 at 6:03

Re: SortingAtomicReader alternate to Tim-Sort...

2015-04-28 Thread Adrien Grand
On Tue, Apr 21, 2015 at 10:00 AM, Ravikumar Govindarajan ravikumar.govindara...@gmail.com wrote: Thanks for the comments… My only concern about using the FixedBitSet is that it would make sorting each postings list run in O(maxDoc) but maybe we can make it better by using SparseFixedBitSet

Re: SortingAtomicReader alternate to Tim-Sort...

2015-04-24 Thread Ravikumar Govindarajan
Thanks. Glad that it has been pro-actively identified and fixed -- Ravi On Thu, Apr 23, 2015 at 10:34 AM, Robert Muir rcm...@gmail.com wrote: On Tue, Apr 21, 2015 at 4:00 AM, Ravikumar Govindarajan ravikumar.govindara...@gmail.com wrote: b) CompressingStoredFieldsReader did not store the

Re: SortingAtomicReader alternate to Tim-Sort...

2015-04-22 Thread Robert Muir
On Tue, Apr 21, 2015 at 4:00 AM, Ravikumar Govindarajan ravikumar.govindara...@gmail.com wrote: b) CompressingStoredFieldsReader did not store the last decoded 32KB chunk. Our segments are already sorted before participating in a merge. On mostly linear merge, we ended up decoding the same

Re: SortingAtomicReader alternate to Tim-Sort...

2015-04-21 Thread Ravikumar Govindarajan
Thanks for the comments… My only concern about using the FixedBitSet is that it would make sorting each postings list run in O(maxDoc) but maybe we can make it better by using SparseFixedBitSet Yes I was also thinking about this. But we are on 4.x and did not take the plunge. But as you

Re: SortingAtomicReader alternate to Tim-Sort...

2015-04-20 Thread Adrien Grand
I like these ideas, the int[] we are using today are wasteful. My only concern about using the FixedBitSet is that it would make sorting each postings list run in O(maxDoc) but maybe we can make it better by using SparseFixedBitSet (added in 5.0, given your code snippets I assume you are still on

SortingAtomicReader alternate to Tim-Sort...

2015-04-14 Thread Ravikumar Govindarajan
We were experimenting with SortingMergePolicy and came across an alternate solution to TimSort of postings-list using FBS GrowableWriter. I have attached relevant code-snippet. It would be nice if someone can clarify whether it is a good idea to implement... public class SortingAtomicReader { …