Re: Lucene DocValuesField, SortedDocValuesField usage for filtering and sorting

2014-12-16 Thread Adrien Grand
Hi Piotr, On Mon, Dec 15, 2014 at 9:43 PM, Piotr Idzikowski piotridzikow...@gmail.com wrote: Hello. I am going to switch to newest (4.10.2) version of Lucene and I'd like to make some optimization in my index and code. I would like to use DocValuesField to get values but also for filtering

Re: Lucene DocValuesField, SortedDocValuesField usage for filtering and sorting

2014-12-16 Thread Piotr Idzikowski
Hello. Thanks for your replay. On Tue, Dec 16, 2014 at 3:14 PM, Adrien Grand jpou...@gmail.com wrote: Hi Piotr, On Mon, Dec 15, 2014 at 9:43 PM, Piotr Idzikowski piotridzikow...@gmail.com wrote: Hello. I am going to switch to newest (4.10.2) version of Lucene and I'd like to make some

Re: Lucene DocValuesField, SortedDocValuesField usage for filtering and sorting

2014-12-16 Thread Piotr Idzikowski
So for instance if I store documents with ie creation date and I have a data (millions of documents) from last let's say 3 years and I'd like to do range filter to get socs from some month only is it better to use ordinary numeric query instead of FieldCacheRangeQuery? Of course I meant

including self-joins in parent/child queries

2014-12-16 Thread Michael Sokolov
I see in the docs of ToParentBlockJoinQuery that: * The child documents must be orthogonal to the parent * documents: the wrapped child query must never * return a parent document. First, it would be helpful if the docs explained what would happen if that assumption were violated. Second,

Re: including self-joins in parent/child queries

2014-12-16 Thread Michael Sokolov
OK - I see looking at the code that an exception is thrown if a parent doc matches the subquery -- so that explains what will happen, but I guess my further question is -- is that necessary? Could we just not throw an exception there? -Mike On 12/16/2014 10:38 AM, Michael Sokolov wrote: I

Re: Question about JoinUtil

2014-12-16 Thread Glen Newton
Anyone? On Thu, Dec 11, 2014 at 2:53 PM, Glen Newton glen.new...@gmail.com wrote: Is there any reason JoinUtil (below) does not have a 'Query toQuery' available? I was wanting to filter on the 'to' side as well. I feel I am missing something here. To make sure this is not an XY problem, here

MMapdirectory Configuration

2014-12-16 Thread Vijay B
Finally we are seeing great improvement once we switch to 64-bit java and MMapDirectory. Our Test run (multiple requests) used to take 26 minutes on 32-bit and is now improved to 10 minutes on 64-bit java. We load stored documents from lucene and pass the documents to a third party libray (closed

Re: Lucene DocValuesField, SortedDocValuesField usage for filtering and sorting

2014-12-16 Thread Adrien Grand
On Tue, Dec 16, 2014 at 3:25 PM, Piotr Idzikowski piotridzikow...@gmail.com wrote: So for instance if I store documents with ie creation date and I have a data (millions of documents) from last let's say 3 years and I'd like to do range filter to get socs from some month only is it better to

ANNOUNCE: CFP and Travel Assistance now open for ApacheCon North America 2015

2014-12-16 Thread Chris Hostetter
(NOTE: cross posted to several lucene lists, if you have replies, please confine them to general@lucene) -- Forwarded message -- In case you've missed it: - ApacheCon North America returns to Austin, Texas, 13-17 April 2015 http://apachecon.com/ - Call for Papers open

Re: Question about JoinUtil

2014-12-16 Thread Gregory Dearing
Glen, Lucene isn't relational at heart and may not be the right tool for what you're trying to accomplish. Note that JoinQuery doesn't join 'left' and 'right' answers; rather it transforms a 'left' answerset into a 'right' answerset. JoinQuery is able to perform this transformation with a single

Re: including self-joins in parent/child queries

2014-12-16 Thread Gregory Dearing
Michael, Note that the index doesn't contain any special information about block-join relationships... it uses a convention that child docs are indexed before parent docs (ie. the root doc in each hierarchy has the largest docId in its block). This means that it can 'join' to parents just by

Re: including self-joins in parent/child queries

2014-12-16 Thread Michael Sokolov
Looking at the code, there are explicit checks for if (childId == parentId) throw an exception ... It seems to me that instead, the logic *could* be if (childId == parentId) then --- accumulate the parentId as if it were a child *and* terminate the block. In your phraseology, we could