Re: Is there a way to implement a IntRangeField in Solr?
First, I really don't understand why you would have OOMs when indexing even a humongous number of dates, that just seems weird. But what happens if you think about it the other way? Instead of indexing open dates, index booked dates. Then construct filter queries like fq=-booked:[5 TO 23], where the range is the proposed reservation date. You'll have to do something creative with the numbers, possibly the number of days since the epoch so you can cross years etc.. You can also prune the booked dates in the past to keep the docs smaller... Best Erick On Wed, Feb 29, 2012 at 2:23 PM, Mikhail Khludnev wrote: > AFAIK join is done in the single core. Same core should have two types of > documents. > Pls let me know about your achievement. > > On Wed, Feb 29, 2012 at 8:46 PM, federico.wachs > wrote: > >> I'll give this a try. I'm not sure I completely understand how to do that >> because I don't have so much experience with Solr. Do I have to use another >> core to post a different kind of document and then join it? >> >> Thanks! >> >> -- >> View this message in context: >> http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3787873.html >> Sent from the Solr - User mailing list archive at Nabble.com. >> > > > > -- > Sincerely yours > Mikhail Khludnev > Lucid Certified > Apache Lucene/Solr Developer > Grid Dynamics > > <http://www.griddynamics.com> >
Re: Is there a way to implement a IntRangeField in Solr?
AFAIK join is done in the single core. Same core should have two types of documents. Pls let me know about your achievement. On Wed, Feb 29, 2012 at 8:46 PM, federico.wachs wrote: > I'll give this a try. I'm not sure I completely understand how to do that > because I don't have so much experience with Solr. Do I have to use another > core to post a different kind of document and then join it? > > Thanks! > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3787873.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Sincerely yours Mikhail Khludnev Lucid Certified Apache Lucene/Solr Developer Grid Dynamics <http://www.griddynamics.com>
Re: Is there a way to implement a IntRangeField in Solr?
I'll give this a try. I'm not sure I completely understand how to do that because I don't have so much experience with Solr. Do I have to use another core to post a different kind of document and then join it? Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3787873.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
Hold on. It's possible to find leases, at the particular date and collapse
them to appartments. But it looks impossible to negate those busy
apartments. Or I don't know how.
Let's try with http://wiki.apache.org/solr/Join
If you have lease documents with "FK" field LEASE_APT_FK *and* appartment
documents with "PK" APT_PK.
you can search leases for the given date (overlappingleases= below), then
join leases to apartments (&busyapts=) and negate those apartments (}&fq=).
overlappingleases=(from:[$need_from TO $need_to] ) OR (to:[$need_from TO
$need_to]) OR ((from:[* TO $need_from]) AND (to:[$need_from TO
*]))&busyapts={!join from=LEASE_APT_FK to=APT_PK
v=$overlappingleases}&fq=NOT _query_:{!v=$busyapts}&q=*:*
FWIW i use some magic from http://wiki.apache.org/solr/LocalParams and
http://wiki.apache.org/solr/SolrQuerySyntax
On Tue, Feb 28, 2012 at 11:11 PM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:
>
>
> On Tue, Feb 28, 2012 at 6:44 PM, federico.wachs > wrote:
>
>> Hi Mikhail, thanks for your concern and reply.
>>
>> I've read a few dozen times your reply and I think I get what you mean,
>> but
>> I'm not exactly sure how to go forward with your approach. You are saying
>> that I should be able to have nested documents, but I haven't been able to
>> submit a Document with another Document on it so far.
>>
>> I'm using SolrJ to integrate with my Solr servers, do you think you could
>> guide me a bit on how you would accomplish to nest two different kinds of
>> documents?
>>
>
> Ok start from Lease documents with ApartmentID and than query leases and
> specify &group.field=ApartmentID&group=true
>
> Disclaimer: I've never done grouping and I am not really familiar with
> SolrJ,
>
>
>>
>> Thank you for your time and explanation I really appreciate it!
>>
>> Regards,
>> Federicp
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3784220.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Lucid Certified
> Apache Lucene/Solr Developer
> Grid Dynamics
>
> <http://www.griddynamics.com>
>
>
>
--
Sincerely yours
Mikhail Khludnev
Lucid Certified
Apache Lucene/Solr Developer
Grid Dynamics
<http://www.griddynamics.com>
Re: Is there a way to implement a IntRangeField in Solr?
On Tue, Feb 28, 2012 at 6:44 PM, federico.wachs wrote: > Hi Mikhail, thanks for your concern and reply. > > I've read a few dozen times your reply and I think I get what you mean, but > I'm not exactly sure how to go forward with your approach. You are saying > that I should be able to have nested documents, but I haven't been able to > submit a Document with another Document on it so far. > > I'm using SolrJ to integrate with my Solr servers, do you think you could > guide me a bit on how you would accomplish to nest two different kinds of > documents? > Ok start from Lease documents with ApartmentID and than query leases and specify &group.field=ApartmentID&group=true Disclaimer: I've never done grouping and I am not really familiar with SolrJ, > > Thank you for your time and explanation I really appreciate it! > > Regards, > Federicp > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3784220.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Sincerely yours Mikhail Khludnev Lucid Certified Apache Lucene/Solr Developer Grid Dynamics <http://www.griddynamics.com>
Re: Is there a way to implement a IntRangeField in Solr?
Hi Mikhail, thanks for your concern and reply. I've read a few dozen times your reply and I think I get what you mean, but I'm not exactly sure how to go forward with your approach. You are saying that I should be able to have nested documents, but I haven't been able to submit a Document with another Document on it so far. I'm using SolrJ to integrate with my Solr servers, do you think you could guide me a bit on how you would accomplish to nest two different kinds of documents? Thank you for your time and explanation I really appreciate it! Regards, Federicp -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3784220.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
Hello, For me it looks like typical nested documents use-case. You have an apartment document with nested lease documents. Where lease has from and to fields. So, you need to find apartments which has no lease in the range provided $need_from $need_to conflicted leases can be found by (from:[$need_from TO $need_to] ) OR (to:[$need_from TO $need_to]) OR ((from:[* TO $need_from]) AND (to:[$need_from TO *]) ) then you need to get apartment which has no such leases. AFAIK it can be done by Joins and Grouping in Solr. Some time ago I did by SpanQueries <http://blog.griddynamics.com/search/label/Solr>, but now I'm collaborating <https://issues.apache.org/jira/browse/SOLR-3076> around rocket science<http://blog.mikemccandless.com/2012/01/searching-relational-content-with.html> . Also I have an idea about OOM. if you rollover range by the app forming SolrInputDocuments it can cause the problem. To solve it you can pass two numbers as a range but provide an own analyzer via schema which will expand them into stream of numbers. Regards On Tue, Feb 28, 2012 at 2:11 AM, federico.wachs wrote: > This is used on an apartment booking system, and what I store as solr > documents can be seen as apartments. These apartments can be booked for a > certain amount of days with a check in and a check out date hence the > ranges > I was speaking of before. > > What I want to do is to filter off the apartments that are booked so my > users won't have a bad user experience while trying to book an apartment > that suits their needs. > > Did I make any sense? Please let me know, otherwise I can explain > furthermore. > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782304.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Sincerely yours Mikhail Khludnev Lucid Certified Apache Lucene/Solr Developer Grid Dynamics <http://www.griddynamics.com>
Re: Is there a way to implement a IntRangeField in Solr?
Exactly, I'm using a tint field type and works really well. The only problem is when I have a set of very wide ranges and make Solr make fireworks out of the blue. Thank you a lot Michael, I appreciate your help on this one :) -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782415.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
I don't know if this would help with OOM conditions, but are you using a tint type field for this? That should be more efficient to search than a regular int or string. -Mike On 02/27/2012 05:27 PM, federico.wachs wrote: Yeah that's what I'm doing right now. But whenever I try to index an apartment that has many wide ranges, my master solr server throws OutOfMemoryError ( I have set max heap to 1024m). So I thought this could be a good workaround but puf it is a lot harder than it seems! -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782347.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
Yeah that's what I'm doing right now. But whenever I try to index an apartment that has many wide ranges, my master solr server throws OutOfMemoryError ( I have set max heap to 1024m). So I thought this could be a good workaround but puf it is a lot harder than it seems! -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782347.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
Yes, I see - I think your best bet is to index every day as a distinct value. Don't worry about having 100's of values. -Mike On 02/27/2012 05:11 PM, federico.wachs wrote: This is used on an apartment booking system, and what I store as solr documents can be seen as apartments. These apartments can be booked for a certain amount of days with a check in and a check out date hence the ranges I was speaking of before. What I want to do is to filter off the apartments that are booked so my users won't have a bad user experience while trying to book an apartment that suits their needs. Did I make any sense? Please let me know, otherwise I can explain furthermore. -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782304.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
This is used on an apartment booking system, and what I store as solr documents can be seen as apartments. These apartments can be booked for a certain amount of days with a check in and a check out date hence the ranges I was speaking of before. What I want to do is to filter off the apartments that are booked so my users won't have a bad user experience while trying to book an apartment that suits their needs. Did I make any sense? Please let me know, otherwise I can explain furthermore. -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782304.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
No; contiguous means there are no gaps between them. You need something like what you described initially. Another approach is to de-normalize your data so that you have a single document for every range. But this might or might not suit your application. You haven't said anything about the context in which this is to be used. -Mike On 02/27/2012 04:43 PM, federico.wachs wrote: Oh No, I think I understood wrong when you said that my ranges where contiguous. I could have ranges like this: 1 TO 15 5 TO 30 50 TO 60 And so on... I'm not sure that what you supposed would work, right? -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782202.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
Oh No, I think I understood wrong when you said that my ranges where contiguous. I could have ranges like this: 1 TO 15 5 TO 30 50 TO 60 And so on... I'm not sure that what you supposed would work, right? -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782202.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
I think your example case would end up like this: ... 1 -- single-valued range field 15 ... On 02/27/2012 04:26 PM, federico.wachs wrote: Michael thanks a lot for your quick answer, but i'm not exactly sure I understand your solution. How would the docuemnt you are proposing would look like? Do you mind showing me a simple xml as example? Again, thank you for your cooperation. And yes, the ranges are contiguous! -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782139.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
Michael thanks a lot for your quick answer, but i'm not exactly sure I understand your solution. How would the docuemnt you are proposing would look like? Do you mind showing me a simple xml as example? Again, thank you for your cooperation. And yes, the ranges are contiguous! -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782139.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Is there a way to implement a IntRangeField in Solr?
If your ranges are always contiguous, you could index two fields: range-start and range-end and then perform queries like: range-start:[* TO 30] AND range-end:[5 TO *] If you have multiple ranges which could have gaps in between then you need something more complicated :) On 02/27/2012 04:09 PM, federico.wachs wrote: Hi all ! Here's my dreadful case, thank you for helping out! I want to have a document like this: ... -- multivalued range field 1 TO 10 5 TO 15 ... And the reason why I want to do this is because it's so much lighter than having all the numbers in there, of course. Just to be clear, I want to avoid having this in solr: ... -- multivalued range field 1 2 3 4 5 6 7 8 9 10 ... And then perform range queries on this range field like: fq=-occupiedDays:[5 TO 30] Anybody has any idea? I have asked and searched all over the internet and seems solr does not support this. Any help would be really helpful! Thanks in advanced. Federico -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782083.html Sent from the Solr - User mailing list archive at Nabble.com.
Is there a way to implement a IntRangeField in Solr?
Hi all ! Here's my dreadful case, thank you for helping out! I want to have a document like this: ... -- multivalued range field 1 TO 10 5 TO 15 ... And the reason why I want to do this is because it's so much lighter than having all the numbers in there, of course. Just to be clear, I want to avoid having this in solr: ... -- multivalued range field 1 2 3 4 5 6 7 8 9 10 ... And then perform range queries on this range field like: fq=-occupiedDays:[5 TO 30] Anybody has any idea? I have asked and searched all over the internet and seems solr does not support this. Any help would be really helpful! Thanks in advanced. Federico -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-implement-a-IntRangeField-in-Solr-tp3782083p3782083.html Sent from the Solr - User mailing list archive at Nabble.com.
