RE: Solr join query
Erick, I'm using query time join, that requires colocated collections. I just have dictionaries replicas on all nodes of my cluster Like this: ={!join score=none from=id fromIndex=collection to=dictionary}*:* -- Vadim > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Monday, July 29, 2019 3:19 PM > To: solr-user@lucene.apache.org > Subject: Re: Solr join query > > Vadim: > > Are you using streaming or the special “cross collection” join that requires > colocated collection? > > > On Jul 29, 2019, at 4:23 AM, Vadim Ivanov intourist.ru> wrote: > > > > I'm using join of multivalued field to the id field of dictionary (another > collection). > > It's working pretty well > > > > -- > > Vadim > > > >> -Original Message- > >> From: Rajdeep Sahoo [mailto:rajdeepsahoo2...@gmail.com] > >> Sent: Monday, July 22, 2019 9:19 PM > >> To: solr-user@lucene.apache.org > >> Subject: Solr join query > >> > >> Can we join two solr collection based on multivalued field. > >
Re: Solr join query
Vadim: Are you using streaming or the special “cross collection” join that requires colocated collection? > On Jul 29, 2019, at 4:23 AM, Vadim Ivanov > wrote: > > I'm using join of multivalued field to the id field of dictionary (another > collection). > It's working pretty well > > -- > Vadim > >> -Original Message- >> From: Rajdeep Sahoo [mailto:rajdeepsahoo2...@gmail.com] >> Sent: Monday, July 22, 2019 9:19 PM >> To: solr-user@lucene.apache.org >> Subject: Solr join query >> >> Can we join two solr collection based on multivalued field. >
RE: Solr join query
I'm using join of multivalued field to the id field of dictionary (another collection). It's working pretty well -- Vadim > -Original Message- > From: Rajdeep Sahoo [mailto:rajdeepsahoo2...@gmail.com] > Sent: Monday, July 22, 2019 9:19 PM > To: solr-user@lucene.apache.org > Subject: Solr join query > > Can we join two solr collection based on multivalued field.
Solr join query
Can we join two solr collection based on multivalued field.
Re: Solr join query takes too long
Hello. There is no way to make it work fast. It executes expensive join operation for all docs/terms and then post filters with resulting docset. On Mon, Feb 5, 2018 at 9:53 AM, Aashish Agarwalwrote: > Hi > > I am using join query that joins 2 cores to get result. Since, number of > docs in both the cores are too high so I want to use post filter with join > query. But that is not working by providing cost=100. > > Is it implemented for join filter query? I use solr 4.6.0 > > Thanks, > Aashish > -- Sincerely yours Mikhail Khludnev
Solr join query takes too long
Hi I am using join query that joins 2 cores to get result. Since, number of docs in both the cores are too high so I want to use post filter with join query. But that is not working by providing cost=100. Is it implemented for join filter query? I use solr 4.6.0 Thanks, Aashish
Re: Question on SOLR join query
> I am reading similar issues and it says "initial join implementation is O(nterms)".. What does this mean? It enumerates all par_id terms every time. As an alternative for some of field types you can add {!join ... score=none ...}.. to trigger Lucene's join algorithm with O(fromDocs) ie if subordinate query returns no result it gets back rapidly. On Tue, Sep 26, 2017 at 4:59 AM, Jaimin Patelwrote: > I am facing a performance problem and could narrow it down to a join query > that we are using. The join is on a unique field. > > We have a person profile stored in RDB in a relational way. Like person > name table , address table etc. SOLR indexes are build using this RDB > data,Each children is stored as separate document with parent's unique id. > At query time , unique id of parent is joined with same in child > documents({!join > to=par_id from=par_id }) to allow search with AND condition for search > terms involving children data > > I am reading similar issues and it says "initial join implementation is > O(nterms)".. What does this mean ? I could not find any reference > explaining meaning of 0 num_terms_in_field. > > Regards, > Jai > -- Sincerely yours Mikhail Khludnev
Re: Question on SOLR join query
First of all, Solr is a _search_ engine, it wasn't built to be an RDBMS. Whenever I see this question (paraphrasing) "I've indexed my tables and want to use Solr just like a DB" I cringe. The join performance goes up with the number of unique values for the join field. High-cardinality fields are the poorer performing type, which is what "initial join implementation is O(nterms)" is expressing. Have you considered either denormalizing the data or using block joins? Best, Erick On Mon, Sep 25, 2017 at 6:59 PM, Jaimin Patelwrote: > I am facing a performance problem and could narrow it down to a join query > that we are using. The join is on a unique field. > > We have a person profile stored in RDB in a relational way. Like person > name table , address table etc. SOLR indexes are build using this RDB > data,Each children is stored as separate document with parent's unique id. > At query time , unique id of parent is joined with same in child > documents({!join > to=par_id from=par_id }) to allow search with AND condition for search > terms involving children data > > I am reading similar issues and it says "initial join implementation is > O(nterms)".. What does this mean ? I could not find any reference > explaining meaning of 0 num_terms_in_field. > > Regards, > Jai
Question on SOLR join query
I am facing a performance problem and could narrow it down to a join query that we are using. The join is on a unique field. We have a person profile stored in RDB in a relational way. Like person name table , address table etc. SOLR indexes are build using this RDB data,Each children is stored as separate document with parent's unique id. At query time , unique id of parent is joined with same in child documents({!join to=par_id from=par_id }) to allow search with AND condition for search terms involving children data I am reading similar issues and it says "initial join implementation is O(nterms)".. What does this mean ? I could not find any reference explaining meaning of 0 num_terms_in_field. Regards, Jai
Re: solr join query
You may want to look at fetch function of Streaming expressions http://lucene.apache.org/solr/guide/6_6/stream-decorators.html Thanks, Susheel On Tue, Sep 12, 2017 at 11:11 AM, Brian Yeewrote: > I have one solr collection used for auto-suggestions. If I submit a query > with q="coffe", I will get a responses back with documents that have a > field suggestion="coffee", "coffee table", "coffee maker", etc. I want to > know if those suggestions would have results if I used them to query a > second solr collection for products. > > Is there a way to do this without multiple solr queries? I could do the > first query to the suggestions collection, then take the results from that > and do more queries to the products collection. But I don't think that will > meet my performance requirements. > > Is this possible with a single join query? Any other ideas? >
solr join query
I have one solr collection used for auto-suggestions. If I submit a query with q="coffe", I will get a responses back with documents that have a field suggestion="coffee", "coffee table", "coffee maker", etc. I want to know if those suggestions would have results if I used them to query a second solr collection for products. Is there a way to do this without multiple solr queries? I could do the first query to the suggestions collection, then take the results from that and do more queries to the products collection. But I don't think that will meet my performance requirements. Is this possible with a single join query? Any other ideas?
Re: Trouble getting a solr join query done
I was to comment the very same solution! I think this will satisfy the user requirement. Thanks Antonio! Cheers 2015-07-13 12:22 GMT+01:00 Antonio David Pérez Morales adperezmora...@gmail.com: Hi again Yusnel Just to confirm, I have tested your use case and the query which returns what you need is this one: http://localhost:8983/solr/category/select?q={!join from=categoryId fromIndex=product to=id}*:*wt=jsonindent=truefq=name:clotheshl=false Please, check and let us know if it works for you Regards 2015-07-12 17:02 GMT+02:00 Antonio David Pérez Morales adperezmora...@gmail.com: Hi Yusnel I think the query is invalid. It should be q=clothesfq={!join from=type_id to=id fromIndex=products} or q=*:*fq={!join from=type_id to=id fromIndex=products}clothes as long as you are using an edismax parser or df param for default field, where clothes query is matched to. Regards 2015-07-11 2:23 GMT+02:00 Yusnel Rojas García yroj...@gmail.com: I have 2 indexes products { id, name, type_id .. } and categories { id, name .. } and I want to get all categories that match a name and have products in it. my best guess would be: http://localhost:8983/solr/categories/select?q=clothesfl=*,scorefq={!join from=type_id http://localhost:8983/solr/categories/select?q=clothesfl=*,scorefq=%7B!joinfrom=type_id to=id fromIndex=products}*:* but always get an empty response. help please! Is a better way of doing that without using another index? -- -- Benedetti Alessandro Visiting card - http://about.me/alessandro_benedetti Blog - http://alexbenedetti.blogspot.co.uk Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry? William Blake - Songs of Experience -1794 England
Re: Trouble getting a solr join query done
Hi again Yusnel Just to confirm, I have tested your use case and the query which returns what you need is this one: http://localhost:8983/solr/category/select?q={!join from=categoryId fromIndex=product to=id}*:*wt=jsonindent=truefq=name:clotheshl=false Please, check and let us know if it works for you Regards 2015-07-12 17:02 GMT+02:00 Antonio David Pérez Morales adperezmora...@gmail.com: Hi Yusnel I think the query is invalid. It should be q=clothesfq={!join from=type_id to=id fromIndex=products} or q=*:*fq={!join from=type_id to=id fromIndex=products}clothes as long as you are using an edismax parser or df param for default field, where clothes query is matched to. Regards 2015-07-11 2:23 GMT+02:00 Yusnel Rojas García yroj...@gmail.com: I have 2 indexes products { id, name, type_id .. } and categories { id, name .. } and I want to get all categories that match a name and have products in it. my best guess would be: http://localhost:8983/solr/categories/select?q=clothesfl=*,scorefq={!join from=type_id http://localhost:8983/solr/categories/select?q=clothesfl=*,scorefq=%7B!joinfrom=type_id to=id fromIndex=products}*:* but always get an empty response. help please! Is a better way of doing that without using another index?
Re: Trouble getting a solr join query done
Hi Yusnel I think the query is invalid. It should be q=clothesfq={!join from=type_id to=id fromIndex=products} or q=*:*fq={!join from=type_id to=id fromIndex=products}clothes as long as you are using an edismax parser or df param for default field, where clothes query is matched to. Regards 2015-07-11 2:23 GMT+02:00 Yusnel Rojas García yroj...@gmail.com: I have 2 indexes products { id, name, type_id .. } and categories { id, name .. } and I want to get all categories that match a name and have products in it. my best guess would be: http://localhost:8983/solr/categories/select?q=clothesfl=*,scorefq={!join from=type_id to=id fromIndex=products}*:* but always get an empty response. help please! Is a better way of doing that without using another index?
Trouble getting a solr join query done
I have 2 indexes products { id, name, type_id .. } and categories { id, name .. } and I want to get all categories that match a name and have products in it. my best guess would be: http://localhost:8983/solr/categories/select?q=clothesfl=*,scorefq={!join from=type_id to=id fromIndex=products}*:* but always get an empty response. help please! Is a better way of doing that without using another index?
Re: SOLR Join Query, Use highest weight.
Thanks! I will take a look at this. I do have an additional question, since after a bunch of digging I believe I am going to run into another dead end. I want to execute the join (or rollup) query, but I want the facets to represent the facets of all the child documents, not the resulting product documents. From what I gather, this is not possible. My thought process of what I want to get goes as follows: 1) Execute my search for children 2) Get the facets for all the children 3) Rollup the child dataset into its parent dataset, keeping the score. Is this easily possible with the tools available today? Thanks! Darin On Dec 1, 2014, at 11:01 PM, Mikhail Khludnev mkhlud...@griddynamics.com wrote: Hello, AFAIK {!join} doesn't supply any meaningful scores. I can suggest https://issues.apache.org/jira/browse/SOLR-6234 https://issues.apache.org/jira/browse/SOLR-6234 On Tue, Dec 2, 2014 at 4:35 AM, Darin Amos dari...@gmail.com mailto:dari...@gmail.com wrote: Hello, I had sent an email a few days ago talking about implementing a custom rollup query component. I have changed directions a little bit because I have learned about the JoinQuery. I have an index that contains a combination of parent and child documents. The parent child relationship is always one-to-many. Here is a very simple sample query: http://localhost:8983/solr/testcore/select?q=*:*fq={!join%20from=parent%20to=id}type:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child When I have a more specific query that actually give some meaningful weights: q=name:(*Shirt*)%20OR%20name:(*Small*) , it appears the rollup query assigns a weight to the parent of the last document encountered. For example, if a parents 2 children has weights of 1.4 and 0.4 without the join query, the parent has a weight of 0.4 after the join query. Is there a way that I can extend or modify the join query so it would assign the highest child weight to the parent document? Thanks!! Darin -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com http://www.griddynamics.com/ mkhlud...@griddynamics.com mailto:mkhlud...@griddynamics.com
Re: SOLR Join Query, Use highest weight.
Have you considered using grouping? If I understand your requirements, I think it does what you want. https://cwiki.apache.org/confluence/display/solr/Result+Grouping On 12/02/2014 12:59 PM, Darin Amos wrote: Thanks! I will take a look at this. I do have an additional question, since after a bunch of digging I believe I am going to run into another dead end. I want to execute the join (or rollup) query, but I want the facets to represent the facets of all the child documents, not the resulting product documents. From what I gather, this is not possible. My thought process of what I want to get goes as follows: 1) Execute my search for children 2) Get the facets for all the children 3) Rollup the child dataset into its parent dataset, keeping the score. Is this easily possible with the tools available today? Thanks! Darin On Dec 1, 2014, at 11:01 PM, Mikhail Khludnev mkhlud...@griddynamics.com wrote: Hello, AFAIK {!join} doesn't supply any meaningful scores. I can suggest https://issues.apache.org/jira/browse/SOLR-6234 https://issues.apache.org/jira/browse/SOLR-6234 On Tue, Dec 2, 2014 at 4:35 AM, Darin Amos dari...@gmail.com mailto:dari...@gmail.com wrote: Hello, I had sent an email a few days ago talking about implementing a custom rollup query component. I have changed directions a little bit because I have learned about the JoinQuery. I have an index that contains a combination of parent and child documents. The parent child relationship is always one-to-many. Here is a very simple sample query: http://localhost:8983/solr/testcore/select?q=*:*fq={!join%20from=parent%20to=id}type:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child When I have a more specific query that actually give some meaningful weights: q=name:(*Shirt*)%20OR%20name:(*Small*) , it appears the rollup query assigns a weight to the parent of the last document encountered. For example, if a parents 2 children has weights of 1.4 and 0.4 without the join query, the parent has a weight of 0.4 after the join query. Is there a way that I can extend or modify the join query so it would assign the highest child weight to the parent document? Thanks!! Darin -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com http://www.griddynamics.com/ mkhlud...@griddynamics.com mailto:mkhlud...@griddynamics.com
Re: SOLR Join Query, Use highest weight.
Hi, Thanks for the response, I have considered grouping often, but grouping does not return the parent document, just the group id. I would still have to add something to take the group id’s and get the parent documents. Thanks Darin On Dec 2, 2014, at 2:11 PM, Michael Sokolov msoko...@safaribooksonline.com wrote: Have you considered using grouping? If I understand your requirements, I think it does what you want. https://cwiki.apache.org/confluence/display/solr/Result+Grouping https://cwiki.apache.org/confluence/display/solr/Result+Grouping On 12/02/2014 12:59 PM, Darin Amos wrote: Thanks! I will take a look at this. I do have an additional question, since after a bunch of digging I believe I am going to run into another dead end. I want to execute the join (or rollup) query, but I want the facets to represent the facets of all the child documents, not the resulting product documents. From what I gather, this is not possible. My thought process of what I want to get goes as follows: 1) Execute my search for children 2) Get the facets for all the children 3) Rollup the child dataset into its parent dataset, keeping the score. Is this easily possible with the tools available today? Thanks! Darin On Dec 1, 2014, at 11:01 PM, Mikhail Khludnev mkhlud...@griddynamics.com wrote: Hello, AFAIK {!join} doesn't supply any meaningful scores. I can suggest https://issues.apache.org/jira/browse/SOLR-6234 https://issues.apache.org/jira/browse/SOLR-6234 https://issues.apache.org/jira/browse/SOLR-6234 https://issues.apache.org/jira/browse/SOLR-6234 On Tue, Dec 2, 2014 at 4:35 AM, Darin Amos dari...@gmail.com mailto:dari...@gmail.com mailto:dari...@gmail.com mailto:dari...@gmail.com wrote: Hello, I had sent an email a few days ago talking about implementing a custom rollup query component. I have changed directions a little bit because I have learned about the JoinQuery. I have an index that contains a combination of parent and child documents. The parent child relationship is always one-to-many. Here is a very simple sample query: http://localhost:8983/solr/testcore/select?q=*:*fq={!join%20from=parent%20to=id}type:child http://localhost:8983/solr/testcore/select?q=*:*fq={!join%20from=parent%20to=id}type:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:childhttp://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child When I have a more specific query that actually give some meaningful weights: q=name:(*Shirt*)%20OR%20name:(*Small*) , it appears the rollup query assigns a weight to the parent of the last document encountered. For example, if a parents 2 children has weights of 1.4 and 0.4 without the join query, the parent has a weight of 0.4 after the join query. Is there a way that I can extend or modify the join query so it would assign the highest child weight to the parent document? Thanks!! Darin -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com http://www.griddynamics.com/ http://www.griddynamics.com/ http://www.griddynamics.com/ mkhlud...@griddynamics.com mailto:mkhlud...@griddynamics.com mailto:mkhlud...@griddynamics.com mailto:mkhlud...@griddynamics.com
Re: SOLR Join Query, Use highest weight.
We simply index parent and child documents with the same field value, and group on that, querying both parent and child documents. If you boost the parent it will show up as the first result in the group. Then you get all related documents together. in the same group. -Mike On 12/02/2014 02:27 PM, Darin Amos wrote: Hi, Thanks for the response, I have considered grouping often, but grouping does not return the parent document, just the group id. I would still have to add something to take the group id’s and get the parent documents. Thanks Darin On Dec 2, 2014, at 2:11 PM, Michael Sokolov msoko...@safaribooksonline.com wrote: Have you considered using grouping? If I understand your requirements, I think it does what you want. https://cwiki.apache.org/confluence/display/solr/Result+Grouping https://cwiki.apache.org/confluence/display/solr/Result+Grouping On 12/02/2014 12:59 PM, Darin Amos wrote: Thanks! I will take a look at this. I do have an additional question, since after a bunch of digging I believe I am going to run into another dead end. I want to execute the join (or rollup) query, but I want the facets to represent the facets of all the child documents, not the resulting product documents. From what I gather, this is not possible. My thought process of what I want to get goes as follows: 1) Execute my search for children 2) Get the facets for all the children 3) Rollup the child dataset into its parent dataset, keeping the score. Is this easily possible with the tools available today? Thanks! Darin On Dec 1, 2014, at 11:01 PM, Mikhail Khludnev mkhlud...@griddynamics.com wrote: Hello, AFAIK {!join} doesn't supply any meaningful scores. I can suggest https://issues.apache.org/jira/browse/SOLR-6234 https://issues.apache.org/jira/browse/SOLR-6234 https://issues.apache.org/jira/browse/SOLR-6234 https://issues.apache.org/jira/browse/SOLR-6234 On Tue, Dec 2, 2014 at 4:35 AM, Darin Amos dari...@gmail.com mailto:dari...@gmail.com mailto:dari...@gmail.com mailto:dari...@gmail.com wrote: Hello, I had sent an email a few days ago talking about implementing a custom rollup query component. I have changed directions a little bit because I have learned about the JoinQuery. I have an index that contains a combination of parent and child documents. The parent child relationship is always one-to-many. Here is a very simple sample query: http://localhost:8983/solr/testcore/select?q=*:*fq={!join%20from=parent%20to=id}type:child http://localhost:8983/solr/testcore/select?q=*:*fq={!join%20from=parent%20to=id}type:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:childhttp://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child When I have a more specific query that actually give some meaningful weights: q=name:(*Shirt*)%20OR%20name:(*Small*) , it appears the rollup query assigns a weight to the parent of the last document encountered. For example, if a parents 2 children has weights of 1.4 and 0.4 without the join query, the parent has a weight of 0.4 after the join query. Is there a way that I can extend or modify the join query so it would assign the highest child weight to the parent document? Thanks!! Darin -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com http://www.griddynamics.com/ http://www.griddynamics.com/ http://www.griddynamics.com/ mkhlud...@griddynamics.com mailto:mkhlud...@griddynamics.com mailto:mkhlud...@griddynamics.com mailto:mkhlud...@griddynamics.com
SOLR Join Query, Use highest weight.
Hello, I had sent an email a few days ago talking about implementing a custom rollup query component. I have changed directions a little bit because I have learned about the JoinQuery. I have an index that contains a combination of parent and child documents. The parent child relationship is always one-to-many. Here is a very simple sample query: http://localhost:8983/solr/testcore/select?q=*:*fq={!join%20from=parent%20to=id}type:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child When I have a more specific query that actually give some meaningful weights: q=name:(*Shirt*)%20OR%20name:(*Small*) , it appears the rollup query assigns a weight to the parent of the last document encountered. For example, if a parents 2 children has weights of 1.4 and 0.4 without the join query, the parent has a weight of 0.4 after the join query. Is there a way that I can extend or modify the join query so it would assign the highest child weight to the parent document? Thanks!! Darin
Re: SOLR Join Query, Use highest weight.
Hello, AFAIK {!join} doesn't supply any meaningful scores. I can suggest https://issues.apache.org/jira/browse/SOLR-6234 On Tue, Dec 2, 2014 at 4:35 AM, Darin Amos dari...@gmail.com wrote: Hello, I had sent an email a few days ago talking about implementing a custom rollup query component. I have changed directions a little bit because I have learned about the JoinQuery. I have an index that contains a combination of parent and child documents. The parent child relationship is always one-to-many. Here is a very simple sample query: http://localhost:8983/solr/testcore/select?q=*:*fq={!join%20from=parent%20to=id}type:child http://localhost:8983/solr/testcore/select?q=*:*fq=%7B!join%20from=parent%20to=id%7Dtype:child When I have a more specific query that actually give some meaningful weights: q=name:(*Shirt*)%20OR%20name:(*Small*) , it appears the rollup query assigns a weight to the parent of the last document encountered. For example, if a parents 2 children has weights of 1.4 and 0.4 without the join query, the parent has a weight of 0.4 after the join query. Is there a way that I can extend or modify the join query so it would assign the highest child weight to the parent document? Thanks!! Darin -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: How Solr join query works?
Hello, Did you consider using CollapsingQueryParser or FieldCollapsing? http://wiki.apache.org/solr/FieldCollapsing https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-CollapsingQueryParser Ahmet On Monday, January 13, 2014 4:56 AM, solr2020 psgoms...@gmail.com wrote: Hi All, Can anyone please explain how solr join query works in solr4.2. we have 2 different documents.Both are in the same index. document1 contains the columns: docdate: 01-12-2012 previousmonthdate :01-11-2012 price:15 and some more fields. document2 contains: docdate :01-11-2012 previousmonthdate :01-10-2012 price:10 and some more fields. Here we have the same value in previousmonthdate (in document1) and docdate (in document2).So we want to make a join query based on this to retrieve these in a single document. the final document should look like this. docdate: 01-12-2012 previousmonthdate :01-11-2012 price:15 price:10(this is from document2) Is is possible using Solr join query??? Or do we have any other approach?. Please help.. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/How-Solr-join-query-works-tp4110982.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How Solr join query works?
It's called pseudo join for a reason. It's not built to do what you want. There's no way to combine fields in the from and to clauses in the output. The from clause can be thought of as a filter. The first choice is usually to denormalize the data if possible. Best, Erick On Sun, Jan 12, 2014 at 9:55 PM, solr2020 psgoms...@gmail.com wrote: Hi All, Can anyone please explain how solr join query works in solr4.2. we have 2 different documents.Both are in the same index. document1 contains the columns: docdate: 01-12-2012 previousmonthdate :01-11-2012 price:15 and some more fields. document2 contains: docdate :01-11-2012 previousmonthdate :01-10-2012 price:10 and some more fields. Here we have the same value in previousmonthdate (in document1) and docdate (in document2).So we want to make a join query based on this to retrieve these in a single document. the final document should look like this. docdate: 01-12-2012 previousmonthdate :01-11-2012 price:15 price:10(this is from document2) Is is possible using Solr join query??? Or do we have any other approach?. Please help.. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/How-Solr-join-query-works-tp4110982.html Sent from the Solr - User mailing list archive at Nabble.com.
How Solr join query works?
Hi All, Can anyone please explain how solr join query works in solr4.2. we have 2 different documents.Both are in the same index. document1 contains the columns: docdate: 01-12-2012 previousmonthdate :01-11-2012 price:15 and some more fields. document2 contains: docdate :01-11-2012 previousmonthdate :01-10-2012 price:10 and some more fields. Here we have the same value in previousmonthdate (in document1) and docdate (in document2).So we want to make a join query based on this to retrieve these in a single document. the final document should look like this. docdate: 01-12-2012 previousmonthdate :01-11-2012 price:15 price:10(this is from document2) Is is possible using Solr join query??? Or do we have any other approach?. Please help.. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/How-Solr-join-query-works-tp4110982.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Join query with fq not correctly filtering results?
Thanks for your persistence in tracking this down Mike! I'm going to start looking into this now... -Yonik lucidimagination.com On Thu, Jan 26, 2012 at 11:06 PM, Mike Hugo m...@piragua.com wrote: I created issue https://issues.apache.org/jira/browse/SOLR-3062 for this problem. I was able to track it down to something in this commit - http://svn.apache.org/viewvc?view=revisionrevision=1188624 (LUCENE-1536: Filters can now be applied down-low, if their DocIdSet implements a new bits() method, returning all documents in a random access way ) - before that commit the join / fq functionality works as expected / documented on the wiki page. After that commit it's broken. Any assistance is greatly appreciated! Thanks, Mike On Thu, Jan 26, 2012 at 11:04 AM, Mike Hugo m...@piragua.com wrote: Hello, I'm trying out the Solr JOIN query functionality on trunk. I have the latest checkout, revision #1236272 - I did the following steps to get the example up and running: cd solr ant example java -jar start.jar cd exampledocs java -jar post.jar *.xml Then I tried a few of the sample queries on the wiki page http://wiki.apache.org/solr/Join. In particular, this is one that I'm interest in Find all manufacturer docs named belkin, then join them against (product) docs and filter that list to only products with a price less than 12 dollars http://localhost:8983/solr/select?q={!join+from=id+to=manu_id_s}compName_s:Belkinfq=price:%5B%2A+TO+12%5Dhttp://localhost:8983/solr/select?q=%7B!join+from=id+to=manu_id_s%7DcompName_s:Belkinfq=price:%5B%2A+TO+12%5D However, when I run that query, I get two results, one with a price of 19.95 and another with a price of 11.5 Because of the filter query, I'm only expecting to see one result - the one with a price of 11.99. I was also able to replicate this in a unit test added to org.apache.solr.TestJoin: @Test public void testJoin_withFilterQuery() throws Exception { assertU(add(doc(id, 1,name, john, title, Director, dept_s,Engineering))); assertU(add(doc(id, 2,name, mark, title, VP, dept_s,Marketing))); assertU(add(doc(id, 3,name, nancy, title, MTS, dept_s,Sales))); assertU(add(doc(id, 4,name, dave, title, MTS, dept_s,Support, dept_s,Engineering))); assertU(add(doc(id, 5,name, tina, title, VP, dept_s,Engineering))); assertU(add(doc(id,10, dept_id_s, Engineering, text,These guys develop stuff))); assertU(add(doc(id,11, dept_id_s, Marketing, text,These guys make you look good))); assertU(add(doc(id,12, dept_id_s, Sales, text,These guys sell stuff))); assertU(add(doc(id,13, dept_id_s, Support, text,These guys help customers))); assertU(commit()); //*** //This works as expected - the correct number of results are found //*** // find people that develop stuff assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id) ,/response=={'numFound':3,'start':0,'docs':[{'id':'1'},{'id':'4'},{'id':'5'}]} ); * // * // this fails - the response returned finds all three people - it should only find John* * // expected =/response=={numFound:1,start:0,docs:[{id:1}]}* * // response = {* * // responseHeader:{* * // status:0,* * // QTime:4},* * // response:{numFound:3,start:0,docs:[* * // {* * // id:1},* * // {* * // id:4},* * // {* * // id:5}]* * // }}* * // * // find people that develop stuff - but limit via filter query to a name of john* * assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id, fq, name:john)* * ,/response=={'numFound':1,'start':0,'docs':[{'id':'1'}]}* * );* } Interestingly, I know this worked at some point. I had a snapshot build in my ivy cache from 10/2/2011 and it was working with that build maven_artifacts/org/apache/solr/ solr/4.0-SNAPSHOT/solr-4.0-20111002.161157-1.pom Mike
Re: Solr Join query with fq not correctly filtering results?
Thanks Yonik!! The join functionality is proving extremely useful for us in a specific use case - we're really looking forward to join and other cool features coming in Solr4!! Mike On Wed, Feb 1, 2012 at 3:30 PM, Yonik Seeley yo...@lucidimagination.comwrote: Thanks for your persistence in tracking this down Mike! I'm going to start looking into this now... -Yonik lucidimagination.com On Thu, Jan 26, 2012 at 11:06 PM, Mike Hugo m...@piragua.com wrote: I created issue https://issues.apache.org/jira/browse/SOLR-3062 for this problem. I was able to track it down to something in this commit - http://svn.apache.org/viewvc?view=revisionrevision=1188624(LUCENE-1536: Filters can now be applied down-low, if their DocIdSet implements a new bits() method, returning all documents in a random access way ) - before that commit the join / fq functionality works as expected / documented on the wiki page. After that commit it's broken. Any assistance is greatly appreciated! Thanks, Mike On Thu, Jan 26, 2012 at 11:04 AM, Mike Hugo m...@piragua.com wrote: Hello, I'm trying out the Solr JOIN query functionality on trunk. I have the latest checkout, revision #1236272 - I did the following steps to get the example up and running: cd solr ant example java -jar start.jar cd exampledocs java -jar post.jar *.xml Then I tried a few of the sample queries on the wiki page http://wiki.apache.org/solr/Join. In particular, this is one that I'm interest in Find all manufacturer docs named belkin, then join them against (product) docs and filter that list to only products with a price less than 12 dollars http://localhost:8983/solr/select?q={!join+from=id+to=manu_id_s}compName_s:Belkinfq=price:%5B%2A+TO+12%5D http://localhost:8983/solr/select?q=%7B!join+from=id+to=manu_id_s%7DcompName_s:Belkinfq=price:%5B%2A+TO+12%5D However, when I run that query, I get two results, one with a price of 19.95 and another with a price of 11.5 Because of the filter query, I'm only expecting to see one result - the one with a price of 11.99. I was also able to replicate this in a unit test added to org.apache.solr.TestJoin: @Test public void testJoin_withFilterQuery() throws Exception { assertU(add(doc(id, 1,name, john, title, Director, dept_s,Engineering))); assertU(add(doc(id, 2,name, mark, title, VP, dept_s,Marketing))); assertU(add(doc(id, 3,name, nancy, title, MTS, dept_s,Sales))); assertU(add(doc(id, 4,name, dave, title, MTS, dept_s,Support, dept_s,Engineering))); assertU(add(doc(id, 5,name, tina, title, VP, dept_s,Engineering))); assertU(add(doc(id,10, dept_id_s, Engineering, text,These guys develop stuff))); assertU(add(doc(id,11, dept_id_s, Marketing, text,These guys make you look good))); assertU(add(doc(id,12, dept_id_s, Sales, text,These guys sell stuff))); assertU(add(doc(id,13, dept_id_s, Support, text,These guys help customers))); assertU(commit()); //*** //This works as expected - the correct number of results are found //*** // find people that develop stuff assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id) ,/response=={'numFound':3,'start':0,'docs':[{'id':'1'},{'id':'4'},{'id':'5'}]} ); *// *// this fails - the response returned finds all three people - it should only find John* *//expected =/response=={numFound:1,start:0,docs:[{id:1}]}* *//response = {* *//responseHeader:{* *// status:0,* *// QTime:4},* *//response:{numFound:3,start:0,docs:[* *// {* *//id:1},* *// {* *//id:4},* *// {* *//id:5}]* *//}}* *// *// find people that develop stuff - but limit via filter query to a name of john* *assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id, fq, name:john)* *,/response=={'numFound':1,'start':0,'docs':[{'id':'1'}]}* *);* } Interestingly, I know this worked at some point. I had a snapshot build in my ivy cache from 10/2/2011 and it was working with that build maven_artifacts/org/apache/solr/ solr/4.0-SNAPSHOT/solr-4.0-20111002.161157-1.pom Mike
Re: Solr Join query with fq not correctly filtering results?
I've been looking into this a bit further and am trying to figure out why the FQ isn't getting applied. Can anyone point me to a good spot in the code to start looking at how FQ parameters are applied to query results in Solr4? Thanks, Mike On Thu, Jan 26, 2012 at 10:06 PM, Mike Hugo m...@piragua.com wrote: I created issue https://issues.apache.org/jira/browse/SOLR-3062 for this problem. I was able to track it down to something in this commit - http://svn.apache.org/viewvc?view=revisionrevision=1188624 (LUCENE-1536: Filters can now be applied down-low, if their DocIdSet implements a new bits() method, returning all documents in a random access way ) - before that commit the join / fq functionality works as expected / documented on the wiki page. After that commit it's broken. Any assistance is greatly appreciated! Thanks, Mike On Thu, Jan 26, 2012 at 11:04 AM, Mike Hugo m...@piragua.com wrote: Hello, I'm trying out the Solr JOIN query functionality on trunk. I have the latest checkout, revision #1236272 - I did the following steps to get the example up and running: cd solr ant example java -jar start.jar cd exampledocs java -jar post.jar *.xml Then I tried a few of the sample queries on the wiki page http://wiki.apache.org/solr/Join. In particular, this is one that I'm interest in Find all manufacturer docs named belkin, then join them against (product) docs and filter that list to only products with a price less than 12 dollars http://localhost:8983/solr/select?q={!join+from=id+to=manu_id_s}compName_s:Belkinfq=price:%5B%2A+TO+12%5Dhttp://localhost:8983/solr/select?q=%7B!join+from=id+to=manu_id_s%7DcompName_s:Belkinfq=price:%5B%2A+TO+12%5D However, when I run that query, I get two results, one with a price of 19.95 and another with a price of 11.5 Because of the filter query, I'm only expecting to see one result - the one with a price of 11.99. I was also able to replicate this in a unit test added to org.apache.solr.TestJoin: @Test public void testJoin_withFilterQuery() throws Exception { assertU(add(doc(id, 1,name, john, title, Director, dept_s,Engineering))); assertU(add(doc(id, 2,name, mark, title, VP, dept_s,Marketing))); assertU(add(doc(id, 3,name, nancy, title, MTS, dept_s,Sales))); assertU(add(doc(id, 4,name, dave, title, MTS, dept_s,Support, dept_s,Engineering))); assertU(add(doc(id, 5,name, tina, title, VP, dept_s,Engineering))); assertU(add(doc(id,10, dept_id_s, Engineering, text,These guys develop stuff))); assertU(add(doc(id,11, dept_id_s, Marketing, text,These guys make you look good))); assertU(add(doc(id,12, dept_id_s, Sales, text,These guys sell stuff))); assertU(add(doc(id,13, dept_id_s, Support, text,These guys help customers))); assertU(commit()); //*** //This works as expected - the correct number of results are found //*** // find people that develop stuff assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id) ,/response=={'numFound':3,'start':0,'docs':[{'id':'1'},{'id':'4'},{'id':'5'}]} ); *// *// this fails - the response returned finds all three people - it should only find John* *//expected =/response=={numFound:1,start:0,docs:[{id:1}]}* *//response = {* *//responseHeader:{* *// status:0,* *// QTime:4},* *//response:{numFound:3,start:0,docs:[* *// {* *//id:1},* *// {* *//id:4},* *// {* *//id:5}]* *//}}* *// *// find people that develop stuff - but limit via filter query to a name of john* *assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id, fq, name:john)* *,/response=={'numFound':1,'start':0,'docs':[{'id':'1'}]}* *);* } Interestingly, I know this worked at some point. I had a snapshot build in my ivy cache from 10/2/2011 and it was working with that build maven_artifacts/org/apache/solr/ solr/4.0-SNAPSHOT/solr-4.0-20111002.161157-1.pom Mike
Solr Join query with fq not correctly filtering results?
Hello, I'm trying out the Solr JOIN query functionality on trunk. I have the latest checkout, revision #1236272 - I did the following steps to get the example up and running: cd solr ant example java -jar start.jar cd exampledocs java -jar post.jar *.xml Then I tried a few of the sample queries on the wiki page http://wiki.apache.org/solr/Join. In particular, this is one that I'm interest in Find all manufacturer docs named belkin, then join them against (product) docs and filter that list to only products with a price less than 12 dollars http://localhost:8983/solr/select?q={!join+from=id+to=manu_id_s}compName_s:Belkinfq=price:%5B%2A+TO+12%5D However, when I run that query, I get two results, one with a price of 19.95 and another with a price of 11.5 Because of the filter query, I'm only expecting to see one result - the one with a price of 11.99. I was also able to replicate this in a unit test added to org.apache.solr.TestJoin: @Test public void testJoin_withFilterQuery() throws Exception { assertU(add(doc(id, 1,name, john, title, Director, dept_s,Engineering))); assertU(add(doc(id, 2,name, mark, title, VP, dept_s,Marketing))); assertU(add(doc(id, 3,name, nancy, title, MTS, dept_s,Sales))); assertU(add(doc(id, 4,name, dave, title, MTS, dept_s,Support, dept_s,Engineering))); assertU(add(doc(id, 5,name, tina, title, VP, dept_s,Engineering))); assertU(add(doc(id,10, dept_id_s, Engineering, text,These guys develop stuff))); assertU(add(doc(id,11, dept_id_s, Marketing, text,These guys make you look good))); assertU(add(doc(id,12, dept_id_s, Sales, text,These guys sell stuff))); assertU(add(doc(id,13, dept_id_s, Support, text,These guys help customers))); assertU(commit()); //*** //This works as expected - the correct number of results are found //*** // find people that develop stuff assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id) ,/response=={'numFound':3,'start':0,'docs':[{'id':'1'},{'id':'4'},{'id':'5'}]} ); *// *// this fails - the response returned finds all three people - it should only find John* *//expected =/response=={numFound:1,start:0,docs:[{id:1}]} * *//response = {* *//responseHeader:{* *// status:0,* *// QTime:4},* *//response:{numFound:3,start:0,docs:[* *// {* *//id:1},* *// {* *//id:4},* *// {* *//id:5}]* *//}}* *// *// find people that develop stuff - but limit via filter query to a name of john* *assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id, fq, name:john)* *,/response=={'numFound':1,'start':0,'docs':[{'id':'1'}]}* *);* } Interestingly, I know this worked at some point. I had a snapshot build in my ivy cache from 10/2/2011 and it was working with that build maven_artifacts/org/apache/solr/ solr/4.0-SNAPSHOT/solr-4.0-20111002.161157-1.pom Mike
Re: Solr Join query with fq not correctly filtering results?
I created issue https://issues.apache.org/jira/browse/SOLR-3062 for this problem. I was able to track it down to something in this commit - http://svn.apache.org/viewvc?view=revisionrevision=1188624 (LUCENE-1536: Filters can now be applied down-low, if their DocIdSet implements a new bits() method, returning all documents in a random access way ) - before that commit the join / fq functionality works as expected / documented on the wiki page. After that commit it's broken. Any assistance is greatly appreciated! Thanks, Mike On Thu, Jan 26, 2012 at 11:04 AM, Mike Hugo m...@piragua.com wrote: Hello, I'm trying out the Solr JOIN query functionality on trunk. I have the latest checkout, revision #1236272 - I did the following steps to get the example up and running: cd solr ant example java -jar start.jar cd exampledocs java -jar post.jar *.xml Then I tried a few of the sample queries on the wiki page http://wiki.apache.org/solr/Join. In particular, this is one that I'm interest in Find all manufacturer docs named belkin, then join them against (product) docs and filter that list to only products with a price less than 12 dollars http://localhost:8983/solr/select?q={!join+from=id+to=manu_id_s}compName_s:Belkinfq=price:%5B%2A+TO+12%5Dhttp://localhost:8983/solr/select?q=%7B!join+from=id+to=manu_id_s%7DcompName_s:Belkinfq=price:%5B%2A+TO+12%5D However, when I run that query, I get two results, one with a price of 19.95 and another with a price of 11.5 Because of the filter query, I'm only expecting to see one result - the one with a price of 11.99. I was also able to replicate this in a unit test added to org.apache.solr.TestJoin: @Test public void testJoin_withFilterQuery() throws Exception { assertU(add(doc(id, 1,name, john, title, Director, dept_s,Engineering))); assertU(add(doc(id, 2,name, mark, title, VP, dept_s,Marketing))); assertU(add(doc(id, 3,name, nancy, title, MTS, dept_s,Sales))); assertU(add(doc(id, 4,name, dave, title, MTS, dept_s,Support, dept_s,Engineering))); assertU(add(doc(id, 5,name, tina, title, VP, dept_s,Engineering))); assertU(add(doc(id,10, dept_id_s, Engineering, text,These guys develop stuff))); assertU(add(doc(id,11, dept_id_s, Marketing, text,These guys make you look good))); assertU(add(doc(id,12, dept_id_s, Sales, text,These guys sell stuff))); assertU(add(doc(id,13, dept_id_s, Support, text,These guys help customers))); assertU(commit()); //*** //This works as expected - the correct number of results are found //*** // find people that develop stuff assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id) ,/response=={'numFound':3,'start':0,'docs':[{'id':'1'},{'id':'4'},{'id':'5'}]} ); *// *// this fails - the response returned finds all three people - it should only find John* *//expected =/response=={numFound:1,start:0,docs:[{id:1}]}* *//response = {* *//responseHeader:{* *// status:0,* *// QTime:4},* *//response:{numFound:3,start:0,docs:[* *// {* *//id:1},* *// {* *//id:4},* *// {* *//id:5}]* *//}}* *// *// find people that develop stuff - but limit via filter query to a name of john* *assertJQ(req(q,{!join from=dept_id_s to=dept_s}text:develop, fl,id, fq, name:john)* *,/response=={'numFound':1,'start':0,'docs':[{'id':'1'}]}* *);* } Interestingly, I know this worked at some point. I had a snapshot build in my ivy cache from 10/2/2011 and it was working with that build maven_artifacts/org/apache/solr/ solr/4.0-SNAPSHOT/solr-4.0-20111002.161157-1.pom Mike