Re: Commit different database rows to solr with same id value?
Just use the address in the url. You don't have to use the core name if the defaults are set, which is usually collection1. So it's something like http://host:port/solr/core2/update? blah blah blah Erick On Wed, Jul 10, 2013 at 4:17 PM, Jason Huang jason.hu...@icare.com wrote: Thanks David. I am actually trying to commit the database row on the fly, not DIH. :) Anyway, if I understand you correctly, basically you are suggesting to modify the value of the primary key and pass the new value to id before committing to solr. This could probably be one solution. What if I want to commit the data from table2 to a new core? Anyone knows how I can do that? thanks, Jason On Wed, Jul 10, 2013 at 11:18 AM, David Quarterman da...@corexe.com wrote: Hi Jason, Assuming you're using DIH, why not build a new, unique id within the query to use as the 'doc_id' for SOLR? We do something like this in one of our collections. In MySQL, try this (don't know what it would be for any other db but there must be equivalents): select @rownum:=@rownum+1 rowid, t.* from (main select query) t, (select @rownum:=0) s Regards, DQ -Original Message- From: Jason Huang [mailto:jason.hu...@icare.com] Sent: 10 July 2013 15:50 To: solr-user@lucene.apache.org Subject: Commit different database rows to solr with same id value? Hello, I am trying to use Solr to store fields from two different database tables, where the primary keys are in the format of 1, 2, 3, In Java, we build different POJO classes for these two database tables: table1.java @SolrIndex(name=id) private String idTable1 table2.java @SolrIndex(name=id) private String idTable2 And later we add these fields defined in the two different types of tables and commit it to solrServer. Here is the scenario where I am having issues: (1) commit a row from table1 with primary key = 3, this generates a document in Solr (2) commit another row from table2 with the same value of primary key = 3, this overwrites the document generated in step (1). What we really want to achieve is to keep both rows in (1) and (2) because they are from different tables. I've read something from google search and it appears that we might be able to do it via keeping multiple cores in solr? Could anyone point at how to implement multiple core to achieve this? To be more specific, when I commit the row as a document, I don't have a place to pick a certain core and I am not sure if it makes any sense for me to specify a core when I commit the document since the layer I am working on should abstract it away from me. The second question is - if we don't want to do a multicore (since we can't easily search for related data between multiple cores), how can we resolve this issue so both rows from different database table which shares the same primary key still exist? We don't want to have to always change the primary key format to ensure a uniqueness of the primary key among all different types of database tables. thanks! Jason
Re: Commit different database rows to solr with same id value?
cool. so far I've been using the default collection 1 only. thanks, Jason On Thu, Jul 11, 2013 at 7:57 AM, Erick Erickson erickerick...@gmail.comwrote: Just use the address in the url. You don't have to use the core name if the defaults are set, which is usually collection1. So it's something like http://host:port/solr/core2/update? blah blah blah Erick On Wed, Jul 10, 2013 at 4:17 PM, Jason Huang jason.hu...@icare.com wrote: Thanks David. I am actually trying to commit the database row on the fly, not DIH. :) Anyway, if I understand you correctly, basically you are suggesting to modify the value of the primary key and pass the new value to id before committing to solr. This could probably be one solution. What if I want to commit the data from table2 to a new core? Anyone knows how I can do that? thanks, Jason On Wed, Jul 10, 2013 at 11:18 AM, David Quarterman da...@corexe.com wrote: Hi Jason, Assuming you're using DIH, why not build a new, unique id within the query to use as the 'doc_id' for SOLR? We do something like this in one of our collections. In MySQL, try this (don't know what it would be for any other db but there must be equivalents): select @rownum:=@rownum+1 rowid, t.* from (main select query) t, (select @rownum:=0) s Regards, DQ -Original Message- From: Jason Huang [mailto:jason.hu...@icare.com] Sent: 10 July 2013 15:50 To: solr-user@lucene.apache.org Subject: Commit different database rows to solr with same id value? Hello, I am trying to use Solr to store fields from two different database tables, where the primary keys are in the format of 1, 2, 3, In Java, we build different POJO classes for these two database tables: table1.java @SolrIndex(name=id) private String idTable1 table2.java @SolrIndex(name=id) private String idTable2 And later we add these fields defined in the two different types of tables and commit it to solrServer. Here is the scenario where I am having issues: (1) commit a row from table1 with primary key = 3, this generates a document in Solr (2) commit another row from table2 with the same value of primary key = 3, this overwrites the document generated in step (1). What we really want to achieve is to keep both rows in (1) and (2) because they are from different tables. I've read something from google search and it appears that we might be able to do it via keeping multiple cores in solr? Could anyone point at how to implement multiple core to achieve this? To be more specific, when I commit the row as a document, I don't have a place to pick a certain core and I am not sure if it makes any sense for me to specify a core when I commit the document since the layer I am working on should abstract it away from me. The second question is - if we don't want to do a multicore (since we can't easily search for related data between multiple cores), how can we resolve this issue so both rows from different database table which shares the same primary key still exist? We don't want to have to always change the primary key format to ensure a uniqueness of the primary key among all different types of database tables. thanks! Jason
Commit different database rows to solr with same id value?
Hello, I am trying to use Solr to store fields from two different database tables, where the primary keys are in the format of 1, 2, 3, In Java, we build different POJO classes for these two database tables: table1.java @SolrIndex(name=id) private String idTable1 table2.java @SolrIndex(name=id) private String idTable2 And later we add these fields defined in the two different types of tables and commit it to solrServer. Here is the scenario where I am having issues: (1) commit a row from table1 with primary key = 3, this generates a document in Solr (2) commit another row from table2 with the same value of primary key = 3, this overwrites the document generated in step (1). What we really want to achieve is to keep both rows in (1) and (2) because they are from different tables. I've read something from google search and it appears that we might be able to do it via keeping multiple cores in solr? Could anyone point at how to implement multiple core to achieve this? To be more specific, when I commit the row as a document, I don't have a place to pick a certain core and I am not sure if it makes any sense for me to specify a core when I commit the document since the layer I am working on should abstract it away from me. The second question is - if we don't want to do a multicore (since we can't easily search for related data between multiple cores), how can we resolve this issue so both rows from different database table which shares the same primary key still exist? We don't want to have to always change the primary key format to ensure a uniqueness of the primary key among all different types of database tables. thanks! Jason
RE: Commit different database rows to solr with same id value?
Hi Jason, Assuming you're using DIH, why not build a new, unique id within the query to use as the 'doc_id' for SOLR? We do something like this in one of our collections. In MySQL, try this (don't know what it would be for any other db but there must be equivalents): select @rownum:=@rownum+1 rowid, t.* from (main select query) t, (select @rownum:=0) s Regards, DQ -Original Message- From: Jason Huang [mailto:jason.hu...@icare.com] Sent: 10 July 2013 15:50 To: solr-user@lucene.apache.org Subject: Commit different database rows to solr with same id value? Hello, I am trying to use Solr to store fields from two different database tables, where the primary keys are in the format of 1, 2, 3, In Java, we build different POJO classes for these two database tables: table1.java @SolrIndex(name=id) private String idTable1 table2.java @SolrIndex(name=id) private String idTable2 And later we add these fields defined in the two different types of tables and commit it to solrServer. Here is the scenario where I am having issues: (1) commit a row from table1 with primary key = 3, this generates a document in Solr (2) commit another row from table2 with the same value of primary key = 3, this overwrites the document generated in step (1). What we really want to achieve is to keep both rows in (1) and (2) because they are from different tables. I've read something from google search and it appears that we might be able to do it via keeping multiple cores in solr? Could anyone point at how to implement multiple core to achieve this? To be more specific, when I commit the row as a document, I don't have a place to pick a certain core and I am not sure if it makes any sense for me to specify a core when I commit the document since the layer I am working on should abstract it away from me. The second question is - if we don't want to do a multicore (since we can't easily search for related data between multiple cores), how can we resolve this issue so both rows from different database table which shares the same primary key still exist? We don't want to have to always change the primary key format to ensure a uniqueness of the primary key among all different types of database tables. thanks! Jason
Re: Commit different database rows to solr with same id value?
Thanks David. I am actually trying to commit the database row on the fly, not DIH. :) Anyway, if I understand you correctly, basically you are suggesting to modify the value of the primary key and pass the new value to id before committing to solr. This could probably be one solution. What if I want to commit the data from table2 to a new core? Anyone knows how I can do that? thanks, Jason On Wed, Jul 10, 2013 at 11:18 AM, David Quarterman da...@corexe.com wrote: Hi Jason, Assuming you're using DIH, why not build a new, unique id within the query to use as the 'doc_id' for SOLR? We do something like this in one of our collections. In MySQL, try this (don't know what it would be for any other db but there must be equivalents): select @rownum:=@rownum+1 rowid, t.* from (main select query) t, (select @rownum:=0) s Regards, DQ -Original Message- From: Jason Huang [mailto:jason.hu...@icare.com] Sent: 10 July 2013 15:50 To: solr-user@lucene.apache.org Subject: Commit different database rows to solr with same id value? Hello, I am trying to use Solr to store fields from two different database tables, where the primary keys are in the format of 1, 2, 3, In Java, we build different POJO classes for these two database tables: table1.java @SolrIndex(name=id) private String idTable1 table2.java @SolrIndex(name=id) private String idTable2 And later we add these fields defined in the two different types of tables and commit it to solrServer. Here is the scenario where I am having issues: (1) commit a row from table1 with primary key = 3, this generates a document in Solr (2) commit another row from table2 with the same value of primary key = 3, this overwrites the document generated in step (1). What we really want to achieve is to keep both rows in (1) and (2) because they are from different tables. I've read something from google search and it appears that we might be able to do it via keeping multiple cores in solr? Could anyone point at how to implement multiple core to achieve this? To be more specific, when I commit the row as a document, I don't have a place to pick a certain core and I am not sure if it makes any sense for me to specify a core when I commit the document since the layer I am working on should abstract it away from me. The second question is - if we don't want to do a multicore (since we can't easily search for related data between multiple cores), how can we resolve this issue so both rows from different database table which shares the same primary key still exist? We don't want to have to always change the primary key format to ensure a uniqueness of the primary key among all different types of database tables. thanks! Jason
