Re: Query does not work when changing param order
Sorry it took me so long to answer, but that issue turned into low
priority and I resume it today.
Adding a filter query for each tag worked great!
Thank you! =o)
Juan M.
On Thu, Jul 7, 2011 at 7:10 PM, Juan Grande wrote:
> Hi Juan!
>
> I think your problem is that in the second case the FieldQParserPlugin is
> building a phrase query for "mytag myothertag". I recommend you to split the
> filter in two different filters, one for each tag. If each tag is used in
> many different filters, and the combination of tags is rarely repeated, this
> will also result in a more efficient use of filterCache.
>
> Regards,
>
> *Juan*
>
>
>
> On Thu, Jul 7, 2011 at 12:07 PM, Juan Manuel Alvarez
> wrote:
>
>> Hi everyone!
>>
>> I would like to ask you a question about a problem I am facing with a
>> Solr query.
>>
>> I have a field "tags" of type "textgen" and some documents with the
>> values "myothertag,mytag".
>>
>> When I use the query:
>> /solr/select?sort=name_sort+asc&start=0&qf=tags&q.alt=*:*&fq={!field
>> q.op=AND f=tags}myothertag mytag&rows=60&defType=dismax
>>
>> everything works as expected, but if I change the order of the
>> parameters in the fq, like this
>> /solr/select?sort=name_sort+asc&start=0&qf=tags&q.alt=*:*&fq={!field
>> q.op=AND f=tags}mytag myothertag&rows=60&defType=dismax
>> I get no results.
>>
>> As far as I have seen, the "textgen" fieldshould tokenize the words in
>> the field, so if I use comma-separated values, like in my example,
>> both words are going to be indexed.
>>
>> Can anyone please point me in the right direction?
>>
>> Cheers!
>> Juan M.
>>
>
Query does not work when changing param order
Hi everyone!
I would like to ask you a question about a problem I am facing with a
Solr query.
I have a field "tags" of type "textgen" and some documents with the
values "myothertag,mytag".
When I use the query:
/solr/select?sort=name_sort+asc&start=0&qf=tags&q.alt=*:*&fq={!field
q.op=AND f=tags}myothertag mytag&rows=60&defType=dismax
everything works as expected, but if I change the order of the
parameters in the fq, like this
/solr/select?sort=name_sort+asc&start=0&qf=tags&q.alt=*:*&fq={!field
q.op=AND f=tags}mytag myothertag&rows=60&defType=dismax
I get no results.
As far as I have seen, the "textgen" fieldshould tokenize the words in
the field, so if I use comma-separated values, like in my example,
both words are going to be indexed.
Can anyone please point me in the right direction?
Cheers!
Juan M.
Re: Search in all the documents
Thanks!!! That did the trick! =o) On Fri, Mar 18, 2011 at 5:09 PM, Ahmet Arslan wrote: >> I am trying to search in all documents using the dismax >> parser. >> >> A sample query that works using the q parameter goes like >> this: >> select/?q=air&qf=description%20name&start=0&rows=60&sort=name+asc&fq=(projectId:1)&defType=dismax&fq=(type:3)&fq=(folder:0) >> >> But when I have nothing to put into the q paramenter, I >> can't make it >> work for all the indices. >> I have tried using q=* but nothing happens. >> >> Is there a way to search in all documents or it is a >> limitation? > > q.alt is used for that. &q.alt=*:* it is better to hard code this into > solrconfig.xml. If no q is present, then q.alt is used. > > > >
Search in all the documents
Hello everyone! I would like to ask you a question. I am trying to search in all documents using the dismax parser. A sample query that works using the q parameter goes like this: select/?q=air&qf=description%20name&start=0&rows=60&sort=name+asc&fq=(projectId:1)&defType=dismax&fq=(type:3)&fq=(folder:0) But when I have nothing to put into the q paramenter, I can't make it work for all the indices. I have tried using q=* but nothing happens. Is there a way to search in all documents or it is a limitation? Thanks in advance! Juan M.
Re: How to integrate Solr with C/CPP client
I use the POCO C++ library (http://pocoproject.org/) to send HTML requests to Solr and to parse the XML response. Cheers! Juan M. On Wed, Jan 5, 2011 at 1:28 AM, Gora Mohanty wrote: > On Wed, Jan 5, 2011 at 9:54 AM, prasad deshpande > wrote: >> It seems Solr/Lucene is completely developed in Java. >> I want to integrate Solr with C/CPP client. Is it possible? If yes, how? > [] > > Lucene has a C port called Lucy ( http://incubator.apache.org/lucy/ ) > > Solr can be accessed as a web service, so it is possible to use a > C/C++ client. > > Regards, > Gora >
Re: UPDATE query in deltaquery
I finally worked it out!
The setting that was giving me trouble was autocommit. I set it to
true and now the UPDATE queries are reflected in the database.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 3:19 PM, Juan Manuel Alvarez wrote:
> Hi Ephraim! Thanks again for taking the time to help me. Really appreciated
> =o)
>
> The UPDATE was before the SELECT, but putting it after leads to the
> same result, with or without the TRANSACTION_SERIALIZABLE.
>
> Cheers!
> Juan M.
>
> 2010/12/30 Ephraim Ofir :
>> Does your function get_deltaimport_items perform the update first and then
>> the select? Does it make a difference if you change the order? Did you try
>> omitting the TRANSACTION_SERIALIZABLE part?
>>
>> Ephraim Ofir
>>
>> -Original Message-
>> From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
>> Sent: Thursday, December 30, 2010 7:04 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: UPDATE query in deltaquery
>>
>> Hi Ephraim! Thanks for the answer!
>>
>> Actually the user has permissions to make UPDATE queries.
>>
>> I changed the configuration to
>> >
>> url="jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}"
>> user="${dataimporter.request.dbUser}"
>> password="${dataimporter.request.dbPassword}"
>> transactionIsolation="TRANSACTION_SERIALIZABLE"
>> readOnly="false"/>
>>
>> and I still get the same results.
>>
>> Cheers!
>> Juan M.
>>
>> On Thu, Dec 30, 2010 at 12:40 PM, Ephraim Ofir wrote:
>>> This may sound silly, but are you sure the user you're using has
>>> permissions to do the updates you want? Not sure about postgres but I
>>> think some jdbc's require that the connection be defined as rw, maybe
>>> you should try adding readOnly="false" to your jdbc definition.
>>>
>>> Ephraim Ofir
>>>
>>> -Original Message-
>>> From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
>>> Sent: Thursday, December 30, 2010 2:52 PM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: UPDATE query in deltaquery
>>>
>>> Hi Erick!
>>>
>>> Here is my DIH configuration:
>>>
>>>
>>> >>
>>> url="jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.req
>>> uest.dbPort}/${dataimporter.request.dbName}"
>>> user="${dataimporter.request.dbUser}"
>>> password="${dataimporter.request.dbPassword}" autoCommit="false"
>>> transactionIsolation="TRANSACTION_READ_UNCOMMITTED"
>>> holdability="CLOSE_CURSORS_AT_COMMIT"/>
>>>
>>> >> query=' . '
>>> deltaImportQuery=' . '
>>> deltaQuery=' . '
>>> >
>>>
>>>
>>>
>>>
>>> I have tried two options for the deltaQuery:
>>> UPDATE "Global"."Projects" SET "prj_lastSync" = now() WHERE "prj_id" =
>>> '2'; < Throws a null pointer exception as described in the
>>> previous email
>>>
>>> The second option is a DB function that I am calling this way:
>>> SELECT "get_deltaimport_items" AS "id" FROM
>>> project.get_deltaimport_items(2, 'project');
>>>
>>> The function inside executes the UPDATE query shown above and a SELECT
>>> query for the ids.
>>> The ids are returned ok, but the UPDATE has no effect on the database.
>>>
>>> Cheers!
>>> Juan M.
>>>
>>>
>>> On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson
>>> wrote:
>>>> Well, let's see the queries you're sending, and your DIH
>>> configuration.
>>>>
>>>> Otherwise, we're just guessing...
>>>>
>>>> Best
>>>> Erick
>>>>
>>>> On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez
>>> wrote:
>>>>
>>>>> Hi! I would like to ask you a question about using a deltaQuery in
>>> DIH.
>>>>> I am syncing with a PostgreSQL database.
>>>>>
>>>>> At first I was calling a function that made two queries: an UPDATE
>>> and a
>>>>> SELECT.
>>>>> The select result was properly returned, but the UPDATE query did not
>>>>> made any changes,
>>>>> so I tried calling the same function from a PostgreSQL client and
>>>>> everything went OK.
>>>>>
>>>>> So I tried calling a simple UPDATE query directly in the deltaQuery
>>>>> and I receive a
>>>>> NullPointerException that I traced to the line 251 of the
>>>>> JdbcDataSource.java
>>>>> colNames = readFieldNames(resultSet.getMetaData());
>>>>>
>>>>> The question is: is there a way I can make the update query work in
>>>>> the deltaQuery
>>>>> or am I doing something wrong?
>>>>>
>>>>> Happy new year
>>>>> Cheers!
>>>>> Juan M.
>>>>>
>>>>
>>>
>>
>
Re: UPDATE query in deltaquery
Hi Ephraim! Thanks again for taking the time to help me. Really appreciated =o)
The UPDATE was before the SELECT, but putting it after leads to the
same result, with or without the TRANSACTION_SERIALIZABLE.
Cheers!
Juan M.
2010/12/30 Ephraim Ofir :
> Does your function get_deltaimport_items perform the update first and then
> the select? Does it make a difference if you change the order? Did you try
> omitting the TRANSACTION_SERIALIZABLE part?
>
> Ephraim Ofir
>
> -Original Message-
> From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
> Sent: Thursday, December 30, 2010 7:04 PM
> To: solr-user@lucene.apache.org
> Subject: Re: UPDATE query in deltaquery
>
> Hi Ephraim! Thanks for the answer!
>
> Actually the user has permissions to make UPDATE queries.
>
> I changed the configuration to
>
> url="jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}"
> user="${dataimporter.request.dbUser}"
> password="${dataimporter.request.dbPassword}"
> transactionIsolation="TRANSACTION_SERIALIZABLE"
> readOnly="false"/>
>
> and I still get the same results.
>
> Cheers!
> Juan M.
>
> On Thu, Dec 30, 2010 at 12:40 PM, Ephraim Ofir wrote:
>> This may sound silly, but are you sure the user you're using has
>> permissions to do the updates you want? Not sure about postgres but I
>> think some jdbc's require that the connection be defined as rw, maybe
>> you should try adding readOnly="false" to your jdbc definition.
>>
>> Ephraim Ofir
>>
>> -Original Message-
>> From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
>> Sent: Thursday, December 30, 2010 2:52 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: UPDATE query in deltaquery
>>
>> Hi Erick!
>>
>> Here is my DIH configuration:
>>
>>
>> >
>> url="jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.req
>> uest.dbPort}/${dataimporter.request.dbName}"
>> user="${dataimporter.request.dbUser}"
>> password="${dataimporter.request.dbPassword}" autoCommit="false"
>> transactionIsolation="TRANSACTION_READ_UNCOMMITTED"
>> holdability="CLOSE_CURSORS_AT_COMMIT"/>
>>
>> > query=' . '
>> deltaImportQuery=' . '
>> deltaQuery=' . '
>> >
>>
>>
>>
>>
>> I have tried two options for the deltaQuery:
>> UPDATE "Global"."Projects" SET "prj_lastSync" = now() WHERE "prj_id" =
>> '2'; < Throws a null pointer exception as described in the
>> previous email
>>
>> The second option is a DB function that I am calling this way:
>> SELECT "get_deltaimport_items" AS "id" FROM
>> project.get_deltaimport_items(2, 'project');
>>
>> The function inside executes the UPDATE query shown above and a SELECT
>> query for the ids.
>> The ids are returned ok, but the UPDATE has no effect on the database.
>>
>> Cheers!
>> Juan M.
>>
>>
>> On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson
>> wrote:
>>> Well, let's see the queries you're sending, and your DIH
>> configuration.
>>>
>>> Otherwise, we're just guessing...
>>>
>>> Best
>>> Erick
>>>
>>> On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez
>> wrote:
>>>
>>>> Hi! I would like to ask you a question about using a deltaQuery in
>> DIH.
>>>> I am syncing with a PostgreSQL database.
>>>>
>>>> At first I was calling a function that made two queries: an UPDATE
>> and a
>>>> SELECT.
>>>> The select result was properly returned, but the UPDATE query did not
>>>> made any changes,
>>>> so I tried calling the same function from a PostgreSQL client and
>>>> everything went OK.
>>>>
>>>> So I tried calling a simple UPDATE query directly in the deltaQuery
>>>> and I receive a
>>>> NullPointerException that I traced to the line 251 of the
>>>> JdbcDataSource.java
>>>> colNames = readFieldNames(resultSet.getMetaData());
>>>>
>>>> The question is: is there a way I can make the update query work in
>>>> the deltaQuery
>>>> or am I doing something wrong?
>>>>
>>>> Happy new year
>>>> Cheers!
>>>> Juan M.
>>>>
>>>
>>
>
Re: UPDATE query in deltaquery
Hi Travis!
I am executing a function in de DB that has two queries: an UPDATE and
a SELECT, and I am getting the select results ok, but the update has
no effects, so it seems that that approach is not working.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 11:26 AM, Travis Low wrote:
> If you are getting a null pointer exception here:
>
> colNames = readFieldNames(resultSet.getMetaData());
>
> Then that implies the DIH code is written to expect a select statement. You
> might be able to fool it with some SQL injection:
>
> update blah set foo=bar where id=1234; select id from blah
>
> But if that doesn't work then you may be out of luck.
>
> cheers,
>
> Travis
>
> On Thu, Dec 30, 2010 at 8:26 AM, Juan Manuel Alvarez
> wrote:
>
>> Erick:
>>
>> Thanks for the quick response.
>>
>> I can't use the timestamp for doing DIH, so I need to use a custom
>> field that I need to update one for each delta-import, so that is why
>> I need to execute an UPDATE on the deltaQuery.
>>
>> Cheers!
>> Juan M.
>>
>> On Thu, Dec 30, 2010 at 10:07 AM, Erick Erickson
>> wrote:
>> > WARNING: DIH isn't my strong suit, I generally prefer doing things
>> > in SolrJ. Mostly I asked for clarification so someone #else# who
>> > actually knows DIH details could chime in...
>> >
>> > That said, I'm a bit confused. As I understand it, you shouldn't
>> > be UPDATEing anything in DIH, it's a select where documents
>> > then get added to Solr "by magic". Your post leads me to believe
>> > that you're trying to change the database via DIH, is that at
>> > all true?
>> >
>> > This is based in part on
>> > "The ids are returned ok, but the UPDATE has no effect on the database"
>> > Or do you mean "effect on the index"? If the latter, then the select
>> > would only have a chance of updating the IDs of the Solr documents...
>> >
>> > At least I think that's close to reality...
>> >
>> > Best
>> > Erick
>> >
>> > On Thu, Dec 30, 2010 at 7:52 AM, Juan Manuel Alvarez > >wrote:
>> >
>> >> Hi Erick!
>> >>
>> >> Here is my DIH configuration:
>> >>
>> >>
>> >> > >>
>> >>
>> url="jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}"
>> >> user="${dataimporter.request.dbUser}"
>> >> password="${dataimporter.request.dbPassword}" autoCommit="false"
>> >> transactionIsolation="TRANSACTION_READ_UNCOMMITTED"
>> >> holdability="CLOSE_CURSORS_AT_COMMIT"/>
>> >>
>> >> > >> query=' . '
>> >> deltaImportQuery=' . '
>> >> deltaQuery=' . '
>> >> >
>> >>
>> >>
>> >>
>> >>
>> >> I have tried two options for the deltaQuery:
>> >> UPDATE "Global"."Projects" SET "prj_lastSync" = now() WHERE "prj_id" =
>> >> '2'; < Throws a null pointer exception as described in the
>> >> previous email
>> >>
>> >> The second option is a DB function that I am calling this way:
>> >> SELECT "get_deltaimport_items" AS "id" FROM
>> >> project.get_deltaimport_items(2, 'project');
>> >>
>> >> The function inside executes the UPDATE query shown above and a SELECT
>> >> query for the ids.
>> >> The ids are returned ok, but the UPDATE has no effect on the database.
>> >>
>> >> Cheers!
>> >> Juan M.
>> >>
>> >>
>> >> On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson <
>> erickerick...@gmail.com>
>> >> wrote:
>> >> > Well, let's see the queries you're sending, and your DIH
>> configuration.
>> >> >
>> >> > Otherwise, we're just guessing...
>> >> >
>> >> > Best
>> >> > Erick
>> >> >
>> >> > On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez <
>> naici...@gmail.com
>> >> >wrote:
>> >> >
>> >> >> Hi! I would like to ask you a question about using a deltaQuery in
>> DIH.
Re: UPDATE query in deltaquery
Hi Ephraim! Thanks for the answer!
Actually the user has permissions to make UPDATE queries.
I changed the configuration to
and I still get the same results.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 12:40 PM, Ephraim Ofir wrote:
> This may sound silly, but are you sure the user you're using has
> permissions to do the updates you want? Not sure about postgres but I
> think some jdbc's require that the connection be defined as rw, maybe
> you should try adding readOnly="false" to your jdbc definition.
>
> Ephraim Ofir
>
> -----Original Message-
> From: Juan Manuel Alvarez [mailto:naici...@gmail.com]
> Sent: Thursday, December 30, 2010 2:52 PM
> To: solr-user@lucene.apache.org
> Subject: Re: UPDATE query in deltaquery
>
> Hi Erick!
>
> Here is my DIH configuration:
>
>
>
> url="jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.req
> uest.dbPort}/${dataimporter.request.dbName}"
> user="${dataimporter.request.dbUser}"
> password="${dataimporter.request.dbPassword}" autoCommit="false"
> transactionIsolation="TRANSACTION_READ_UNCOMMITTED"
> holdability="CLOSE_CURSORS_AT_COMMIT"/>
>
> query=' . '
> deltaImportQuery=' . '
> deltaQuery=' . '
> >
>
>
>
>
> I have tried two options for the deltaQuery:
> UPDATE "Global"."Projects" SET "prj_lastSync" = now() WHERE "prj_id" =
> '2'; < Throws a null pointer exception as described in the
> previous email
>
> The second option is a DB function that I am calling this way:
> SELECT "get_deltaimport_items" AS "id" FROM
> project.get_deltaimport_items(2, 'project');
>
> The function inside executes the UPDATE query shown above and a SELECT
> query for the ids.
> The ids are returned ok, but the UPDATE has no effect on the database.
>
> Cheers!
> Juan M.
>
>
> On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson
> wrote:
>> Well, let's see the queries you're sending, and your DIH
> configuration.
>>
>> Otherwise, we're just guessing...
>>
>> Best
>> Erick
>>
>> On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez
> wrote:
>>
>>> Hi! I would like to ask you a question about using a deltaQuery in
> DIH.
>>> I am syncing with a PostgreSQL database.
>>>
>>> At first I was calling a function that made two queries: an UPDATE
> and a
>>> SELECT.
>>> The select result was properly returned, but the UPDATE query did not
>>> made any changes,
>>> so I tried calling the same function from a PostgreSQL client and
>>> everything went OK.
>>>
>>> So I tried calling a simple UPDATE query directly in the deltaQuery
>>> and I receive a
>>> NullPointerException that I traced to the line 251 of the
>>> JdbcDataSource.java
>>> colNames = readFieldNames(resultSet.getMetaData());
>>>
>>> The question is: is there a way I can make the update query work in
>>> the deltaQuery
>>> or am I doing something wrong?
>>>
>>> Happy new year
>>> Cheers!
>>> Juan M.
>>>
>>
>
Re: UPDATE query in deltaquery
Erick:
Thanks for the quick response.
I can't use the timestamp for doing DIH, so I need to use a custom
field that I need to update one for each delta-import, so that is why
I need to execute an UPDATE on the deltaQuery.
Cheers!
Juan M.
On Thu, Dec 30, 2010 at 10:07 AM, Erick Erickson
wrote:
> WARNING: DIH isn't my strong suit, I generally prefer doing things
> in SolrJ. Mostly I asked for clarification so someone #else# who
> actually knows DIH details could chime in...
>
> That said, I'm a bit confused. As I understand it, you shouldn't
> be UPDATEing anything in DIH, it's a select where documents
> then get added to Solr "by magic". Your post leads me to believe
> that you're trying to change the database via DIH, is that at
> all true?
>
> This is based in part on
> "The ids are returned ok, but the UPDATE has no effect on the database"
> Or do you mean "effect on the index"? If the latter, then the select
> would only have a chance of updating the IDs of the Solr documents...
>
> At least I think that's close to reality...
>
> Best
> Erick
>
> On Thu, Dec 30, 2010 at 7:52 AM, Juan Manuel Alvarez
> wrote:
>
>> Hi Erick!
>>
>> Here is my DIH configuration:
>>
>>
>> >
>> url="jdbc:postgresql://${dataimporter.request.dbHost}:${dataimporter.request.dbPort}/${dataimporter.request.dbName}"
>> user="${dataimporter.request.dbUser}"
>> password="${dataimporter.request.dbPassword}" autoCommit="false"
>> transactionIsolation="TRANSACTION_READ_UNCOMMITTED"
>> holdability="CLOSE_CURSORS_AT_COMMIT"/>
>>
>> > query=' . '
>> deltaImportQuery=' . '
>> deltaQuery=' . '
>> >
>>
>>
>>
>>
>> I have tried two options for the deltaQuery:
>> UPDATE "Global"."Projects" SET "prj_lastSync" = now() WHERE "prj_id" =
>> '2'; < Throws a null pointer exception as described in the
>> previous email
>>
>> The second option is a DB function that I am calling this way:
>> SELECT "get_deltaimport_items" AS "id" FROM
>> project.get_deltaimport_items(2, 'project');
>>
>> The function inside executes the UPDATE query shown above and a SELECT
>> query for the ids.
>> The ids are returned ok, but the UPDATE has no effect on the database.
>>
>> Cheers!
>> Juan M.
>>
>>
>> On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson
>> wrote:
>> > Well, let's see the queries you're sending, and your DIH configuration.
>> >
>> > Otherwise, we're just guessing...
>> >
>> > Best
>> > Erick
>> >
>> > On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez > >wrote:
>> >
>> >> Hi! I would like to ask you a question about using a deltaQuery in DIH.
>> >> I am syncing with a PostgreSQL database.
>> >>
>> >> At first I was calling a function that made two queries: an UPDATE and a
>> >> SELECT.
>> >> The select result was properly returned, but the UPDATE query did not
>> >> made any changes,
>> >> so I tried calling the same function from a PostgreSQL client and
>> >> everything went OK.
>> >>
>> >> So I tried calling a simple UPDATE query directly in the deltaQuery
>> >> and I receive a
>> >> NullPointerException that I traced to the line 251 of the
>> >> JdbcDataSource.java
>> >> colNames = readFieldNames(resultSet.getMetaData());
>> >>
>> >> The question is: is there a way I can make the update query work in
>> >> the deltaQuery
>> >> or am I doing something wrong?
>> >>
>> >> Happy new year
>> >> Cheers!
>> >> Juan M.
>> >>
>> >
>>
>
Re: UPDATE query in deltaquery
Hi Erick! Here is my DIH configuration: I have tried two options for the deltaQuery: UPDATE "Global"."Projects" SET "prj_lastSync" = now() WHERE "prj_id" = '2'; < Throws a null pointer exception as described in the previous email The second option is a DB function that I am calling this way: SELECT "get_deltaimport_items" AS "id" FROM project.get_deltaimport_items(2, 'project'); The function inside executes the UPDATE query shown above and a SELECT query for the ids. The ids are returned ok, but the UPDATE has no effect on the database. Cheers! Juan M. On Thu, Dec 30, 2010 at 1:32 AM, Erick Erickson wrote: > Well, let's see the queries you're sending, and your DIH configuration. > > Otherwise, we're just guessing... > > Best > Erick > > On Wed, Dec 29, 2010 at 9:58 PM, Juan Manuel Alvarez > wrote: > >> Hi! I would like to ask you a question about using a deltaQuery in DIH. >> I am syncing with a PostgreSQL database. >> >> At first I was calling a function that made two queries: an UPDATE and a >> SELECT. >> The select result was properly returned, but the UPDATE query did not >> made any changes, >> so I tried calling the same function from a PostgreSQL client and >> everything went OK. >> >> So I tried calling a simple UPDATE query directly in the deltaQuery >> and I receive a >> NullPointerException that I traced to the line 251 of the >> JdbcDataSource.java >> colNames = readFieldNames(resultSet.getMetaData()); >> >> The question is: is there a way I can make the update query work in >> the deltaQuery >> or am I doing something wrong? >> >> Happy new year >> Cheers! >> Juan M. >> >
UPDATE query in deltaquery
Hi! I would like to ask you a question about using a deltaQuery in DIH. I am syncing with a PostgreSQL database. At first I was calling a function that made two queries: an UPDATE and a SELECT. The select result was properly returned, but the UPDATE query did not made any changes, so I tried calling the same function from a PostgreSQL client and everything went OK. So I tried calling a simple UPDATE query directly in the deltaQuery and I receive a NullPointerException that I traced to the line 251 of the JdbcDataSource.java colNames = readFieldNames(resultSet.getMetaData()); The question is: is there a way I can make the update query work in the deltaQuery or am I doing something wrong? Happy new year Cheers! Juan M.
Re: Syncing 'delta-import' with 'select' query
Oops! That seems to be the problem, since I am using 1.4.
Thanks!
Juan M.
On Tue, Dec 14, 2010 at 8:40 PM, Alexey Serba wrote:
> What Solr version do you use?
>
> It seems that sync flag has been added to 3.1 and 4.0 (trunk) branches
> and not to 1.4
> https://issues.apache.org/jira/browse/SOLR-1721
>
> On Wed, Dec 8, 2010 at 11:21 PM, Juan Manuel Alvarez
> wrote:
>> Hello everyone!
>> I have been doing some tests, but it seems I can't make the
>> synchronize flag work.
>>
>> I have made two tests:
>> 1) DIH with commit=false
>> 2) DIH with commit=false + commit via Solr XML update protocol
>>
>> And here are the log results:
>> For (1) the command is
>> "/solr/dataimport?command=delta-import&commit=false&synchronous=true"
>> and the first part of the output is:
>>
>> Dec 8, 2010 4:42:51 PM org.apache.solr.core.SolrCore execute
>> INFO: [] webapp=/solr path=/dataimport params={command=status} status=0
>> QTime=0
>> Dec 8, 2010 4:42:51 PM org.apache.solr.core.SolrCore execute
>> INFO: [] webapp=/solr path=/dataimport
>> params={schema=testproject&dbHost=127.0.0.1&dbPassword=fuz10n!&dbName=fzm&commit=false&dbUser=fzm&command=delta-import&projectId=1&synchronous=true&dbPort=5432}
>> status=0 QTime=4
>> Dec 8, 2010 4:42:51 PM org.apache.solr.handler.dataimport.DataImporter
>> doDeltaImport
>> INFO: Starting Delta Import
>> Dec 8, 2010 4:42:51 PM org.apache.solr.handler.dataimport.SolrWriter
>> readIndexerProperties
>> INFO: Read dataimport.properties
>> Dec 8, 2010 4:42:51 PM org.apache.solr.handler.dataimport.DocBuilder doDelta
>> INFO: Starting delta collection.
>> Dec 8, 2010 4:42:51 PM org.apache.solr.handler.dataimport.DocBuilder
>> collectDelta
>>
>>
>> For (2) the commands are
>> "/solr/dataimport?command=delta-import&commit=false&synchronous=true"
>> and "/solr/update?commit=true&waitFlush=true&waitSearcher=true" and
>> the first part of the output is:
>>
>> Dec 8, 2010 4:22:50 PM org.apache.solr.core.SolrCore execute
>> INFO: [] webapp=/solr path=/dataimport params={command=status} status=0
>> QTime=0
>> Dec 8, 2010 4:22:50 PM org.apache.solr.core.SolrCore execute
>> INFO: [] webapp=/solr path=/dataimport
>> params={schema=testproject&dbHost=127.0.0.1&dbPassword=fuz10n!&dbName=fzm&commit=false&dbUser=fzm&command=delta-import&projectId=1&synchronous=true&dbPort=5432}
>> status=0 QTime=1
>> Dec 8, 2010 4:22:50 PM org.apache.solr.core.SolrCore execute
>> INFO: [] webapp=/solr path=/dataimport params={command=status} status=0
>> QTime=0
>> Dec 8, 2010 4:22:50 PM org.apache.solr.handler.dataimport.DataImporter
>> doDeltaImport
>> INFO: Starting Delta Import
>> Dec 8, 2010 4:22:50 PM org.apache.solr.handler.dataimport.SolrWriter
>> readIndexerProperties
>> INFO: Read dataimport.properties
>> Dec 8, 2010 4:22:50 PM org.apache.solr.update.DirectUpdateHandler2 commit
>> INFO: start
>> commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
>>
>> In (2) it seems like the commit is being fired before the delta-update
>> finishes.
>>
>> Am I using the "synchronous" flag right?
>>
>> Thanks in advance!
>> Juan M.
>>
>> On Mon, Dec 6, 2010 at 6:46 PM, Juan Manuel Alvarez
>> wrote:
>>> Thanks for all the help! It is really appreciated.
>>>
>>> For now, I can afford the parallel requests problem, but when I put
>>> synchronous=true in the delta import, the call still returns with
>>> outdated items.
>>> Examining the log, it seems that the commit operation is being
>>> executed after the operation returns, even when I am using
>>> commit=true.
>>> Is it possible to also execute the commit synchronously?
>>>
>>> Cheers!
>>> Juan M.
>>>
>>> On Mon, Dec 6, 2010 at 4:29 PM, Alexey Serba wrote:
>>>>> When you say "two parallel requests from two users to single DIH
>>>>> request handler", what do you mean by "request handler"?
>>>> I mean DIH.
>>>>
>>>>> Are you
>>>>> refering to the HTTP request? Would that mean that if I make the
>>>>> request from different HTTP sessions it would work?
>>>> No.
>>>>
>>>> It means that when you have two users that simultaneously changed two
>>>> objects in the UI then you have two HTTP requests to DIH to pull
>>>> changes from the db into Solr index. If the second request comes when
>>>> the first is not fully processed then the second request will be
>>>> rejected. As a result your index would be outdated (w/o the latest
>>>> update) until the next update.
>>>>
>>>
>>
>
Re: Concurrent DIH calls
Thanks for the answer Barani! I was doing the same thing (queuing requests and querying solr status), but I was hoping some flag/configuration would do the trick. I will continue with that approach then! =o) Thanks! Juan M. On Sat, Dec 11, 2010 at 3:50 AM, bbarani wrote: > > Hi, > > As far as I know there is no queuing mechanism in SOLR for concurrent > indexing request. It would simple ignore the concurrent request (first come > first serve basis).. Solr experts, please correct me if I am wrong.. > > To achieve concurrency, we have implemented a queue using JMS and we send > the data one by one for indexing (for performing push indexing / real time > indexing).. > > We have also written a simple java program with SOLRj which will check if > the status is idle or busy before it starts indexing next batch (This is for > batch indexing program).. > > I would say the same thing applies for commit also.. As far as I know there > is not inbuilt queuing system in SOLR for indexing. > > Thanks, > Barani > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Concurrent-DIH-calls-tp2059517p2067937.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Concurrent DIH calls
Hello!!! I am working with Solr on my first project and I am really happy so far, both with the product and with the community. I am having some doubts about how DIH works. How does DIH handles concurrent requests from different users? Does it queue them? Or if an import is in progress it rejects new requests? I would also like to know if the "commit=true" flag. Does it enqueue a commit call in Solr? Or if another commit is in progress it blocks until is finished? Cheers! Juan M.
Re: Syncing 'delta-import' with 'select' query
Hello everyone!
I have been doing some tests, but it seems I can't make the
synchronize flag work.
I have made two tests:
1) DIH with commit=false
2) DIH with commit=false + commit via Solr XML update protocol
And here are the log results:
For (1) the command is
"/solr/dataimport?command=delta-import&commit=false&synchronous=true"
and the first part of the output is:
Dec 8, 2010 4:42:51 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/dataimport params={command=status} status=0 QTime=0
Dec 8, 2010 4:42:51 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/dataimport
params={schema=testproject&dbHost=127.0.0.1&dbPassword=fuz10n!&dbName=fzm&commit=false&dbUser=fzm&command=delta-import&projectId=1&synchronous=true&dbPort=5432}
status=0 QTime=4
Dec 8, 2010 4:42:51 PM org.apache.solr.handler.dataimport.DataImporter
doDeltaImport
INFO: Starting Delta Import
Dec 8, 2010 4:42:51 PM org.apache.solr.handler.dataimport.SolrWriter
readIndexerProperties
INFO: Read dataimport.properties
Dec 8, 2010 4:42:51 PM org.apache.solr.handler.dataimport.DocBuilder doDelta
INFO: Starting delta collection.
Dec 8, 2010 4:42:51 PM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
For (2) the commands are
"/solr/dataimport?command=delta-import&commit=false&synchronous=true"
and "/solr/update?commit=true&waitFlush=true&waitSearcher=true" and
the first part of the output is:
Dec 8, 2010 4:22:50 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/dataimport params={command=status} status=0 QTime=0
Dec 8, 2010 4:22:50 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/dataimport
params={schema=testproject&dbHost=127.0.0.1&dbPassword=fuz10n!&dbName=fzm&commit=false&dbUser=fzm&command=delta-import&projectId=1&synchronous=true&dbPort=5432}
status=0 QTime=1
Dec 8, 2010 4:22:50 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/dataimport params={command=status} status=0 QTime=0
Dec 8, 2010 4:22:50 PM org.apache.solr.handler.dataimport.DataImporter
doDeltaImport
INFO: Starting Delta Import
Dec 8, 2010 4:22:50 PM org.apache.solr.handler.dataimport.SolrWriter
readIndexerProperties
INFO: Read dataimport.properties
Dec 8, 2010 4:22:50 PM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
In (2) it seems like the commit is being fired before the delta-update finishes.
Am I using the "synchronous" flag right?
Thanks in advance!
Juan M.
On Mon, Dec 6, 2010 at 6:46 PM, Juan Manuel Alvarez wrote:
> Thanks for all the help! It is really appreciated.
>
> For now, I can afford the parallel requests problem, but when I put
> synchronous=true in the delta import, the call still returns with
> outdated items.
> Examining the log, it seems that the commit operation is being
> executed after the operation returns, even when I am using
> commit=true.
> Is it possible to also execute the commit synchronously?
>
> Cheers!
> Juan M.
>
> On Mon, Dec 6, 2010 at 4:29 PM, Alexey Serba wrote:
>>> When you say "two parallel requests from two users to single DIH
>>> request handler", what do you mean by "request handler"?
>> I mean DIH.
>>
>>> Are you
>>> refering to the HTTP request? Would that mean that if I make the
>>> request from different HTTP sessions it would work?
>> No.
>>
>> It means that when you have two users that simultaneously changed two
>> objects in the UI then you have two HTTP requests to DIH to pull
>> changes from the db into Solr index. If the second request comes when
>> the first is not fully processed then the second request will be
>> rejected. As a result your index would be outdated (w/o the latest
>> update) until the next update.
>>
>
Re: Syncing 'delta-import' with 'select' query
Thanks for all the help! It is really appreciated. For now, I can afford the parallel requests problem, but when I put synchronous=true in the delta import, the call still returns with outdated items. Examining the log, it seems that the commit operation is being executed after the operation returns, even when I am using commit=true. Is it possible to also execute the commit synchronously? Cheers! Juan M. On Mon, Dec 6, 2010 at 4:29 PM, Alexey Serba wrote: >> When you say "two parallel requests from two users to single DIH >> request handler", what do you mean by "request handler"? > I mean DIH. > >> Are you >> refering to the HTTP request? Would that mean that if I make the >> request from different HTTP sessions it would work? > No. > > It means that when you have two users that simultaneously changed two > objects in the UI then you have two HTTP requests to DIH to pull > changes from the db into Solr index. If the second request comes when > the first is not fully processed then the second request will be > rejected. As a result your index would be outdated (w/o the latest > update) until the next update. >
Re: Syncing 'delta-import' with 'select' query
Alex: Thanks for the quick reply. When you say "two parallel requests from two users to single DIH request handler", what do you mean by "request handler"? Are you refering to the HTTP request? Would that mean that if I make the request from different HTTP sessions it would work? Cheers! Juan M. On Mon, Dec 6, 2010 at 1:12 PM, Alexey Serba wrote: > Hey Juan, > > It seems that DataImportHandler is not a right tool for your scenario > and you'd better use Solr XML update protocol. > * http://wiki.apache.org/solr/UpdateXmlMessages > > You still can work around your outdated GUI view problem with calling > DIH synchronously, by adding synchronous=true to your request. But it > won't solve the problem with two parallel requests from two users to > single DIH request handler, because DIH doesn't support that, and if > previous request is still running it bounces the second request. > > HTH, > Alex > > > > On Fri, Dec 3, 2010 at 10:33 PM, Juan Manuel Alvarez > wrote: >> Hello everyone! I would like to ask you a question about DIH. >> >> I am using a database and DIH to sync against Solr, and a GUI to >> display and operate on the items retrieved from Solr. >> When I change the state of an item through the GUI, the following happens: >> a. The item is updated in the DB. >> b. A delta-import command is fired to sync the DB with Solr. >> c. The GUI is refreshed by making a query to Solr. >> >> My problem comes between (b) and (c). The delta-import operation is >> executed in a new thread, so my call returns immediately, refreshing >> the GUI before the Solr index is updated causing the item state in the >> GUI to be outdated. >> >> I had two ideas so far: >> 1. Querying the status of the DIH after the delta-import operation and >> do not return until it is "idle". The problem I see with this is that >> if other users execute delta-imports, the status will be "busy" until >> all operations are finished. >> 2. Use Zoie. The first problem is that configuring it is not as >> straightforward as it seems, so I don't want to spend more time trying >> it until I am sure that this will solve my issue. On the other hand, I >> think that I may suffer the same problem since the delta-import is >> still firing in another thread, so I can't be sure it will be called >> fast enough. >> >> Am I pointing on the right direction or is there another way to >> achieve my goal? >> >> Thanks in advance! >> Juan M. >> >
Syncing 'delta-import' with 'select' query
Hello everyone! I would like to ask you a question about DIH. I am using a database and DIH to sync against Solr, and a GUI to display and operate on the items retrieved from Solr. When I change the state of an item through the GUI, the following happens: a. The item is updated in the DB. b. A delta-import command is fired to sync the DB with Solr. c. The GUI is refreshed by making a query to Solr. My problem comes between (b) and (c). The delta-import operation is executed in a new thread, so my call returns immediately, refreshing the GUI before the Solr index is updated causing the item state in the GUI to be outdated. I had two ideas so far: 1. Querying the status of the DIH after the delta-import operation and do not return until it is "idle". The problem I see with this is that if other users execute delta-imports, the status will be "busy" until all operations are finished. 2. Use Zoie. The first problem is that configuring it is not as straightforward as it seems, so I don't want to spend more time trying it until I am sure that this will solve my issue. On the other hand, I think that I may suffer the same problem since the delta-import is still firing in another thread, so I can't be sure it will be called fast enough. Am I pointing on the right direction or is there another way to achieve my goal? Thanks in advance! Juan M.
Re: Updating last_modified field when using DIH
Stephan, Ephraim. Thanks for the answers!!!
I am finding Solr to be a useful product, but definitely the community
is what makes it a great product!
So far everyone has been very helpful. Thanks!
Cheers!
Juan M.
On Wed, Nov 3, 2010 at 9:13 AM, Ephraim Ofir wrote:
> Also, your deltaImportQuery should be:
> deltaImportQuery='SELECT * FROM "Entities" WHERE
> "ent_id"=${dataimporter.delta.id}"'
>
> Otherwise you're just importing the ids and not the rest of the data.
>
> If performance is important to you, you might also want to check out
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201008.mbox/%3
> c9f8b39cb3b7c6d4594293ea29ccf438b01702...@icq-mail.icq.il.office.aol.com
> %3E
>
> Ephraim Ofir
>
>
> -Original Message-
> From: Stefan Matheis [mailto:matheis.ste...@googlemail.com]
> Sent: Wednesday, November 03, 2010 12:58 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Updating last_modified field when using DIH
>
> Juan,
>
> that's correct .. solr will not touch your database, that's part of your
> application-code. solr uses an updated timestamp (which is available
> through dataimporter.last_index_time).
>
> so, image the following situation, solr import runs every 10 minutes ..
> last
> run at 11:00, your entity gets updated at 11:03, next solr-run at 11:10
> will
> detect this as changed, import the entity and run again at 11:20 ..
> then, no
> entity will match the delta-query because solr will ask for a
> modification_date > 11:10 (last solr-run at this time).
>
> you'll only need to update the last_modified field (in your application)
> when the entity is changed and you want solr to (re-)index your data.
>
> HTH,
> Stefan
>
> On Tue, Nov 2, 2010 at 7:35 PM, Juan Manuel Alvarez
> wrote:
>
>> Hello everyone!
>>
>> I would like to ask you a question about DIH and delta import.
>>
>> I am trying to sync Solr with a PostgreSQL database and I have a field
>> "ent_lastModified" of type "timestamp without timezone".
>>
>> Here is my xml file:
>>
>>
>> > url="jdbc:postgresql://host" user="XXX" password="XXX" readOnly="true"
>> autoCommit="false"
>> transactionIsolation="TRANSACTION_READ_COMMITTED"
>> holdability="CLOSE_CURSORS_AT_COMMIT"/>
>>
>> > query=' SELECT * FROM Entities'
>> deltaImportQuery='SELECT "ent_id" AS "id" FROM
>> "Entities" WHERE "ent_id"=${dataimporter.delta.id}"'
>> deltaQuery=' SELECT "ent_id" AS "id" FROM "Entities" WHERE
>> "ent_lastModified" > '${dataimporter.last_index_time}''
>> >
>>
>>
>>
>>
>> Full-import works fine, but when I run a delta-import the
>> "ent_lastModified" field, I get the corresponding records, but the
>> "ent_lastModified" stays the same, so if I make another delta-import,
>> the same records are retreived.
>>
>> I have read all the documentation at
>> http://wiki.apache.org/solr/DataImportHandler but I could not find an
>> update query for the "last_modified" field and Solr does not seem to
>> do this automatically.
>> I have also tried to name the field "last_modified" as in the example,
>> but its value keeps unchanged after a delta-import.
>>
>> Can anyone point me in the right direction?
>>
>> Thanks in advance!
>> Juan M.
>>
>
Updating last_modified field when using DIH
Hello everyone! I would like to ask you a question about DIH and delta import. I am trying to sync Solr with a PostgreSQL database and I have a field "ent_lastModified" of type "timestamp without timezone". Here is my xml file: Full-import works fine, but when I run a delta-import the "ent_lastModified" field, I get the corresponding records, but the "ent_lastModified" stays the same, so if I make another delta-import, the same records are retreived. I have read all the documentation at http://wiki.apache.org/solr/DataImportHandler but I could not find an update query for the "last_modified" field and Solr does not seem to do this automatically. I have also tried to name the field "last_modified" as in the example, but its value keeps unchanged after a delta-import. Can anyone point me in the right direction? Thanks in advance! Juan M.
Re: Using different schemas when syncing with PostgreSQL and DIH
Thank you Shawn! That was exactly what I was looking for! =o)
On Fri, Oct 22, 2010 at 4:29 PM, Shawn Heisey wrote:
> On 10/22/2010 10:06 AM, Juan Manuel Alvarez wrote:
>>
>> My question is:
>> Every time I do an import operation (delta or full) with DIH, I only
>> need to sync the index with one schema only, so... is there a way to
>> pass a custom parameter with the schema name to DIH so I can build the
>> query with the corresponding schema name?
>
> Yes, there is. Below is the latest version of my dih config used with a
> MySQL database. I've got almost everything in the SELECT statement
> specified by the input URL, which gets built using the following template:
>
> http://HOST:PORT/solr/CORE/dataimport?command=COMMAND&dbHost=DBHOST&dbSchema=DBSCHEMA&dataTable=DATATABLE&sgTable=SGTABLE&numShards=NUMSHARDS&modVal=MODVAL&minDid=MINDID&maxDid=MAXDID
>
>
> driver="com.mysql.jdbc.Driver"
> encoding="UTF-8"
>
> url="jdbc:mysql://${dataimporter.request.dbServer}:3306/${dataimporter.request.dbSchema}?zeroDateTimeBehavior=convertToNull"
> batchSize="-1"
> user="removed"
> password="removed"/>
>
> query="
> SELECT d.*,FROM_UNIXTIME(d.post_date) AS pd,
> s.search_group_map AS sg
> FROM ${dataimporter.request.dataTable} d
> LEFT JOIN ${dataimporter.request.sgTable} s
> ON d.feature=s.featurecode
> WHERE did > ${dataimporter.request.minDid}
> AND did <= ${dataimporter.request.maxDid}
> AND (did % ${dataimporter.request.numShards})
> IN (${dataimporter.request.modVal})
> GROUP BY d.did"
> deltaImportQuery="
> SELECT d.*,FROM_UNIXTIME(d.post_date) AS pd,
> s.search_group_map AS sg
> FROM ${dataimporter.request.dataTable} d
> LEFT JOIN ${dataimporter.request.sgTable} s
> ON d.feature=s.featurecode
> WHERE did > ${dataimporter.request.minDid}
> AND did <= ${dataimporter.request.maxDid}
> AND (did % ${dataimporter.request.numShards})
> IN (${dataimporter.request.modVal})
> GROUP BY d.did"
> deltaQuery="SELECT MAX(d.did) FROM ${dataimporter.request.dataTable} d"
>>
>
>
>
>
>
>
>
Using different schemas when syncing with PostgreSQL and DIH
Hello everyone! I am using Solr synced with a PostgreSQL database using DIH and I am facing an issue. The thing is that I use one Solr server and different Postgre schemas in the same database, with the same tables inside each one, so the following queries: SELECT * FROM "schema1"."Objects"; and SELECT * FROM "schema2"."Objects"; are both valid. The schemas are completely dynamic, so I can't do anything manually each time I add a new schema. In the DIH id field I am using a combination of the schema name and PK of the Objects table, to avoid duplicates. My question is: Every time I do an import operation (delta or full) with DIH, I only need to sync the index with one schema only, so... is there a way to pass a custom parameter with the schema name to DIH so I can build the query with the corresponding schema name? Thank you very much! Juan M.
Re: Synchronizing Solr with a PostgreDB
Thanks for the quick response! =o) We will go with that approach. On Thu, Oct 14, 2010 at 7:19 PM, Allistair Crossley wrote: > i would not cross-reference solr results with your database to merge unless > you want to spank your database. nor would i load solr with all your data. > what i have found is that the search results page is generally a small subset > of data relating to the fuller document/result. therefore i store only the > data required to present the search results wholly from solr. the user can > choose to click into a specific result which then uses just the database to > present it. > > use data import handler - define an xml config to import as many entities > into your document as you need and map columns to fields in schema.xml. use > the Wiki page on DIH - it's all there, as well as example config in the solr > distro. > > allistair > > On Oct 14, 2010, at 6:13 PM, Juan Manuel Alvarez wrote: > >> Hello everyone! I am new to Solr and Lucene and I would like to ask >> you a couple of questions. >> >> I am working on an existing system that has the data saved in a >> Postgre DB and now I am trying to integrate Solr to use full-text >> search and faceted search, but I am having a couple of doubts about >> it. >> >> 1) I see two ways of storing the data and make the search: >> - Duplicate all the DB data in Solr, so complete results are returned >> from a search query, or... >> - Put in Solr just the data that I need to search and, after finding >> the elements with a Solr query, use the result to make a more specific >> query to the DB. >> >> Which is the way this is normally done? >> >> 2) How do I synchronize Solr and Postgre? Do I have to use the >> DataImportHandler or when I do the INSERT command into Postgre, I have >> to execute a command into Solr? >> >> Thanks for your time! >> >> Cheers! >> Juan M. > >
Synchronizing Solr with a PostgreDB
Hello everyone! I am new to Solr and Lucene and I would like to ask you a couple of questions. I am working on an existing system that has the data saved in a Postgre DB and now I am trying to integrate Solr to use full-text search and faceted search, but I am having a couple of doubts about it. 1) I see two ways of storing the data and make the search: - Duplicate all the DB data in Solr, so complete results are returned from a search query, or... - Put in Solr just the data that I need to search and, after finding the elements with a Solr query, use the result to make a more specific query to the DB. Which is the way this is normally done? 2) How do I synchronize Solr and Postgre? Do I have to use the DataImportHandler or when I do the INSERT command into Postgre, I have to execute a command into Solr? Thanks for your time! Cheers! Juan M.
