Search for partial name in Solr 4.x
I was wondering if there is a way to search on partial names? Ex; Field is a string and stores values like titles of a book; When searching part of the title may be supplied; How do I resolve this? Please let me know Thanks -PeriS *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Turn off suggester
Is there a way to turn off the solr suggester? I have about 30M records and when tomcat starts up, it takes a long time (~10 minutes) for the suggester to decompress the data or its doing soothing as it hangs on SolrSuggester.build(); Any ideas please? Thanks -Peri *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Re: Help needed in Indexing and Search on xml content
Hi Sangeetha, If you can tell me a little bit more about your setup, I can try and help. If you are on skype, that would be the easiest. Thanks -Peri On Sep 25, 2014, at 3:50 AM, sangeetha.subraman...@gtnexus.com wrote: Hi Team, I am a newbie to SOLR. I have got search fields stored in a xml file which is stored in MSSQL. I want to index on the content of the xml file in SOLR. We need to provide search based on the fields present in the XML file. The reason why we are storing the input details as XML file is , the users will be able to add custom input fields on their own with values. Storing these custom fields as columns in MSSQL seems to be not an optimal solution. So we thought of putting it in XML file and store that file in RDBMS. But I am not sure on how we can index the content of the file to make search better. I believe this can be done by ExtractingRequestHandler. Could someone help me on how we can implement this/ direct me to some pages which could be of help to me ? Thanks Sangeetha --- This message has been scanned for viruses and dangerous content by HTC E-Mail Virus Protection Service. *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Re: DIH Issue
db-import.xml:
entity name=doctor pk= person_id transformer=“Some transformer
query=SELECT p.person_id, p.details, p.authz_id,
p.last_modified, a.address_info from person p, address a where p.person_id =
a.person_id
deltaImportQuery=SELECT p.person_id, p.authz_id, p.details,
p.last_modified, a.address_info from person p, address a where p.person_id =
a.person_id and p.person_id='${dataimporter.delta.person_id}'
deltaQuery=SELECT person_id from person where last_modified
gt; '${dataimporter.last_index_time}'
/entity
schema.xml:
dynamicField name=person_* type=string indexed=true stored=true/
field name=person_id type=string indexed=true stored=true/
uniqueKeyperson_id/uniqueKey
Thanks
Peri
On May 31, 2014, at 8:05 AM, Ahmet Arslan iori...@yahoo.com wrote:
Hi,
Did you restart solr? Can you paste relevant portions of data-config.xml and
schema.xml?
On Saturday, May 31, 2014 3:46 AM, PeriS peri.subrahma...@htcinc.com wrote:
I added the primaryKey as the uniqueKey and still same result.
On May 30, 2014, at 8:38 PM, Ahmet Arslan iori...@yahoo.com wrote:
Hi,
Sure, have a look at uniqueKey definition in example schema.xml
http://wiki.apache.org/solr/UniqueKey
On Saturday, May 31, 2014 3:35 AM, PeriS peri.subrahma...@htcinc.com wrote:
No. Is there a way to have the primary key of my entity be the unique key?
On May 30, 2014, at 7:00 PM, Ahmet Arslan iori...@yahoo.com wrote:
Hi,
Do you have uniqueKey defined in schema.xml ?
On Saturday, May 31, 2014 1:23 AM, PeriS peri.subrahma...@htcinc.com
wrote:
Hi,
I have followed the documentation to set up my delta query, but when I call
the delta-import, the index is happening again for the same record and
ends up being indexed twice. Any clues please?
Thanks
-Peri
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC
Global Services to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
Thank you,
Peri Subrahmanya
HTC Global Services
(KOLE)
Cell: (+1) 618.407.3521
Skype/Gtalk: peri.subrahmanya
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC
Global Services to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC
Global Services to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
Thank you,
Peri Subrahmanya
HTC Global Services
(KOLE)
Cell: (+1) 618.407.3521
Skype/Gtalk: peri.subrahmanya
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC
Global Services to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global
Services to any order or other contract unless pursuant to explicit written
agreement or government initiative expressly permitting the use of e-mail for
such purpose.
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of the
mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global
Services to any order or other contract unless pursuant to explicit written
agreement or government initiative expressly permitting the use of e-mail for
such purpose.
Re: DIH Issue
Seems like the issue got fixed after I updated my import query to the following;
entity name=item pk=person_id transformer=“some transformer
query=SELECT p.person_id, p.details, p.authz_id,
p.last_modified, a.address_info from person p, address a where p.person_id =
a.person_id
AND '${dataimporter.request.clean}' != 'false'
OR p.last_modified '${dataimporter.last_index_time}'
/entity
On May 31, 2014, at 9:24 AM, PeriS peri.subrahma...@htcinc.com wrote:
db-import.xml:
entity name=doctor pk= person_id transformer=“Some transformer
query=SELECT p.person_id, p.details, p.authz_id,
p.last_modified, a.address_info from person p, address a where p.person_id =
a.person_id
deltaImportQuery=SELECT p.person_id, p.authz_id, p.details,
p.last_modified, a.address_info from person p, address a where p.person_id =
a.person_id and p.person_id='${dataimporter.delta.person_id}'
deltaQuery=SELECT person_id from person where last_modified
gt; '${dataimporter.last_index_time}'
/entity
schema.xml:
dynamicField name=person_* type=string indexed=true stored=true/
field name=person_id type=string indexed=true stored=true/
uniqueKeyperson_id/uniqueKey
Thanks
Peri
On May 31, 2014, at 8:05 AM, Ahmet Arslan iori...@yahoo.com wrote:
Hi,
Did you restart solr? Can you paste relevant portions of data-config.xml and
schema.xml?
On Saturday, May 31, 2014 3:46 AM, PeriS peri.subrahma...@htcinc.com wrote:
I added the primaryKey as the uniqueKey and still same result.
On May 30, 2014, at 8:38 PM, Ahmet Arslan iori...@yahoo.com wrote:
Hi,
Sure, have a look at uniqueKey definition in example schema.xml
http://wiki.apache.org/solr/UniqueKey
On Saturday, May 31, 2014 3:35 AM, PeriS peri.subrahma...@htcinc.com
wrote:
No. Is there a way to have the primary key of my entity be the unique key?
On May 30, 2014, at 7:00 PM, Ahmet Arslan iori...@yahoo.com wrote:
Hi,
Do you have uniqueKey defined in schema.xml ?
On Saturday, May 31, 2014 1:23 AM, PeriS peri.subrahma...@htcinc.com
wrote:
Hi,
I have followed the documentation to set up my delta query, but when I
call the delta-import, the index is happening again for the same record
and ends up being indexed twice. Any clues please?
Thanks
-Peri
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC
Global Services to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
Thank you,
Peri Subrahmanya
HTC Global Services
(KOLE)
Cell: (+1) 618.407.3521
Skype/Gtalk: peri.subrahmanya
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC
Global Services to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC
Global Services to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
Thank you,
Peri Subrahmanya
HTC Global Services
(KOLE)
Cell: (+1) 618.407.3521
Skype/Gtalk: peri.subrahmanya
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC
Global Services to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of
the mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC
Global Services to any order or other contract unless pursuant to explicit
written agreement or government initiative expressly permitting the use of
e-mail for such purpose.
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail
DIH Issue
Hi, I have followed the documentation to set up my delta query, but when I call the delta-import, the index is happening again for the same record and ends up being indexed twice. Any clues please? Thanks -Peri *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Re: DIH Issue
No. Is there a way to have the primary key of my entity be the unique key? On May 30, 2014, at 7:00 PM, Ahmet Arslan iori...@yahoo.com wrote: Hi, Do you have uniqueKey defined in schema.xml ? On Saturday, May 31, 2014 1:23 AM, PeriS peri.subrahma...@htcinc.com wrote: Hi, I have followed the documentation to set up my delta query, but when I call the delta-import, the index is happening again for the same record and ends up being indexed twice. Any clues please? Thanks -Peri *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. Thank you, Peri Subrahmanya HTC Global Services (KOLE) Cell: (+1) 618.407.3521 Skype/Gtalk: peri.subrahmanya *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Re: DIH Issue
I added the primaryKey as the uniqueKey and still same result. On May 30, 2014, at 8:38 PM, Ahmet Arslan iori...@yahoo.com wrote: Hi, Sure, have a look at uniqueKey definition in example schema.xml http://wiki.apache.org/solr/UniqueKey On Saturday, May 31, 2014 3:35 AM, PeriS peri.subrahma...@htcinc.com wrote: No. Is there a way to have the primary key of my entity be the unique key? On May 30, 2014, at 7:00 PM, Ahmet Arslan iori...@yahoo.com wrote: Hi, Do you have uniqueKey defined in schema.xml ? On Saturday, May 31, 2014 1:23 AM, PeriS peri.subrahma...@htcinc.com wrote: Hi, I have followed the documentation to set up my delta query, but when I call the delta-import, the index is happening again for the same record and ends up being indexed twice. Any clues please? Thanks -Peri *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. Thank you, Peri Subrahmanya HTC Global Services (KOLE) Cell: (+1) 618.407.3521 Skype/Gtalk: peri.subrahmanya *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. Thank you, Peri Subrahmanya HTC Global Services (KOLE) Cell: (+1) 618.407.3521 Skype/Gtalk: peri.subrahmanya *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
DIH - LastModifiedDate - Format
Hi, I am using MySQL as the datastore and for the last_modified_date use the java.util.Date. I m seeing that the DIH doesn’t seem to pick records; Is there a date format that I should use for DIH to compare properly and pick up the records for indexing? Thanks -Peri.S *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
DIH - Query
I have a simple join query;
entity name=“person query=SELECT person_id, details, last_modified FROM
person WHERE '${dataimporter.request.clean}' != 'false' OR last_modified
'${dataimporter.last_index_time}'
entity name=address query=SELECT address_info FROM address
WHERE person_id = '${person.person_id}' /
/entity
This doesn’t bring back the child address records. Am I missing something?
Thanks
-Peri.S
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of the
mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global
Services to any order or other contract unless pursuant to explicit written
agreement or government initiative expressly permitting the use of e-mail for
such purpose.
Re: DIH - Query
Thanks; I verified that SOLR is intact returning the parent/child rows, but for some reason, the transformer that I have on the parent table, only receives the rows from the parent record. I was wondering how the transformer can get the result of the join query which would be both parent/child records. On Feb 9, 2014, at 1:28 PM, Shawn Heisey s...@elyograg.org wrote: This is probably easiest to do with the Logging tab in the admin UI, *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Re: Index a new record in MySQL
Hi,
Version of SOLR is 4.6 and here is my db-import-config.xml
dataConfig
dataSource name=dbDS type=JdbcDataSource driver=com.mysql.jdbc.Driver
url=[fill me]
user=“fill me password=“fill me/
document name=“example name
entity name=“table name pk=id
transformer=com.example.sometransformer
query=SELECT * from [table]
deltaImportQuery=SELECT * from [table] where
[pk]='${dataimporter.delta.pk}'
deltaQuery=SELECT [pk] from [table] where last_modified gt;
'${dataimporter.last_index_time}'
field column=“col name=“col name/
field column=last_modified name=lastModified/
/entity
/document
/dataConfig
On Feb 8, 2014, at 7:29 AM, tamanjit.bin...@yahoo.co.in wrote:
But delta indexing is supposed to do that. Please share details of your
datatimport-config.xml and version of Solr for further help.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Index-a-new-record-in-MySQL-tp4116164p4116184.html
Sent from the Solr - User mailing list archive at Nabble.com.
*** DISCLAIMER *** This is a PRIVATE message. If you are not the intended
recipient, please delete without copying and kindly advise us by e-mail of the
mistake in delivery.
NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global
Services to any order or other contract unless pursuant to explicit written
agreement or government initiative expressly permitting the use of e-mail for
such purpose.
Index a new record in MySQL
Hi, How do I approach the issue of firing the DIH without it having to index the whole DB when adding a new record? It appears that when a new record is added the delta query on DIH doesn’t pick up the record. And I don’t want to run a full index on the DB when adding 1 single row. Any suggestions please? Thanks -Peri *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
DIH Update question
What would be the process to update a new record in an existing db using DIH? Thanks
DIH Question
When creating a new record in the db, and running the deal-import command, i m not seeing the new record being indexed. Is there some configuration I need to set? The use is the db already has records loaded and I would like to index new records. Whats the process? Any ideas please? Thanks -Peri.S
Re: DIH Question
1. Current num records in db: 4 2. Created a new record and called delta-import using the following url; localhost:8983/dataimport?command=delta-import SOLR Log here : http://apaste.info/gF3N 3. When i tried checking the status from the browser - logs are here: http://apaste.info/gxDF db-import-config.xml is here : http://apaste.info/3t0K Thanks -Peri On Jun 12, 2013, at 12:11 PM, Gora Mohanty g...@mimirtech.com wrote: Please provide sufficient details as we do not have access to your server, and there could be a million things that are wrong. Start by sharing your DIH configuration file, the exact URL that you are using for doing the delta import, and the message that you get in the browser when the delta import completes: This will have details about how many documents were picked up. Also, how are you checking that the new document was not indexed. Regards, Gora
Re: DIH Question
What would be the process to update a new record in an existing db using DIH? On Jun 12, 2013, at 1:06 PM, PeriS pvsub...@indiana.edu wrote: 1. Current num records in db: 4 2. Created a new record and called delta-import using the following url; localhost:8983/dataimport?command=delta-import SOLR Log here : http://apaste.info/gF3N 3. When i tried checking the status from the browser - logs are here: http://apaste.info/gxDF db-import-config.xml is here : http://apaste.info/3t0K Thanks -Peri On Jun 12, 2013, at 12:11 PM, Gora Mohanty g...@mimirtech.com wrote: Please provide sufficient details as we do not have access to your server, and there could be a million things that are wrong. Start by sharing your DIH configuration file, the exact URL that you are using for doing the delta import, and the message that you get in the browser when the delta import completes: This will have details about how many documents were picked up. Also, how are you checking that the new document was not indexed. Regards, Gora
Re: Field Names
So what I m trying to do is index MARC21 fields which have a schema defined at http://www.loc.gov/marc/bibliographic/. Now the reason I wanted to use dynamic fields is so that once they are defined I don't have to touch the schema.xml and control indexed fields configuration from outside i.e. a particular user may want to index only certain marc fields as opposed to another user. This will also enable for quick on the fly adjustments rather than having to restart the solr server every time the schema.xml changes (unless that is not needed). Since the users and the application is aware of the schema, they know how to query the fields or what to query for. Does that make sense? On Jun 10, 2013, at 10:48 PM, Jack Krupansky j...@basetechnology.com wrote: One idea: DON'T DO IT! Seriously, if you find yourself trying to play games with field names, it says that you probably have a data model that is grossly out of line with the strengths (and weaknesses) of Solr. Dynamic fields are fine - when used in moderation, but not when pushed to extremes. What are you really trying to model and why does it seem to depend on dynamic fields? In particular, how would users and the application query those dynamic fields? -- Jack Krupansky -Original Message- From: PeriS Sent: Monday, June 10, 2013 9:54 PM To: solr-user@lucene.apache.org Subject: Field Names I was wondering if there was a way to define field names that are more less dynamic in nature but follow a regular expression pattern. I know you can have asterisk either as a prefix or a suffix but not both or somewhere int he middle of a name. Goal: to define a field that takes up the form like 10*_* which would translate to 100_whatever thru 109_whatever; Any ideas please? Thanks -PeriS
Re: Field Names
Nothing wrong and would work too, but I wanted to see if I could restrict it more as per the schema at http://www.loc.gov/marc/bibliographic/. On Jun 10, 2013, at 10:09 PM, Gora Mohanty g...@mimirtech.com wrote: What is wrong with, e.g., dynamicField name=10*whatever type=integer indexed=true stored=true/ Regards, Gora
Re: Field Names
And besides, the user is not going to be aware of the actual field names to query on - its abstracted to some simple names like author which could be a combination of 100a,b which the application would know how to construct based on an external configuration etc… You see what i mean. On Jun 11, 2013, at 7:14 AM, PeriS pvsub...@indiana.edu wrote: So what I m trying to do is index MARC21 fields which have a schema defined at http://www.loc.gov/marc/bibliographic/. Now the reason I wanted to use dynamic fields is so that once they are defined I don't have to touch the schema.xml and control indexed fields configuration from outside i.e. a particular user may want to index only certain marc fields as opposed to another user. This will also enable for quick on the fly adjustments rather than having to restart the solr server every time the schema.xml changes (unless that is not needed). Since the users and the application is aware of the schema, they know how to query the fields or what to query for. Does that make sense? On Jun 10, 2013, at 10:48 PM, Jack Krupansky j...@basetechnology.com wrote: One idea: DON'T DO IT! Seriously, if you find yourself trying to play games with field names, it says that you probably have a data model that is grossly out of line with the strengths (and weaknesses) of Solr. Dynamic fields are fine - when used in moderation, but not when pushed to extremes. What are you really trying to model and why does it seem to depend on dynamic fields? In particular, how would users and the application query those dynamic fields? -- Jack Krupansky -Original Message- From: PeriS Sent: Monday, June 10, 2013 9:54 PM To: solr-user@lucene.apache.org Subject: Field Names I was wondering if there was a way to define field names that are more less dynamic in nature but follow a regular expression pattern. I know you can have asterisk either as a prefix or a suffix but not both or somewhere int he middle of a name. Goal: to define a field that takes up the form like 10*_* which would translate to 100_whatever thru 109_whatever; Any ideas please? Thanks -PeriS
Error indexing string field
I have a field declared as type string, so should it care whats inside the string? Caused by: java.lang.NumberFormatException: For input string: 1835-1910. Thanks -Peri
Field Names
I was wondering if there was a way to define field names that are more less dynamic in nature but follow a regular expression pattern. I know you can have asterisk either as a prefix or a suffix but not both or somewhere int he middle of a name. Goal: to define a field that takes up the form like 10*_* which would translate to 100_whatever thru 109_whatever; Any ideas please? Thanks -PeriS
Re: Dynamic Indexing using DB and DIH
Shawn, You got the point; I do have a the unique key defined, but for some reason, when i run the delta-import; a new entry is created for the same record with a new unique key. Its almost somehow it doesn't detect the existing record. On Jun 3, 2013, at 3:51 PM, Shawn Heisey s...@elyograg.org wrote: On 6/3/2013 12:35 PM, PeriS wrote: I noticed the delta-import is creating a new indexed entry on top of the existing one..is that normal? Not sure what you are asking here, so I'll give an answer to the question I think you're asking: If you have a uniqueKey defined in your schema, then new documents with matching values in the uniqueKey field will replace the existing documents. Solr will delete the old one before inserting the new one. Thanks, Shawn
Re: Delta Import failing - DataImportHandler SOLR 4.2
BTW the primary key is a combination of 2 fields. So not sure if thats the issue. On Jun 2, 2013, at 1:08 AM, PeriS pvsub...@indiana.edu wrote: I have configured the delta query properly, but not sure why the DIH is throwing the following error; SEVERE: Delta Import Failed java.lang.RuntimeException: java.lang.NullPointerException at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:266) at org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:451) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:489) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: java.lang.NullPointerException at org.apache.solr.handler.dataimport.DocBuilder.findMatchingPkColumn(DocBuilder.java:718) at org.apache.solr.handler.dataimport.DocBuilder.collectDelta(DocBuilder.java:783) at org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:334) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:219) ... 3 more Thanks -Peri.S
Re: Delta Import failing - DataImportHandler SOLR 4.2
Ok, so I fixed the issue by providing the pk= in the entity definition as mentioned in http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command I also have a transformer declared for the entity and the DIH during the deltaImport doesn't seem to be passing all the fields to the transformer (only the primary key) which causes the indexes for other fields not to be updated. Any ideas? On Jun 2, 2013, at 9:38 AM, PeriS pvsub...@indiana.edu wrote: BTW the primary key is a combination of 2 fields. So not sure if thats the issue. On Jun 2, 2013, at 1:08 AM, PeriS pvsub...@indiana.edu wrote: I have configured the delta query properly, but not sure why the DIH is throwing the following error; SEVERE: Delta Import Failed java.lang.RuntimeException: java.lang.NullPointerException at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:266) at org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:451) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:489) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: java.lang.NullPointerException at org.apache.solr.handler.dataimport.DocBuilder.findMatchingPkColumn(DocBuilder.java:718) at org.apache.solr.handler.dataimport.DocBuilder.collectDelta(DocBuilder.java:783) at org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:334) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:219) ... 3 more Thanks -Peri.S
Re: Delta Import failing - DataImportHandler SOLR 4.2
Shawn, The db-import-config.xml snippet can be found here:http://apaste.info/sTUw Thanks -Peri.S On Jun 2, 2013, at 11:15 AM, Shawn Heisey s...@elyograg.org wrote: On 6/2/2013 8:45 AM, PeriS wrote: Ok, so I fixed the issue by providing the pk= in the entity definition as mentioned in http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command I also have a transformer declared for the entity and the DIH during the deltaImport doesn't seem to be passing all the fields to the transformer (only the primary key) which causes the indexes for other fields not to be updated. Any ideas? Can you provide your dataimport config file? It may have a name like db-data-config.xml, but its name is configurable, so I have no way of knowing what yours is. If you need sanitize it, change the username/password/hostname values to something else, don't remove them entirely. Depending on what comes out of that, we might also need schema.xml and solrconfig.xml. Please use a paste website and give us the URL(s), don't try to include it in your message. There is an Apache paste website for use by people outside the project: http://apaste.info Thanks, Shawn
Re: Delta Import failing - DataImportHandler SOLR 4.2
I did try with din namespace and that didn't seem to make any difference. Since
the PK is a composite in my case, just specifying the bib_id was throwing an
exception stating could not find the matching pk column or something to that
effect. Although I realize the use cases for using one or the other strategy,
since in my case I am only updating one row at a time or perhaps 1K at most, I
should be able to use the all in one query strategy for the imports rather
than having three separate ones.
Thanks
On Jun 2, 2013, at 1:25 PM, Shawn Heisey s...@elyograg.org wrote:
On 6/2/2013 10:11 AM, PeriS wrote:
I found using the strategy mentioned at
http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport, it
works for me. Not sure what the difference is between this one and writing
individual queries for fetching the IDs first and then getting the data; I
mean I know the difference, but not sure why the previous way doesn't work.
Glad you found a solution. As for the initial problem:
I am guessing that the pk value in the config file should be just
bib_id. That value is only used within dataimporter, it has no
bearing on the Solr index. I think it also has nothing to do with the
interaction with the database. As far as I know, it only gets used to
validate SQL input data to make sure it is sane, especially with delta
imports.
Also, there might be a problem with the dataimporter namespace for
subsitution variables, that has migrated to dih in newer versions, so
you need things like this:
${dih.delta.bib_id}
${dih.last_index_time}
I have seen problems where the dataimporter namespace hasn't worked, dih
is much safer.
Thanks,
Shawn
Dynamic Indexing using DB and DIH
Currently I have wired up the dataimporthandler to do a full and incremental indexing. I was wondering if there was way to automatically update the indexes as soon as the row in the table gets updated. I don't want to get into any sort of cron jobs, triggers etc; Current what I do is as soon as I update the row, i follow it up by calling the delta import. But in this case its about timing and if SOLR doesn't see the row as updated, then it doesn't do anything….any ideas? -Peri.S
Delta Import failing - DataImportHandler SOLR 4.2
I have configured the delta query properly, but not sure why the DIH is throwing the following error; SEVERE: Delta Import Failed java.lang.RuntimeException: java.lang.NullPointerException at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:266) at org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:451) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:489) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: java.lang.NullPointerException at org.apache.solr.handler.dataimport.DocBuilder.findMatchingPkColumn(DocBuilder.java:718) at org.apache.solr.handler.dataimport.DocBuilder.collectDelta(DocBuilder.java:783) at org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:334) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:219) ... 3 more Thanks -Peri.S
solrj 4.3.0
I m trying to use EmbeddedSolrServer but when trying to initialize the coreContainer, get the following error; java.lang.NoClassDefFoundError: org/apache/solr/common/ResourceLoader Any ideas please? -Peri.S *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Re: solrj 4.3.0
Actually fixed it. So by accident i was using sole-core 3.x version. Once I upgraded the version of solar-core to 4.x it got resolved. Thanks -Peri.S On May 15, 2013, at 7:14 PM, Shawn Heisey s...@elyograg.org wrote: On 5/15/2013 5:02 PM, PeriS wrote: I m trying to use EmbeddedSolrServer but when trying to initialize the coreContainer, get the following error; java.lang.NoClassDefFoundError: org/apache/solr/common/ResourceLoader Did you only use the solrj jar and the jars in solrj-libs? This is enough for a solrj client to access a separate solr server, but if you want to run the full embedded server, you'll need a whole lot more jars, starting with solr-core and extending to lucene and other third-party requirements. If you extract the .war file, you'll find everything that you need in there for a basic install. Thanks, Shawn *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Re: solrj 4.3.0
Although now its complaining about even though i have provided the correct core name. org.apache.solr.common.SolrException: No such core On May 15, 2013, at 7:21 PM, PeriS peri.subrahma...@htcinc.com wrote: Actually fixed it. So by accident i was using sole-core 3.x version. Once I upgraded the version of solar-core to 4.x it got resolved. Thanks -Peri.S On May 15, 2013, at 7:14 PM, Shawn Heisey s...@elyograg.org wrote: On 5/15/2013 5:02 PM, PeriS wrote: I m trying to use EmbeddedSolrServer but when trying to initialize the coreContainer, get the following error; java.lang.NoClassDefFoundError: org/apache/solr/common/ResourceLoader Did you only use the solrj jar and the jars in solrj-libs? This is enough for a solrj client to access a separate solr server, but if you want to run the full embedded server, you'll need a whole lot more jars, starting with solr-core and extending to lucene and other third-party requirements. If you extract the .war file, you'll find everything that you need in there for a basic install. Thanks, Shawn *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Re: solrj 4.3.0
After I turned on the logging, I found the following stack trace: Error loading class 'org.apache.solr.handler.dataimport.DataImportHandler' Not sure why the embeddedsolrserver is looking for it... On May 15, 2013, at 7:26 PM, PeriS pvsub...@indiana.edu wrote: Although now its complaining about even though i have provided the correct core name. org.apache.solr.common.SolrException: No such core On May 15, 2013, at 7:21 PM, PeriS peri.subrahma...@htcinc.com wrote: Actually fixed it. So by accident i was using sole-core 3.x version. Once I upgraded the version of solar-core to 4.x it got resolved. Thanks -Peri.S On May 15, 2013, at 7:14 PM, Shawn Heisey s...@elyograg.org wrote: On 5/15/2013 5:02 PM, PeriS wrote: I m trying to use EmbeddedSolrServer but when trying to initialize the coreContainer, get the following error; java.lang.NoClassDefFoundError: org/apache/solr/common/ResourceLoader Did you only use the solrj jar and the jars in solrj-libs? This is enough for a solrj client to access a separate solr server, but if you want to run the full embedded server, you'll need a whole lot more jars, starting with solr-core and extending to lucene and other third-party requirements. If you extract the .war file, you'll find everything that you need in there for a basic install. Thanks, Shawn *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
Re: solrj 4.3.0
Figured it out; Added a dependency for the dataimporthandler in my pom file. On May 15, 2013, at 11:59 PM, PeriS pvsub...@indiana.edu wrote: After I turned on the logging, I found the following stack trace: Error loading class 'org.apache.solr.handler.dataimport.DataImportHandler' Not sure why the embeddedsolrserver is looking for it... On May 15, 2013, at 7:26 PM, PeriS pvsub...@indiana.edu wrote: Although now its complaining about even though i have provided the correct core name. org.apache.solr.common.SolrException: No such core On May 15, 2013, at 7:21 PM, PeriS peri.subrahma...@htcinc.com wrote: Actually fixed it. So by accident i was using sole-core 3.x version. Once I upgraded the version of solar-core to 4.x it got resolved. Thanks -Peri.S On May 15, 2013, at 7:14 PM, Shawn Heisey s...@elyograg.org wrote: On 5/15/2013 5:02 PM, PeriS wrote: I m trying to use EmbeddedSolrServer but when trying to initialize the coreContainer, get the following error; java.lang.NoClassDefFoundError: org/apache/solr/common/ResourceLoader Did you only use the solrj jar and the jars in solrj-libs? This is enough for a solrj client to access a separate solr server, but if you want to run the full embedded server, you'll need a whole lot more jars, starting with solr-core and extending to lucene and other third-party requirements. If you extract the .war file, you'll find everything that you need in there for a basic install. Thanks, Shawn *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. *** DISCLAIMER *** This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind HTC Global Services to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose.
