Getting Status 400 - org.apache.lucene.queryParser.ParseException on submitting a search query having ^ symbol through Web Application
Hi,
I am getting the following error while triggering a search query through the
web application :
HTTP Status 400 - org.apache.lucene.queryParser.ParseException: Cannot parse
'Sigma Survey for Police
Officersfield=Index_Type_s:productIndexfield=productType_s:productfield=Index_Type_s:productIndexfield=productType_s:programfield=Index_Type_s:productIndexfield=productType_s:course^1000qt=dismaxrequesthl=true':
Encountered : at line 1, column 84. Was expecting one of: EOF AND ...
OR ... NOT ... + ... - ... ( ... * ... ^ ... QUOTED ...
TERM ... FUZZY_SLOP ... PREFIXTERM ... WILDTERM ... [ ... { ...
NUMBER ...
Search Query URL :
http://localhost:8080/apache-solr-1.3.0/core51043/select/?
q=The Rough Guide to Film Musicals
1field=Index_Type_s:productIndexfield=Index_Type_s:productIndexfield=productType_s:course^1000qt=dismaxrequesthl=true
(The same query returns proper result when hit on the Apache Solr server
directly)
In the application I m applying URLEncoding on the search string thus the
entire search string gets converted into :
http://localhost:8080/apache-solr-1.3.0/core51043/select/?
q=Sigma+Survey+for+Police+Officers%26field%3DIndex_Type_s%3AproductIndex%26field%3DproductType_s%3Aproduct%26field%3DIndex_Type_s%3AproductIndex%26field%3DproductType_s%3Aprogram%26field%3DIndex_Type_s%3AproductIndex%26field%3DproductType_s%3Acourse%5E1000%26qt%3Ddismaxrequest%26hl%3Dtrue
The error comes only when ^1000 gets appended into the search string , issue
persists even if even URLEncoding is applied.
Please provide suggestion for the same.
Regards,
Payal
--
View this message in context:
http://www.nabble.com/Getting-Status-400---org.apache.lucene.queryParser.ParseException-on-submitting-a-search-query-having-%5E-symbol-through-Web-Application-tp20419668p20419668.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: What is a cor in solr
A core is kind of like a single solr index and all of its config/settings. So multicore allows you to have multiple indexes, each with its own config/settings. Its an alternative to multiple solr installs. - Mark Sajith Vimukthi wrote: Hi all, Can someone of u tell me what a core is? Especially with regard to multycore. Regards, Sajith Vimukthi Weerakoon Associate Software Engineer | ZONE24X7 | Tel: +94 11 2882390 ext 101 | Fax: +94 11 2878261 | http://www.zone24x7.com
Re: Solr with Wordpress - Anyone doing this?
I don't know of anyone that has done this, but I would welcome it as well. I suspect the main issue is that most WP users live in a shared hosting world, where Java doesn't play very nicely. That being said, it would be fairly easy to use the DataImportHandler's feed import for indexing (I think) and then it's just a matter of pointing the search box at the Solr instance, I suppose. On Nov 6, 2008, at 11:19 PM, Stephen Weiss wrote: Hi, We recently implemented Solr for one major search component of our site, and now that this is complete we're turning to other areas of our site to see where Solr can help us improve results relevancy and performance. One major area where I think Solr could do a lot of good is to replace Wordpress's search function. Wordpress powers a solid 1/3 of our site, and moving this over could provide dramatic performance benefits. I see there is a Lucene plugin for WP but I have not seen any plugin yet using Solr. While I'm not terribly afraid of writing one (we've already completely replaced the built-in routine with our own plugin to optimize MySQL searching), it would of course be even better if there was some sort of plugin already out there (why reinvent the wheel)? Somehow it just seems strange that no one would have tried this yet. I figure if anyone knows, someone on this list knows. Thanks for any info! -- Steve -- Grant Ingersoll Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ
Re: solr 1.3 - Problem Full Import
Hi,
I can't get what's going on ??
If somebody could help me it would be very nice
thanks
sunnyfr wrote:
Hi,
I don't get what's happening
I tried to make a full import with a limit inside between to id ...
everything works fine then the time keep running but it looks stuck and
Idon't have any error in my logs:
Like if a process couldn't be executed, I didn't turn on snapshooter or
autocommit is properly setteled I think :
autoCommit
maxDocs1/maxDocs
maxTime1000/maxTime
/autoCommit
Data-Config:
str name=configdata-config.xml/str
/lst
/lst
str name=statusbusy/str
str name=importResponseA command is still running.../str
−
lst name=statusMessages
str name=Time Elapsed0:16:32.47/str ... str name=Time
Elapsed0:19:18.160/str
str name=Total Requests made to DataSource661141/str
str name=Total Rows Fetched1647265/str
str name=Total Documents Processed110189/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-06 16:50:17/str
/lst
−
str name=WARNING
This response format is experimental. It is likely to change in the
future.
/str
/response
My stats :
name: /dataimport
class:org.apache.solr.handler.dataimport.DataImportHandler
version: 1.0
description: Manage data import from databases to Solr
stats:Status : RUNNING_FULL_DUMP
Documents Processed : 110189
Requests made to DataSource : 661141
Rows Fetched : 1647265
Documents Deleted : 0
Documents Skipped : 0
Total Documents Processed : 0
Total Requests made to DataSource : 0
Total Rows Fetched : 0
Total Documents Deleted : 0
Total Documents Skipped : 0
handlerStart : 1225986600254
requests : 86
errors : 0
timeouts : 0
totalTime : 0
avgTimePerRequest : 0.0
avgRequestsPerSecond : 0.05853414
searcherName : [EMAIL PROTECTED] main
caching : true
numDocs : 109686
maxDoc : 109686
readerImpl : MultiSegmentReader
readerDir :
org.apache.lucene.store.FSDirectory@/data/solr/video/data/index
indexVersion : 1225986600336
openedAt : Thu Nov 06 16:57:37 CET 2008
registeredAt : Thu Nov 06 16:57:37 CET 2008
warmupTime : 0
Logs:
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result for
[EMAIL PROTECTED] main
^IfilterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming
[EMAIL PROTECTED] main from [EMAIL PROTECTED] main
^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=50,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.core.SolrDeletionPolicy onInit INFO:
SolrDeletionPolicy.onInit: commits:num=1
^Icommit{dir=/data/solr/video/data/index,segFN=segments_6g,version=1225986600336,generation=232,filenames=[segments_6g,
_33.fnm, _33.frq, _33.prx, _33.tis, _33.tii, _33.nrm, _33.tvx, _33.tvf,
_33.tvd, _33.fdx, _33.fdt, _66.fnm, _66.frq, _66.prx, _66.tis, _66.tii,
_66.nrm, _66.tvx, _66.tvf, _66.tvd, _66.fdx, _66.fdt, _6g.fnm, _6g.frq,
_6g.prx, _6g.tis, _6g.tii, _6g.nrm, _6g.tvx, _6g.tvf, _6g.tvd, _6g.fdx,
_6g.fdt, _6r.fnm, _6r.frq, _6r.prx, _6r.tis, _6r.tii, _6r.nrm, _6r.tvx,
_6r.tvf, _6r.tvd, _6r.fdx, _6r.fdt, _72.fnm, _72.frq, _72.prx, _72.tis,
_72.tii, _72.nrm, _72.tvx, _72.tvf, _72.tvd, _72.fdx, _72.fdt, _73.fnm,
_73.frq, _73.prx, _73.tis, _73.tii, _73.nrm, _73.tvx, _73.tvf, _73.tvd,
_73.fdx, _73.fdt]
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit =
1225986600336
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result for
[EMAIL PROTECTED] main
^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming
[EMAIL PROTECTED] main from [EMAIL PROTECTED] main
^IdocumentCache{lookups=0,hits=0,hitratio=0.00,inserts=20,evictions=0,size=20,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result for
[EMAIL PROTECTED] main
Re: Filters: acute accent characters replaced with their english counterpart
Wiadomość napisana w dniu 2008-11-10, o godz. 11:14, przez joeMcElroy: I need a custom filter to be added to a field which will replace special foreign characters with their english counterpart. for example ø = o Grave À È Ì Ò Ù à è ì ò ù = A E I O U a e i o u Circumflex Â Ê Î Ô Û â ê î ô û = A E I O U a e i o u is this possible? http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#head-4ebf7aea23b3d6d34a1f8314f9de17334a3e2fac I wish such filter exist for Latin2... -- We read Knuth so you don't have to. - Tim Peters Jarek Zgoda, RD, Redefine [EMAIL PROTECTED]
RE: Filters: acute accent characters replaced with their english counterpart
Hi Jarek, On 11/10/2008 at 6:08 AM, Jarek Zgoda wrote: Wiadomość napisana w dniu 2008-11-10, o godz. 11:14, przez joeMcElroy: I need a custom filter to be added to a field which will replace special foreign characters with their english counterpart. for example ø = o Grave À È Ì Ò Ù à è ì ò ù = A E I O U a e i o u Circumflex Â Ê Î Ô Û â ê î ô û = A E I O U a e i o u is this possible? http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#head-4ebf7aea23b3d6d34a1f8314f9de17334a3e2fac I wish such filter exist for Latin2... The following Lucene patch hasn't been committed yet, and there is no Solr Factory counterpart yet, but: ASCIIFoldingFilter folds all accented letters to their (accent-stripped, if necessary) ASCII equivalents: https://issues.apache.org/jira/browse/LUCENE-1390 Steve
What is a cor in solr
Hi all, Can someone of u tell me what a core is? Especially with regard to multycore. Regards, Sajith Vimukthi Weerakoon Associate Software Engineer | ZONE24X7 | Tel: +94 11 2882390 ext 101 | Fax: +94 11 2878261 | http://www.zone24x7.com
Re: solr 1.3 - Problem Full Import
Hi Shalin
I don't get Ive executed :
http://com:8180/solr/video/dataimport?command=full-import
Then when I Checked :
http://solr-test.adm.video.com:8180/solr/video/dataimport
It looks like document are created :
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=statusbusy/str
str name=importResponseA command is still running.../str
−
lst name=statusMessages
str name=Time Elapsed0:0:46.331/str
str name=Total Requests made to DataSource37347/str
str name=Total Rows Fetched75108/str
str name=Total Documents Processed6224/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-10 17:02:24/str
/lst
but then after a while, time keep increasing but just time no documents ;..
and it can be like that for hours.
str name=statusbusy/str
str name=importResponseA command is still running.../str
−
lst name=statusMessages
str name=Time Elapsed0:4:41.295/str
str name=Total Requests made to DataSource84103/str
str name=Total Rows Fetched113842/str
str name=Total Documents Processed14016/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-10 17:02:24/str
/lst
and no exception : http://www.nabble.com/file/p20422919/syslog syslog
Thanks for your help,
Shalin Shekhar Mangar wrote:
Hi,
I'm confused by the log you have posted. There is no import command to be
seen to /dataimport. Only one request is in the log to /dataimport at the
very end. The logs say that Solr started up and some warming queries were
executed and that's it.
What do you mean by it looks stuck? Does the count of processed documents
increase? Any exceptions during import would cause DataImportHandler to
abort. The only case that DIH will not catch is when a error like out of
memory happened. But still Solr or Tomcat should catch and log that
exception (or it may just die).
On Mon, Nov 10, 2008 at 8:33 PM, sunnyfr [EMAIL PROTECTED] wrote:
Hi,
I can't get what's going on ??
If somebody could help me it would be very nice
thanks
sunnyfr wrote:
Hi,
I don't get what's happening
I tried to make a full import with a limit inside between to id ...
everything works fine then the time keep running but it looks stuck and
Idon't have any error in my logs:
Like if a process couldn't be executed, I didn't turn on snapshooter or
autocommit is properly setteled I think :
autoCommit
maxDocs1/maxDocs
maxTime1000/maxTime
/autoCommit
Data-Config:
str name=configdata-config.xml/str
/lst
/lst
str name=statusbusy/str
str name=importResponseA command is still running.../str
−
lst name=statusMessages
str name=Time Elapsed0:16:32.47/str ... str name=Time
Elapsed0:19:18.160/str
str name=Total Requests made to DataSource661141/str
str name=Total Rows Fetched1647265/str
str name=Total Documents Processed110189/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-06 16:50:17/str
/lst
−
str name=WARNING
This response format is experimental. It is likely to change in the
future.
/str
/response
My stats :
name: /dataimport
class:org.apache.solr.handler.dataimport.DataImportHandler
version: 1.0
description: Manage data import from databases to Solr
stats:Status : RUNNING_FULL_DUMP
Documents Processed : 110189
Requests made to DataSource : 661141
Rows Fetched : 1647265
Documents Deleted : 0
Documents Skipped : 0
Total Documents Processed : 0
Total Requests made to DataSource : 0
Total Rows Fetched : 0
Total Documents Deleted : 0
Total Documents Skipped : 0
handlerStart : 1225986600254
requests : 86
errors : 0
timeouts : 0
totalTime : 0
avgTimePerRequest : 0.0
avgRequestsPerSecond : 0.05853414
searcherName : [EMAIL PROTECTED] main
caching : true
numDocs : 109686
maxDoc : 109686
readerImpl : MultiSegmentReader
readerDir :
org.apache.lucene.store.FSDirectory@/data/solr/video/data/index
indexVersion : 1225986600336
openedAt : Thu Nov 06 16:57:37 CET 2008
registeredAt : Thu Nov 06 16:57:37 CET 2008
warmupTime : 0
Logs:
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result
for
[EMAIL PROTECTED] main
^IfilterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming
[EMAIL PROTECTED] main from [EMAIL PROTECTED] main
^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=50,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
Re: Solr with Wordpress - Anyone doing this?
I'm not sure what kind of interfaces WordPress expose. Does it have a DB/REST end point? If so, it would be very easy to write a sample data-config.xml for wordpress. --Noble On Mon, Nov 10, 2008 at 8:13 PM, Grant Ingersoll [EMAIL PROTECTED] wrote: I don't know of anyone that has done this, but I would welcome it as well. I suspect the main issue is that most WP users live in a shared hosting world, where Java doesn't play very nicely. That being said, it would be fairly easy to use the DataImportHandler's feed import for indexing (I think) and then it's just a matter of pointing the search box at the Solr instance, I suppose. On Nov 6, 2008, at 11:19 PM, Stephen Weiss wrote: Hi, We recently implemented Solr for one major search component of our site, and now that this is complete we're turning to other areas of our site to see where Solr can help us improve results relevancy and performance. One major area where I think Solr could do a lot of good is to replace Wordpress's search function. Wordpress powers a solid 1/3 of our site, and moving this over could provide dramatic performance benefits. I see there is a Lucene plugin for WP but I have not seen any plugin yet using Solr. While I'm not terribly afraid of writing one (we've already completely replaced the built-in routine with our own plugin to optimize MySQL searching), it would of course be even better if there was some sort of plugin already out there (why reinvent the wheel)? Somehow it just seems strange that no one would have tried this yet. I figure if anyone knows, someone on this list knows. Thanks for any info! -- Steve -- Grant Ingersoll Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ -- --Noble Paul
Re: solr 1.3 - Problem Full Import
Hi Shalin
I don't get Ive executed :
http://com:8180/solr/video/dataimport?command=full-import
Then when I Checked :
http://solr-test.adm.dailymotion.com:8180/solr/video/dataimport
It looks like document are created :
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=statusbusy/str
str name=importResponseA command is still running.../str
−
lst name=statusMessages
str name=Time Elapsed0:0:46.331/str
str name=Total Requests made to DataSource37347/str
str name=Total Rows Fetched75108/str
str name=Total Documents Processed6224/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-10 17:02:24/str
/lst
but then after a while, time keep increasing but just time no documents ;..
and it can be like that for hours.
str name=statusbusy/str
str name=importResponseA command is still running.../str
−
lst name=statusMessages
str name=Time Elapsed0:4:41.295/str
str name=Total Requests made to DataSource84103/str
str name=Total Rows Fetched113842/str
str name=Total Documents Processed14016/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-10 17:02:24/str
/lst
and no exception : http://www.nabble.com/file/p20422854/syslog syslog
Thanks for your help,
Shalin Shekhar Mangar wrote:
Hi,
I'm confused by the log you have posted. There is no import command to be
seen to /dataimport. Only one request is in the log to /dataimport at the
very end. The logs say that Solr started up and some warming queries were
executed and that's it.
What do you mean by it looks stuck? Does the count of processed documents
increase? Any exceptions during import would cause DataImportHandler to
abort. The only case that DIH will not catch is when a error like out of
memory happened. But still Solr or Tomcat should catch and log that
exception (or it may just die).
On Mon, Nov 10, 2008 at 8:33 PM, sunnyfr [EMAIL PROTECTED] wrote:
Hi,
I can't get what's going on ??
If somebody could help me it would be very nice
thanks
sunnyfr wrote:
Hi,
I don't get what's happening
I tried to make a full import with a limit inside between to id ...
everything works fine then the time keep running but it looks stuck and
Idon't have any error in my logs:
Like if a process couldn't be executed, I didn't turn on snapshooter or
autocommit is properly setteled I think :
autoCommit
maxDocs1/maxDocs
maxTime1000/maxTime
/autoCommit
Data-Config:
str name=configdata-config.xml/str
/lst
/lst
str name=statusbusy/str
str name=importResponseA command is still running.../str
−
lst name=statusMessages
str name=Time Elapsed0:16:32.47/str ... str name=Time
Elapsed0:19:18.160/str
str name=Total Requests made to DataSource661141/str
str name=Total Rows Fetched1647265/str
str name=Total Documents Processed110189/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-06 16:50:17/str
/lst
−
str name=WARNING
This response format is experimental. It is likely to change in the
future.
/str
/response
My stats :
name: /dataimport
class:org.apache.solr.handler.dataimport.DataImportHandler
version: 1.0
description: Manage data import from databases to Solr
stats:Status : RUNNING_FULL_DUMP
Documents Processed : 110189
Requests made to DataSource : 661141
Rows Fetched : 1647265
Documents Deleted : 0
Documents Skipped : 0
Total Documents Processed : 0
Total Requests made to DataSource : 0
Total Rows Fetched : 0
Total Documents Deleted : 0
Total Documents Skipped : 0
handlerStart : 1225986600254
requests : 86
errors : 0
timeouts : 0
totalTime : 0
avgTimePerRequest : 0.0
avgRequestsPerSecond : 0.05853414
searcherName : [EMAIL PROTECTED] main
caching : true
numDocs : 109686
maxDoc : 109686
readerImpl : MultiSegmentReader
readerDir :
org.apache.lucene.store.FSDirectory@/data/solr/video/data/index
indexVersion : 1225986600336
openedAt : Thu Nov 06 16:57:37 CET 2008
registeredAt : Thu Nov 06 16:57:37 CET 2008
warmupTime : 0
Logs:
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result
for
[EMAIL PROTECTED] main
^IfilterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming
[EMAIL PROTECTED] main from [EMAIL PROTECTED] main
^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=50,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37
Re: Solr with Wordpress - Anyone doing this?
Unfortunately I don't think it's that sophisticated... There was a request out in the Wordpress world for an extendable search interface (like Drupal) but I don't think it got much traction. The plugin we use now for searching simply implements hooks that modify the query (a lot) before it goes to the DB. Our use of Wordpress is not exactly typical... for us it's more of a CMS than a blog. We'd have no problem integrating it with Java... Right now I'm using version 1.2 still (I'm waiting for the bug reports on 1.3 to settle down, especially in regard to DataImportHandler), so I'm probably not going to be using DataImportHandler. My plan was just to have it set up to send add document commands whenever a post is published or modified (after being published). Right now I'm looking at a plugin that exists to integrate WP with Lucene (using ZendSearchLucene, I think). I think it could be fairly easily modified to work with Solr instead (it just has to send commands over HTTP instead of using direct access to the index). My only thing is we wanted to use faceting as well, which isn't really implemented with that plugin at all. After I didn't find anything or get a response for a while we've already started working on it... Nothing complete yet but at least so far I can see everywhere where I think things need to be modified (it's not the top priority so we're not exactly moving quickly). It's not so bad, just work. Since I'm not using DataImportHandler my plugin may not end up being very useful to others but perhaps I'll put it up somewhere anyway as a rough example. Thanks for the replies. -- Steve On Nov 10, 2008, at 11:27 AM, Noble Paul നോബിള് नोब्ळ् wrote: I'm not sure what kind of interfaces WordPress expose. Does it have a DB/REST end point? If so, it would be very easy to write a sample data-config.xml for wordpress. --Noble On Mon, Nov 10, 2008 at 8:13 PM, Grant Ingersoll [EMAIL PROTECTED] wrote: I don't know of anyone that has done this, but I would welcome it as well. I suspect the main issue is that most WP users live in a shared hosting world, where Java doesn't play very nicely. That being said, it would be fairly easy to use the DataImportHandler's feed import for indexing (I think) and then it's just a matter of pointing the search box at the Solr instance, I suppose. On Nov 6, 2008, at 11:19 PM, Stephen Weiss wrote: Hi, We recently implemented Solr for one major search component of our site, and now that this is complete we're turning to other areas of our site to see where Solr can help us improve results relevancy and performance. One major area where I think Solr could do a lot of good is to replace Wordpress's search function. Wordpress powers a solid 1/3 of our site, and moving this over could provide dramatic performance benefits. I see there is a Lucene plugin for WP but I have not seen any plugin yet using Solr. While I'm not terribly afraid of writing one (we've already completely replaced the built-in routine with our own plugin to optimize MySQL searching), it would of course be even better if there was some sort of plugin already out there (why reinvent the wheel)? Somehow it just seems strange that no one would have tried this yet. I figure if anyone knows, someone on this list knows. Thanks for any info! -- Steve -- Grant Ingersoll Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ -- --Noble Paul
Re: solr 1.3 - Problem Full Import
Hi,
I'm confused by the log you have posted. There is no import command to be
seen to /dataimport. Only one request is in the log to /dataimport at the
very end. The logs say that Solr started up and some warming queries were
executed and that's it.
What do you mean by it looks stuck? Does the count of processed documents
increase? Any exceptions during import would cause DataImportHandler to
abort. The only case that DIH will not catch is when a error like out of
memory happened. But still Solr or Tomcat should catch and log that
exception (or it may just die).
On Mon, Nov 10, 2008 at 8:33 PM, sunnyfr [EMAIL PROTECTED] wrote:
Hi,
I can't get what's going on ??
If somebody could help me it would be very nice
thanks
sunnyfr wrote:
Hi,
I don't get what's happening
I tried to make a full import with a limit inside between to id ...
everything works fine then the time keep running but it looks stuck and
Idon't have any error in my logs:
Like if a process couldn't be executed, I didn't turn on snapshooter or
autocommit is properly setteled I think :
autoCommit
maxDocs1/maxDocs
maxTime1000/maxTime
/autoCommit
Data-Config:
str name=configdata-config.xml/str
/lst
/lst
str name=statusbusy/str
str name=importResponseA command is still running.../str
−
lst name=statusMessages
str name=Time Elapsed0:16:32.47/str ... str name=Time
Elapsed0:19:18.160/str
str name=Total Requests made to DataSource661141/str
str name=Total Rows Fetched1647265/str
str name=Total Documents Processed110189/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-06 16:50:17/str
/lst
−
str name=WARNING
This response format is experimental. It is likely to change in the
future.
/str
/response
My stats :
name: /dataimport
class:org.apache.solr.handler.dataimport.DataImportHandler
version: 1.0
description: Manage data import from databases to Solr
stats:Status : RUNNING_FULL_DUMP
Documents Processed : 110189
Requests made to DataSource : 661141
Rows Fetched : 1647265
Documents Deleted : 0
Documents Skipped : 0
Total Documents Processed : 0
Total Requests made to DataSource : 0
Total Rows Fetched : 0
Total Documents Deleted : 0
Total Documents Skipped : 0
handlerStart : 1225986600254
requests : 86
errors : 0
timeouts : 0
totalTime : 0
avgTimePerRequest : 0.0
avgRequestsPerSecond : 0.05853414
searcherName : [EMAIL PROTECTED] main
caching : true
numDocs : 109686
maxDoc : 109686
readerImpl : MultiSegmentReader
readerDir :
org.apache.lucene.store.FSDirectory@/data/solr/video/data/index
indexVersion : 1225986600336
openedAt : Thu Nov 06 16:57:37 CET 2008
registeredAt : Thu Nov 06 16:57:37 CET 2008
warmupTime : 0
Logs:
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result
for
[EMAIL PROTECTED] main
^IfilterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming
[EMAIL PROTECTED] main from [EMAIL PROTECTED] main
^IqueryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=50,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.core.SolrDeletionPolicy onInit INFO:
SolrDeletionPolicy.onInit: commits:num=1
^Icommit{dir=/data/solr/video/data/index,segFN=segments_6g,version=1225986600336,generation=232,filenames=[segments_6g,
_33.fnm, _33.frq, _33.prx, _33.tis, _33.tii, _33.nrm, _33.tvx, _33.tvf,
_33.tvd, _33.fdx, _33.fdt, _66.fnm, _66.frq, _66.prx, _66.tis, _66.tii,
_66.nrm, _66.tvx, _66.tvf, _66.tvd, _66.fdx, _66.fdt, _6g.fnm, _6g.frq,
_6g.prx, _6g.tis, _6g.tii, _6g.nrm, _6g.tvx, _6g.tvf, _6g.tvd, _6g.fdx,
_6g.fdt, _6r.fnm, _6r.frq, _6r.prx, _6r.tis, _6r.tii, _6r.nrm, _6r.tvx,
_6r.tvf, _6r.tvd, _6r.fdx, _6r.fdt, _72.fnm, _72.frq, _72.prx, _72.tis,
_72.tii, _72.nrm, _72.tvx, _72.tvf, _72.tvd, _72.fdx, _72.fdt, _73.fnm,
_73.frq, _73.prx, _73.tis, _73.tii, _73.nrm, _73.tvx, _73.tvf, _73.tvd,
_73.fdx, _73.fdt]
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.core.SolrDeletionPolicy updateCommits INFO: last commit =
1225986600336
Nov 6 16:57:37 solr-test jsvc.exec[24862]: Nov 6, 2008 4:57:37 PM
org.apache.solr.search.SolrIndexSearcher warm INFO: autowarming result
for
[EMAIL PROTECTED] main
simple filter query solr processing
i have instances of felds which have commas seperating them. i need it to be treated as one filter (even tho its seperated by commas) for example facet : film,media,mass communication rendered result by solr Cannot parse ' +i_subjects:Film': Lexical error at line 1, column 19. Encountered: EOF after : \Film i_subjects:Filmfq=+mediafq=+mass+communication i do not want it splitting commas and replacing them with fq, but completely matching on i_subjects:film,media,mass communication is this possible? i reckon there is some config option to turn this off? background to the problem i am indexing items which have a metadata called subjects. this would be for example humanities/film, media, mass communication/film im tokenizing on the / which results in humanities ; film, media, mass communication everything works except when the 'category' involves , tried with removing the quotes fq=+%2Bi_subjects:Filmfq=+mediafq=+mass+communication but results in no matches tried splitting on the commas and inserting the key at each part +%2Bi_subjects:Film+%2Bi_subjects:+media+%2Bi_subjects:+mass+communication showed no results. tried removing the plusses i am inserting but now shows too many results fq=+i_subjects:Film+i_subjects:+media+i_subjects:+mass+communication tried putting quotes in between each filter part fq=+i_subjects:Film+i_subjects:+media+i_subjects:+mass+communication now shows too few results any pointers please? joe -- View this message in context: http://www.nabble.com/simple-filter-query-solr-processing-tp20418363p20418363.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Core Size limit
Hi, Im using Solr multicore functionality in my app. I want to know the size limit of holding the index files in each core.How can i identify the maximum size limit of the cores. Thanks in advance Prabhu.K -- View this message in context: http://www.nabble.com/Solr-Core-Size-limit-tp20416899p20416899.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Core Size limit
Hi, I don't think there is a limit other than your hardware and the internal Doc ID which limits you to 2B docs on 32-bit machines. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch From: RaghavPrabhu [EMAIL PROTECTED] To: solr-user@lucene.apache.org Sent: Monday, November 10, 2008 5:43:17 AM Subject: Solr Core Size limit Hi, Im using Solr multicore functionality in my app. I want to know the size limit of holding the index files in each core.How can i identify the maximum size limit of the cores. Thanks in advance Prabhu.K -- View this message in context: http://www.nabble.com/Solr-Core-Size-limit-tp20416899p20416899.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Filters: acute accent characters replaced with their english counterpart
joe, This hasn't been committed yet, but SOLR-822 may be your answer. https://issues.apache.org/jira/browse/SOLR-822 Koji joeMcElroy wrote: I need a custom filter to be added to a field which will replace special foreign characters with their english counterpart. for example ø = o Grave À È Ì Ò Ù à è ì ò ù = A E I O U a e i o u Circumflex Â Ê Î Ô Û â ê î ô û = A E I O U a e i o u is this possible? joe
Filters: acute accent characters replaced with their english counterpart
I need a custom filter to be added to a field which will replace special foreign characters with their english counterpart. for example ø = o Grave À È Ì Ò Ù à è ì ò ù = A E I O U a e i o u Circumflex Â Ê Î Ô Û â ê î ô û = A E I O U a e i o u is this possible? joe -- View this message in context: http://www.nabble.com/Filters%3A-acute-accent-characters-replaced-with-their-english-counterpart-tp20416888p20416888.html Sent from the Solr - User mailing list archive at Nabble.com.
Using DataImportHandler with mysql database
Hey there, I am trying to use the DataImportHandler to index data from a mysql database. I am having the same error all the time just when I start tomcat: Nov 10, 2008 7:39:49 PM org.apache.solr.handler.dataimport.DataImporter loadDataConfig INFO: Data Configuration loaded successfully Nov 10, 2008 7:39:49 PM org.apache.solr.handler.dataimport.DataImportHandler inform SEVERE: Exception while loading DataImporter java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.init(DataImporter.java:95) at org.apache.solr.handler.dataimport.DataImportHandler.inform(DataImportHandler.java:106) ... I am using the oficial release of solar 1.3. First I tried to add the compiled DataImportHandler jar. As it didn't work what I did was: I downloaded the package org/apache/solr/handler/dataimport from a nightly build and have added it and compiled to my solr 1.3 oficial source release. This way I have my solr1.3 release with the DataImporthandler In solrconfig.xml I have created a request handler to make the import: requestHandler name=/dataimport class=org.apache.solr.handler.dataimport.DataImportHandler lst name=defaults str name=config/path_to_/data-config.xml/str /lst /requestHandler To connect to the database , in data-config.xml I am doing: dataConfig dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver url=jdbc:mysql://localhost/db_name user=root password=/ ...and here I do the select and the mapping db_field - index_field *The mysql connector is correctly added in the classpath I think I must be missing something in my configuration but can't find what... Anyone can give me a hand? I am a bit lost with this problem... Thanks in advanced Marc Sturlese -- View this message in context: http://www.nabble.com/Using-DataImportHandler-with-mysql-database-tp20425791p20425791.html Sent from the Solr - User mailing list archive at Nabble.com.
DataImportHandler - composite multivalued fields for merging db rows?
Hi, I'm setting up a Solr 1.3 server for our existing app, and have an issue
importing.
We have a marketing system and want to make campaigns more searchable. Each
campaign has a number of terms which specify either a fixed or percentage
commission (2 separate fields). In addition each term points to a tracker
that can be of type CLICK, IMPRESSION, etc.
I need each campaign to be a document. Since a campaign can have multiple
terms (with one tracker each) I've setup the fields from terms and tracker
as multivalued. However this just gets me 3 separate lists, and even though
there is a relationship between their elements, I can't be certain that
element 1 in the commissionfixed list goes with element 1 in the type list.
The ideal thing would be to encapsulate each terms with its tracker in some
sort of composite (or at least tokenized) field, and then make that
multi-valued. Is this possible somehow? I've been looking at the
documentation for fieldType without luck.
Here are my simplified schema and data-config files:
entity name=campaign pk=id
field column=name/
entity name=terms pk=id query=select * from terms where
campaign_id='${campaign.id}'
field column=commissionfixed/
field column=commissionpercent/
entity name=tracker pk=id query=select type from tracker
where id='${terms.tracker_id}'
field column=type/
/entity
/entity
/entity
field name=name type=text indexed=true stored=true/
field name=type type=text multiValued=true indexed=true
stored=true/
field name=commissionfixed type=float multiValued=true
stored=true/
field name=commissionpercent type=float multiValued=true
stored=true/
Thanks.
Re: Using DataImportHandler with mysql database
It seems like your data-config does not have any document tag. The
following is the correct structure:
dataConfig
document
entity .../
/document
/dataConfig
On Tue, Nov 11, 2008 at 12:31 AM, Marc Sturlese [EMAIL PROTECTED]wrote:
Hey there,
I am trying to use the DataImportHandler to index data from a mysql
database. I am having the same error all the time just when I start tomcat:
Nov 10, 2008 7:39:49 PM
org.apache.solr.handler.dataimport.DataImportHandler
processConfiguration
INFO: Processing configuration from solrconfig.xml:
{config=/path_to/data-config.xml}
Nov 10, 2008 7:39:49 PM org.apache.solr.handler.dataimport.DataImporter
loadDataConfig
INFO: Data Configuration loaded successfully
Nov 10, 2008 7:39:49 PM
org.apache.solr.handler.dataimport.DataImportHandler
inform
SEVERE: Exception while loading DataImporter
java.lang.NullPointerException
at
org.apache.solr.handler.dataimport.DataImporter.init(DataImporter.java:95)
at
org.apache.solr.handler.dataimport.DataImportHandler.inform(DataImportHandler.java:106)
...
I am using the oficial release of solar 1.3. First I tried to add the
compiled DataImportHandler jar. As it didn't work what I did was:
I downloaded the package org/apache/solr/handler/dataimport from a nightly
build and have added it and compiled to my solr 1.3 oficial source release.
This way I have my solr1.3 release with the DataImporthandler
In solrconfig.xml I have created a request handler to make the import:
requestHandler name=/dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
str name=config/path_to_/data-config.xml/str
/lst
/requestHandler
To connect to the database , in data-config.xml I am doing:
dataConfig
dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver
url=jdbc:mysql://localhost/db_name user=root password=/ ...and here
I
do the select and the mapping db_field - index_field
*The mysql connector is correctly added in the classpath
I think I must be missing something in my configuration but can't find
what...
Anyone can give me a hand? I am a bit lost with this problem...
Thanks in advanced
Marc Sturlese
--
View this message in context:
http://www.nabble.com/Using-DataImportHandler-with-mysql-database-tp20425791p20425791.html
Sent from the Solr - User mailing list archive at Nabble.com.
--
Regards,
Shalin Shekhar Mangar.
Boosting newer inserted items in search?
Hi! When doing a search we want to have a have parameters that we set of how to boost different scores. for example we want to have the ability to boost newer items in some case and in some not, like a sliding scale from only time to only relevancy. What is the best way of doing this, using Solrj? Regards Erik
Re: How to use multicore feature in JBOSS
: One final question: : Even thogh things are working fine, The debug messages that i get from the : JBOSS console shows STDERR. : : A few lines from the JBOSS console while deploying solr.war: : : 11:44:20,578 ERROR [STDERR] 7 Nov, 2008 11:44:20 AM : org.apache.solr.servlet.SolrDispatchFilter init : INFO: SolrDispatchFilter.init() ... none of those messages are errors they are log messages coming from Solr using a variety of log levels (that first one for example is is logged at the INFO level. The ERROR and [STDERR] labels seem to be getting applied by JBoss -- you should consult JBoss documentation to find out how to control it's behavior when dealing with logging from applications. (if i had to guess: the ConsoleHandler writes all messages to System.err and is the default hadler created by the JVM. If JBoss doesn't register any java.util.logging controls (which seems somewhat ridiculous for any non trivial servlet container) the messages might be going through System.err and *then* JBoss is seeing them and reporting them -- which would explain the double timestamps in the log message formating as well) -Hoss
Re: Solr 1.3 stack overflow when accessing solr/admin page
Olga: noticing this thread now, are you still having this problem? did you find a problem with your configuration? : However I am getting stack overflow when I try to access solr/admin : page. ... : java.lang.StackOverflowError javax.servlet.ServletException: ... : org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j : ava:273) at that line is where the SolrDispatchFilter tries to forward the request if the path starts with /admin and no handler has been registered to deal with it. the fact that you're getting a StackOverflowError suggests that i'ts causing an infinite recursion. knowing exactly what URL you are trying ot hit, what the name of your webapp is, whether you are using multiple cores or not, and what your solr.xml file (if you have one) looks like would help. Also: if you try to hit a handler registered with a name thta starts with /admin (like /admin/luke) does that work? -Hoss
Re: Boosting newer inserted items in search?
My gut tells me that multiplying a function of the date field with the full-text relevancy score will work better than adding a function of the date field. BoostedQuery can do this via BoostQParserPlugin http://lucene.apache.org/solr/api/org/apache/solr/search/BoostQParserPlugin.html And for the function, you could make it flatter or steeper depending on how much you wanted date to influence the score. Might try a recip function or a linear function on the ord of the date field. We don't yet have any functions that work on the actual date values. http://wiki.apache.org/solr/FunctionQuery -Yonik On Mon, Nov 10, 2008 at 3:25 PM, Erik Holstad [EMAIL PROTECTED] wrote: Hi! When doing a search we want to have a have parameters that we set of how to boost different scores. for example we want to have the ability to boost newer items in some case and in some not, like a sliding scale from only time to only relevancy. What is the best way of doing this, using Solrj? Regards Erik
Re: Solr Core Size limit
On Mon, 10 Nov 2008 10:24:47 -0800 (PST)
Otis Gospodnetic [EMAIL PROTECTED] wrote:
I don't think there is a limit other than your hardware and the internal Doc
ID which limits you to 2B docs on 32-bit machines.
Hi Otis,
just curious is this internal doc ID reused when an optimise happens? or
gaps left and re-filled when 2B is reached ?
cheers,
b
_
{Beto|Norberto|Numard} Meijome
Whenever you find that you are on the side of the majority, it is time to
reform.
Mark Twain
I speak for myself, not my employer. Contents may be hot. Slippery when wet.
Reading disclaimers makes you go blind. Writing them is worse. You have been
Warned.
Re: Boosting newer inserted items in search?
Thanks Yonik for the answer1 Will try to implement that, but I can't seem to find how to do that using Solrj, do I just add it to the Query field or how is it done? Regards Erik On Mon, Nov 10, 2008 at 2:59 PM, Yonik Seeley [EMAIL PROTECTED] wrote: My gut tells me that multiplying a function of the date field with the full-text relevancy score will work better than adding a function of the date field. BoostedQuery can do this via BoostQParserPlugin http://lucene.apache.org/solr/api/org/apache/solr/search/BoostQParserPlugin.html And for the function, you could make it flatter or steeper depending on how much you wanted date to influence the score. Might try a recip function or a linear function on the ord of the date field. We don't yet have any functions that work on the actual date values. http://wiki.apache.org/solr/FunctionQuery -Yonik On Mon, Nov 10, 2008 at 3:25 PM, Erik Holstad [EMAIL PROTECTED] wrote: Hi! When doing a search we want to have a have parameters that we set of how to boost different scores. for example we want to have the ability to boost newer items in some case and in some not, like a sliding scale from only time to only relevancy. What is the best way of doing this, using Solrj? Regards Erik
Newbie Question - getting search results from dataimport request handler
My Question is: what is the format of a search that will return data?
i.e. /solr/select?q=developerqt=dataimport (won’t work) nor will
/solr/dataimport?q=developer (won’t work)
“HTTP ERROR: 404
NOT_FOUND
RequestURI=/solr/dataimport“
I have created a ‘dataimport’ set that contains data from a sql db.
I can view meta data from this url: /solr/dataimport
response
−
lst name=responseHeader
int name=status0/int
int name=QTime0/int
/lst
−
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=statusidle/str
str name=importResponse/
−
lst name=statusMessages
str name=Total Requests made to DataSource1/str
str name=Total Rows Fetched10/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-10 21:51:40/str
str name=Time taken 0:0:4.594/str
/lst
−
str name=WARNING
This response format is experimental. It is likely to change in the future.
/str
/response
I can verify that the data is there by going through /solr/admin/dataimport.jsp
and doing ‘verbose’ true and debug now.
It shows me the xml data set on the right as such:
response
−
lst name=responseHeader
int name=status0/int
int name=QTime4594/int
/lst
−
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=commandfull-import/str
str name=modedebug/str
−
arr name=documents
−
arr
−
arr
int87133/int
/arr
/arr
−
arr
−
arr
int87134/int
/arr
/arr
−
arr
−
arr
int87135/int
/arr
/arr
−
arr
−
arr
int87136/int
/arr
/arr
−
arr
−
arr
int87137/int
/arr
/arr
−
arr
−
arr
int87138/int
/arr
/arr
−
arr
−
arr
int87139/int
/arr
/arr
−
arr
−
arr
int87140/int
/arr
/arr
−
arr
−
arr
int87141/int
/arr
/arr
−
arr
−
arr
int87142/int
/arr
/arr
/arr
−
lst name=verbose-output
−
lst name=entity:item
−
lst name=document#1
−
str name=query
SELECT j.id , j.title , FROM dbo.jobs j WITH (NOLOCK)
LEFT WHERE j.siteid = 46 and j.active = 1
/str
str name=time-taken0:0:4.578/str
str--- row #1-/str
str name=zip/
str name=urltitleOperations Software Developer Job/str
str name=altlocationSAN ANTONIO, TX, 78229/str
str name=alttitleOpe…
Here is my solconfig.xml
…
requestHandler name=dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
str name=configdata-config.xml/str
/lst
/requestHandler
…
Data-config.xml is in the same dir as solconfig.xml
My data-config.xml is like any other:
dataConfig
dataSource driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
url=jdbc:sqlserver://:1433;databaseName=x user=x
password=x /
document name=jobs
entity name=item pk=id query=SELECT j.id
, j.title
…
FROM
dbo.jobs …
WHERE j.siteid = 46 and
j.active = 1
deltaQuery=select id from dbo.jobs where lastmodified
'${dataimporter.last_index_time}'
/entity
/document
/dataConfig
I’m using win xp with apache – and jetty + solr 1.3.0
Thanks
Re: Newbie Question - getting search results from dataimport request handler
you cannot query the DIH. It can only do indexing
after indexing you must do the indexing on the regular query interface
On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote:
My Question is: what is the format of a search that will return data?
i.e. /solr/select?q=developerqt=dataimport (won't work) nor will
/solr/dataimport?q=developer (won't work)
HTTP ERROR: 404
NOT_FOUND
RequestURI=/solr/dataimport
I have created a 'dataimport' set that contains data from a sql db.
I can view meta data from this url: /solr/dataimport
response
−
lst name=responseHeader
int name=status0/int
int name=QTime0/int
/lst
−
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=statusidle/str
str name=importResponse/
−
lst name=statusMessages
str name=Total Requests made to DataSource1/str
str name=Total Rows Fetched10/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-10 21:51:40/str
str name=Time taken 0:0:4.594/str
/lst
−
str name=WARNING
This response format is experimental. It is likely to change in the future.
/str
/response
I can verify that the data is there by going through
/solr/admin/dataimport.jsp and doing 'verbose' true and debug now.
It shows me the xml data set on the right as such:
response
−
lst name=responseHeader
int name=status0/int
int name=QTime4594/int
/lst
−
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=commandfull-import/str
str name=modedebug/str
−
arr name=documents
−
arr
−
arr
int87133/int
/arr
/arr
−
arr
−
arr
int87134/int
/arr
/arr
−
arr
−
arr
int87135/int
/arr
/arr
−
arr
−
arr
int87136/int
/arr
/arr
−
arr
−
arr
int87137/int
/arr
/arr
−
arr
−
arr
int87138/int
/arr
/arr
−
arr
−
arr
int87139/int
/arr
/arr
−
arr
−
arr
int87140/int
/arr
/arr
−
arr
−
arr
int87141/int
/arr
/arr
−
arr
−
arr
int87142/int
/arr
/arr
/arr
−
lst name=verbose-output
−
lst name=entity:item
−
lst name=document#1
−
str name=query
SELECT j.id , j.title , FROM dbo.jobs j WITH (NOLOCK)
LEFT WHERE j.siteid = 46 and j.active = 1
/str
str name=time-taken0:0:4.578/str
str--- row #1-/str
str name=zip/
str name=urltitleOperations Software Developer Job/str
str name=altlocationSAN ANTONIO, TX, 78229/str
str name=alttitleOpe…
Here is my solconfig.xml
…
requestHandler name=dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
str name=configdata-config.xml/str
/lst
/requestHandler
…
Data-config.xml is in the same dir as solconfig.xml
My data-config.xml is like any other:
dataConfig
dataSource driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
url=jdbc:sqlserver://:1433;databaseName=x user=x
password=x /
document name=jobs
entity name=item pk=id query=SELECT j.id
, j.title
…
FROM
dbo.jobs …
WHERE j.siteid = 46 and
j.active = 1
deltaQuery=select id from dbo.jobs where lastmodified
'${dataimporter.last_index_time}'
/entity
/document
/dataConfig
I'm using win xp with apache – and jetty + solr 1.3.0
Thanks
--
--Noble Paul
RE: Newbie Question - getting search results from dataimport request handler
Ok - and what would that be? (query interface)
I need the URL format that would work in this situation to return data from my
setup.
I've gone through the tutorial and used execution strings like:
http://localhost:8983/solr/select/?indent=onq=videosort=price+desc
etc however I'm working with sql data and not xml data.
Thanks
-Original Message-
From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED]
Sent: Monday, November 10, 2008 10:18 PM
To: solr-user@lucene.apache.org
Subject: Re: Newbie Question - getting search results from dataimport request
handler
you cannot query the DIH. It can only do indexing
after indexing you must do the indexing on the regular query interface
On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote:
My Question is: what is the format of a search that will return data?
i.e. /solr/select?q=developerqt=dataimport (won't work) nor will
/solr/dataimport?q=developer (won't work)
HTTP ERROR: 404
NOT_FOUND
RequestURI=/solr/dataimport
I have created a 'dataimport' set that contains data from a sql db.
I can view meta data from this url: /solr/dataimport
response
−
lst name=responseHeader
int name=status0/int
int name=QTime0/int
/lst
−
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=statusidle/str
str name=importResponse/
−
lst name=statusMessages
str name=Total Requests made to DataSource1/str
str name=Total Rows Fetched10/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-10 21:51:40/str
str name=Time taken 0:0:4.594/str
/lst
−
str name=WARNING
This response format is experimental. It is likely to change in the future.
/str
/response
I can verify that the data is there by going through
/solr/admin/dataimport.jsp and doing 'verbose' true and debug now.
It shows me the xml data set on the right as such:
response
−
lst name=responseHeader
int name=status0/int
int name=QTime4594/int
/lst
−
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=commandfull-import/str
str name=modedebug/str
−
arr name=documents
−
arr
−
arr
int87133/int
/arr
/arr
−
arr
−
arr
int87134/int
/arr
/arr
−
arr
−
arr
int87135/int
/arr
/arr
−
arr
−
arr
int87136/int
/arr
/arr
−
arr
−
arr
int87137/int
/arr
/arr
−
arr
−
arr
int87138/int
/arr
/arr
−
arr
−
arr
int87139/int
/arr
/arr
−
arr
−
arr
int87140/int
/arr
/arr
−
arr
−
arr
int87141/int
/arr
/arr
−
arr
−
arr
int87142/int
/arr
/arr
/arr
−
lst name=verbose-output
−
lst name=entity:item
−
lst name=document#1
−
str name=query
SELECT j.id , j.title , FROM dbo.jobs j WITH (NOLOCK)
LEFT WHERE j.siteid = 46 and j.active = 1
/str
str name=time-taken0:0:4.578/str
str--- row #1-/str
str name=zip/
str name=urltitleOperations Software Developer Job/str
str name=altlocationSAN ANTONIO, TX, 78229/str
str name=alttitleOpe…
Here is my solconfig.xml
…
requestHandler name=dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
str name=configdata-config.xml/str
/lst
/requestHandler
…
Data-config.xml is in the same dir as solconfig.xml
My data-config.xml is like any other:
dataConfig
dataSource driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
url=jdbc:sqlserver://:1433;databaseName=x user=x
password=x /
document name=jobs
entity name=item pk=id query=SELECT j.id
, j.title
…
FROM
dbo.jobs …
WHERE j.siteid = 46 and
j.active = 1
deltaQuery=select id from dbo.jobs where lastmodified
'${dataimporter.last_index_time}'
/entity
/document
/dataConfig
I'm using win xp with apache – and jetty + solr 1.3.0
Thanks
--
--Noble Paul
Re: Newbie Question - getting search results from dataimport request handler
XML is just an intermediete data format Solr internally has no XML
data. When the data comes out XML is just another representation of
the same data.
Whether you put in data using XML or DB (SQL) it all goes into the
same index . Query must be done on that index using the syntax
http://localhost:8983/solr/select/?q=your-query-goes-here
On Tue, Nov 11, 2008 at 9:55 AM, Kevin Penny [EMAIL PROTECTED] wrote:
Ok - and what would that be? (query interface)
I need the URL format that would work in this situation to return data from
my setup.
I've gone through the tutorial and used execution strings like:
http://localhost:8983/solr/select/?indent=onq=videosort=price+desc
etc however I'm working with sql data and not xml data.
Thanks
-Original Message-
From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED]
Sent: Monday, November 10, 2008 10:18 PM
To: solr-user@lucene.apache.org
Subject: Re: Newbie Question - getting search results from dataimport request
handler
you cannot query the DIH. It can only do indexing
after indexing you must do the indexing on the regular query interface
On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote:
My Question is: what is the format of a search that will return data?
i.e. /solr/select?q=developerqt=dataimport (won't work) nor will
/solr/dataimport?q=developer (won't work)
HTTP ERROR: 404
NOT_FOUND
RequestURI=/solr/dataimport
I have created a 'dataimport' set that contains data from a sql db.
I can view meta data from this url: /solr/dataimport
response
−
lst name=responseHeader
int name=status0/int
int name=QTime0/int
/lst
−
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=statusidle/str
str name=importResponse/
−
lst name=statusMessages
str name=Total Requests made to DataSource1/str
str name=Total Rows Fetched10/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2008-11-10 21:51:40/str
str name=Time taken 0:0:4.594/str
/lst
−
str name=WARNING
This response format is experimental. It is likely to change in the future.
/str
/response
I can verify that the data is there by going through
/solr/admin/dataimport.jsp and doing 'verbose' true and debug now.
It shows me the xml data set on the right as such:
response
−
lst name=responseHeader
int name=status0/int
int name=QTime4594/int
/lst
−
lst name=initArgs
−
lst name=defaults
str name=configdata-config.xml/str
/lst
/lst
str name=commandfull-import/str
str name=modedebug/str
−
arr name=documents
−
arr
−
arr
int87133/int
/arr
/arr
−
arr
−
arr
int87134/int
/arr
/arr
−
arr
−
arr
int87135/int
/arr
/arr
−
arr
−
arr
int87136/int
/arr
/arr
−
arr
−
arr
int87137/int
/arr
/arr
−
arr
−
arr
int87138/int
/arr
/arr
−
arr
−
arr
int87139/int
/arr
/arr
−
arr
−
arr
int87140/int
/arr
/arr
−
arr
−
arr
int87141/int
/arr
/arr
−
arr
−
arr
int87142/int
/arr
/arr
/arr
−
lst name=verbose-output
−
lst name=entity:item
−
lst name=document#1
−
str name=query
SELECT j.id , j.title , FROM dbo.jobs j WITH (NOLOCK)
LEFT WHERE j.siteid = 46 and j.active = 1
/str
str name=time-taken0:0:4.578/str
str--- row #1-/str
str name=zip/
str name=urltitleOperations Software Developer Job/str
str name=altlocationSAN ANTONIO, TX, 78229/str
str name=alttitleOpe…
Here is my solconfig.xml
…
requestHandler name=dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
lst name=defaults
str name=configdata-config.xml/str
/lst
/requestHandler
…
Data-config.xml is in the same dir as solconfig.xml
My data-config.xml is like any other:
dataConfig
dataSource driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
url=jdbc:sqlserver://:1433;databaseName=x user=x
password=x /
document name=jobs
entity name=item pk=id query=SELECT j.id
, j.title
…
FROM
dbo.jobs …
WHERE j.siteid = 46 and
j.active = 1
deltaQuery=select id from dbo.jobs where lastmodified
'${dataimporter.last_index_time}'
/entity
/document
/dataConfig
I'm using win xp with apache – and jetty + solr 1.3.0
Thanks
--
--Noble Paul
--
--Noble Paul
Re: Newbie Question - getting search results from dataimport request handler
search for *:* and see if the index indeed has the documents . Once you ensure the docs are there go through the lucene query syntax and check your query On Tue, Nov 11, 2008 at 10:07 AM, Kevin Penny [EMAIL PROTECTED] wrote: Ok so I executed a: solr/dataimport?command=full-import then I checked here: solr/dataimport I get a good xml message (figure 1.1) showing me that 125 records have been indexed (good) and I know one of them contains the word 'job'. I sould get results from this query string then right (figure 1.0 is my result - 0 records found)? solr/select?q=job figure 1.0 response − lst name=responseHeader int name=status0/int int name=QTime0/int − lst name=params str name=qjob/str /lst /lst result name=response numFound=0 start=0/ /response figure 1.1 response − lst name=responseHeader int name=status0/int int name=QTime0/int /lst − lst name=initArgs − lst name=defaults str name=configdata-config.xml/str /lst /lst str name=statusidle/str str name=importResponse/ − lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched125/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-11-10 22:33:55/str − str name= Indexing completed. Added/Updated: 125 documents. Deleted 0 documents. /str str name=Committed2008-11-10 22:34:00/str str name=Optimized2008-11-10 22:34:00/str str name=Time taken 0:0:5.79/str /lst − str name=WARNING This response format is experimental. It is likely to change in the future. /str /response Kevin -Original Message- From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 10:30 PM To: solr-user@lucene.apache.org Subject: Re: Newbie Question - getting search results from dataimport request handler XML is just an intermediete data format Solr internally has no XML data. When the data comes out XML is just another representation of the same data. Whether you put in data using XML or DB (SQL) it all goes into the same index . Query must be done on that index using the syntax http://localhost:8983/solr/select/?q=your-query-goes-here On Tue, Nov 11, 2008 at 9:55 AM, Kevin Penny [EMAIL PROTECTED] wrote: Ok - and what would that be? (query interface) I need the URL format that would work in this situation to return data from my setup. I've gone through the tutorial and used execution strings like: http://localhost:8983/solr/select/?indent=onq=videosort=price+desc etc however I'm working with sql data and not xml data. Thanks -Original Message- From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 10:18 PM To: solr-user@lucene.apache.org Subject: Re: Newbie Question - getting search results from dataimport request handler you cannot query the DIH. It can only do indexing after indexing you must do the indexing on the regular query interface On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote: My Question is: what is the format of a search that will return data? i.e. /solr/select?q=developerqt=dataimport (won't work) nor will /solr/dataimport?q=developer (won't work) HTTP ERROR: 404 NOT_FOUND RequestURI=/solr/dataimport I have created a 'dataimport' set that contains data from a sql db. I can view meta data from this url: /solr/dataimport response − lst name=responseHeader int name=status0/int int name=QTime0/int /lst − lst name=initArgs − lst name=defaults str name=configdata-config.xml/str /lst /lst str name=statusidle/str str name=importResponse/ − lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched10/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-11-10 21:51:40/str str name=Time taken 0:0:4.594/str /lst − str name=WARNING This response format is experimental. It is likely to change in the future. /str /response I can verify that the data is there by going through /solr/admin/dataimport.jsp and doing 'verbose' true and debug now. It shows me the xml data set on the right as such: response − lst name=responseHeader int name=status0/int int name=QTime4594/int /lst − lst name=initArgs − lst name=defaults str name=configdata-config.xml/str /lst /lst str name=commandfull-import/str str name=modedebug/str − arr name=documents − arr − arr int87133/int /arr /arr − arr − arr int87134/int /arr /arr − arr − arr int87135/int /arr /arr − arr − arr int87136/int /arr /arr − arr − arr int87137/int /arr /arr − arr − arr int87138/int /arr /arr − arr − arr int87139/int /arr /arr − arr − arr int87140/int /arr /arr − arr − arr int87141/int /arr /arr − arr − arr int87142/int /arr /arr /arr − lst name=verbose-output − lst name=entity:item − lst name=document#1 − str name=query SELECT j.id , j.title , FROM
posting error in solr
I tried to post an xml file and returned me the below mentioned error: entity reference names can not start with character '\ufffd' (position: START_TAG seen ...ients with our topping service.lt;br /gt;\r\n\ufffd\u2f2\ufffd\u1b7\ufffd \ufffd\ufffd\u6b5\ufffd\ufffd\ufffd\ufffd\ubd6f\ufffd\ufffd\u5ae\ufffd \ufffd\ufffd\ufffd... @5167:22) org.xmlpull.v1.XmlPullParserException: entity reference names can not start with character '\ufffd' (position: START_TAG seen ...ients with our topping service.lt;br /gt;\r\n\ufffd\u2f2\ufffd\u1b7\ufffd \ufffd\ufffd\u6b5\ufffd\ufffd\ufffd\ufffd\ubd6f\ufffd\ufffd\u5ae\ufffd \ufffd\ufffd\ufffd... @5167:22) at org.xmlpull.mxp1.MXParser.parseEntityRef(MXParser.java:2212) at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1275) at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093) at org.xmlpull.mxp1.MXParser.nextText(MXParser.java:1058) at org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:332) at org.apache.solr.handler.XmlUpdateRequestHandler.update(XmlUpdateRequestHandler.java:162) at org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:84) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77) at org.apache.solr.core.SolrCore.execute(SolrCore.java:658) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:191) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:159) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection $RequestHandler.content(HttpConnection.java:835) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector $Connection.run(SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool $PoolThread.run(BoundedThreadPool.java:442) What could be the probable mistake done by me in my xml file, as I am unable to discover it? regards, Sushan Rungta
Re: Newbie Question - getting search results from dataimport request handler
Hi Kevin, You need to modify the schema which came with Solr to suit your data. There should be a schema.xml inside example/solr/conf directory. Once you do that, re-import your data. Take a look at http://wiki.apache.org/solr/SchemaXml On Tue, Nov 11, 2008 at 10:59 AM, Kevin Penny [EMAIL PROTECTED] wrote: I can execute: /solr/select?q=id:87133 So there is data there, however I have not defined any 'Fields' in my data-config and am hoping my column names are the 'fields', yet I'm not seeing any of them being returned in the 'doc' node below : response - lst name=responseHeader int name=status0/int int name=QTime0/int - lst name=params str name=qid:87133/str /lst /lst - result name=response numFound=1 start=0 - doc str name=id87133/str int name=popularity0/int str name=sku87133/str date name=timestamp2008-11-11T05:25:29Z/date /doc /result /response Kevin -Original Message- From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 11:23 PM To: solr-user@lucene.apache.org Subject: Re: Newbie Question - getting search results from dataimport request handler search for *:* and see if the index indeed has the documents . Once you ensure the docs are there go through the lucene query syntax and check your query On Tue, Nov 11, 2008 at 10:07 AM, Kevin Penny [EMAIL PROTECTED] wrote: Ok so I executed a: solr/dataimport?command=full-import then I checked here: solr/dataimport I get a good xml message (figure 1.1) showing me that 125 records have been indexed (good) and I know one of them contains the word 'job'. I sould get results from this query string then right (figure 1.0 is my result - 0 records found)? solr/select?q=job figure 1.0 response − lst name=responseHeader int name=status0/int int name=QTime0/int − lst name=params str name=qjob/str /lst /lst result name=response numFound=0 start=0/ /response figure 1.1 response − lst name=responseHeader int name=status0/int int name=QTime0/int /lst − lst name=initArgs − lst name=defaults str name=configdata-config.xml/str /lst /lst str name=statusidle/str str name=importResponse/ − lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched125/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-11-10 22:33:55/str − str name= Indexing completed. Added/Updated: 125 documents. Deleted 0 documents. /str str name=Committed2008-11-10 22:34:00/str str name=Optimized2008-11-10 22:34:00/str str name=Time taken 0:0:5.79/str /lst − str name=WARNING This response format is experimental. It is likely to change in the future. /str /response Kevin -Original Message- From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 10:30 PM To: solr-user@lucene.apache.org Subject: Re: Newbie Question - getting search results from dataimport request handler XML is just an intermediete data format Solr internally has no XML data. When the data comes out XML is just another representation of the same data. Whether you put in data using XML or DB (SQL) it all goes into the same index . Query must be done on that index using the syntax http://localhost:8983/solr/select/?q=your-query-goes-here On Tue, Nov 11, 2008 at 9:55 AM, Kevin Penny [EMAIL PROTECTED] wrote: Ok - and what would that be? (query interface) I need the URL format that would work in this situation to return data from my setup. I've gone through the tutorial and used execution strings like: http://localhost:8983/solr/select/?indent=onq=videosort=price+desc etc however I'm working with sql data and not xml data. Thanks -Original Message- From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 10:18 PM To: solr-user@lucene.apache.org Subject: Re: Newbie Question - getting search results from dataimport request handler you cannot query the DIH. It can only do indexing after indexing you must do the indexing on the regular query interface On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote: My Question is: what is the format of a search that will return data? i.e. /solr/select?q=developerqt=dataimport (won't work) nor will /solr/dataimport?q=developer (won't work) HTTP ERROR: 404 NOT_FOUND RequestURI=/solr/dataimport I have created a 'dataimport' set that contains data from a sql db. I can view meta data from this url: /solr/dataimport response − lst name=responseHeader int name=status0/int int name=QTime0/int /lst − lst name=initArgs − lst name=defaults str name=configdata-config.xml/str /lst /lst str name=statusidle/str str name=importResponse/ − lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total
Re: posting error in solr
I am using Solr Lucene - 2.0 regards, Sushan Rungta Mob: +91-9312098968 At 11:07 AM 11/11/2008, Shalin Shekhar Mangar wrote: Which version of Solr are you using? On Tue, Nov 11, 2008 at 11:00 AM, Sushan Rungta [EMAIL PROTECTED] wrote: I tried to post an xml file and returned me the below mentioned error: entity reference names can not start with character '\ufffd' (position: START_TAG seen ...ients with our topping service.lt;br /gt;\r\n\ufffd\u2f2\ufffd\u1b7\ufffd \ufffd\ufffd\u6b5\ufffd\ufffd\ufffd\ufffd\ubd6f\ufffd\ufffd\u5ae\ufffd \ufffd\ufffd\ufffd... @5167:22) org.xmlpull.v1.XmlPullParserException: entity reference names can not start with character '\ufffd' (position: START_TAG seen ...ients with our topping service.lt;br /gt;\r\n\ufffd\u2f2\ufffd\u1b7\ufffd \ufffd\ufffd\u6b5\ufffd\ufffd\ufffd\ufffd\ubd6f\ufffd\ufffd\u5ae\ufffd \ufffd\ufffd\ufffd... @5167:22) at org.xmlpull.mxp1.MXParser.parseEntityRef(MXParser.java:2212) at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1275) at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093) at org.xmlpull.mxp1.MXParser.nextText(MXParser.java:1058) at org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:332) at org.apache.solr.handler.XmlUpdateRequestHandler.update(XmlUpdateRequestHandler.java:162) at org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:84) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77) at org.apache.solr.core.SolrCore.execute(SolrCore.java:658) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:191) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:159) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection $RequestHandler.content(HttpConnection.java:835) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector $Connection.run(SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool $PoolThread.run(BoundedThreadPool.java:442) What could be the probable mistake done by me in my xml file, as I am unable to discover it? regards, Sushan Rungta -- Regards, Shalin Shekhar Mangar.
RE: Newbie Question - getting search results from dataimport request handler
Excellent! Thanks a bunch - that did the trick - all defined and my terms are being returned nicely - schema.xml was the ticket - not sure how I missed that in the docs. Kevin -Original Message- From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 11:35 PM To: solr-user@lucene.apache.org Subject: Re: Newbie Question - getting search results from dataimport request handler Hi Kevin, You need to modify the schema which came with Solr to suit your data. There should be a schema.xml inside example/solr/conf directory. Once you do that, re-import your data. Take a look at http://wiki.apache.org/solr/SchemaXml On Tue, Nov 11, 2008 at 10:59 AM, Kevin Penny [EMAIL PROTECTED] wrote: I can execute: /solr/select?q=id:87133 So there is data there, however I have not defined any 'Fields' in my data-config and am hoping my column names are the 'fields', yet I'm not seeing any of them being returned in the 'doc' node below : response - lst name=responseHeader int name=status0/int int name=QTime0/int - lst name=params str name=qid:87133/str /lst /lst - result name=response numFound=1 start=0 - doc str name=id87133/str int name=popularity0/int str name=sku87133/str date name=timestamp2008-11-11T05:25:29Z/date /doc /result /response Kevin -Original Message- From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 11:23 PM To: solr-user@lucene.apache.org Subject: Re: Newbie Question - getting search results from dataimport request handler search for *:* and see if the index indeed has the documents . Once you ensure the docs are there go through the lucene query syntax and check your query On Tue, Nov 11, 2008 at 10:07 AM, Kevin Penny [EMAIL PROTECTED] wrote: Ok so I executed a: solr/dataimport?command=full-import then I checked here: solr/dataimport I get a good xml message (figure 1.1) showing me that 125 records have been indexed (good) and I know one of them contains the word 'job'. I sould get results from this query string then right (figure 1.0 is my result - 0 records found)? solr/select?q=job figure 1.0 response − lst name=responseHeader int name=status0/int int name=QTime0/int − lst name=params str name=qjob/str /lst /lst result name=response numFound=0 start=0/ /response figure 1.1 response − lst name=responseHeader int name=status0/int int name=QTime0/int /lst − lst name=initArgs − lst name=defaults str name=configdata-config.xml/str /lst /lst str name=statusidle/str str name=importResponse/ − lst name=statusMessages str name=Total Requests made to DataSource1/str str name=Total Rows Fetched125/str str name=Total Documents Skipped0/str str name=Full Dump Started2008-11-10 22:33:55/str − str name= Indexing completed. Added/Updated: 125 documents. Deleted 0 documents. /str str name=Committed2008-11-10 22:34:00/str str name=Optimized2008-11-10 22:34:00/str str name=Time taken 0:0:5.79/str /lst − str name=WARNING This response format is experimental. It is likely to change in the future. /str /response Kevin -Original Message- From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 10:30 PM To: solr-user@lucene.apache.org Subject: Re: Newbie Question - getting search results from dataimport request handler XML is just an intermediete data format Solr internally has no XML data. When the data comes out XML is just another representation of the same data. Whether you put in data using XML or DB (SQL) it all goes into the same index . Query must be done on that index using the syntax http://localhost:8983/solr/select/?q=your-query-goes-here On Tue, Nov 11, 2008 at 9:55 AM, Kevin Penny [EMAIL PROTECTED] wrote: Ok - and what would that be? (query interface) I need the URL format that would work in this situation to return data from my setup. I've gone through the tutorial and used execution strings like: http://localhost:8983/solr/select/?indent=onq=videosort=price+desc etc however I'm working with sql data and not xml data. Thanks -Original Message- From: Noble Paul നോബിള് नोब्ळ् [mailto:[EMAIL PROTECTED] Sent: Monday, November 10, 2008 10:18 PM To: solr-user@lucene.apache.org Subject: Re: Newbie Question - getting search results from dataimport request handler you cannot query the DIH. It can only do indexing after indexing you must do the indexing on the regular query interface On Tue, Nov 11, 2008 at 9:45 AM, Kevin Penny [EMAIL PROTECTED] wrote: My Question is: what is the format of a search that will return data? i.e. /solr/select?q=developerqt=dataimport (won't work) nor will /solr/dataimport?q=developer (won't work) HTTP ERROR: 404 NOT_FOUND RequestURI=/solr/dataimport I have created a 'dataimport' set that
How to create Dynamic core?
Hi all, I want to create dynamic cores in my app. Case Study --- 1. I want to create the number of cores, that should be based on number of employees. 2. How can i create the dynamic core for each employee? kindly let me know, if u know the solution.. Thanks in advance Prabhu.K -- View this message in context: http://www.nabble.com/How-to-create-Dynamic-core--tp20434246p20434246.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to create Dynamic core?
are you using SolrJ?. then look at org.apache.solr.client.solrj.request.CoreAdminRequest Even if you are not using SolrJ the same commands can be issued over http On Tue, Nov 11, 2008 at 12:00 PM, RaghavPrabhu [EMAIL PROTECTED] wrote: Hi all, I want to create dynamic cores in my app. Case Study --- 1. I want to create the number of cores, that should be based on number of employees. 2. How can i create the dynamic core for each employee? kindly let me know, if u know the solution.. Thanks in advance Prabhu.K -- View this message in context: http://www.nabble.com/How-to-create-Dynamic-core--tp20434246p20434246.html Sent from the Solr - User mailing list archive at Nabble.com. -- --Noble Paul
Re: How to create Dynamic core?
Before rushing to create a large number of cores, think about de-normalizing your schema. Every core has an overhead. You probably do not need multiple cores at all. It would help if you tell us about the use-case so that we can recommend alternate solutions. On Tue, Nov 11, 2008 at 12:00 PM, RaghavPrabhu [EMAIL PROTECTED]wrote: Hi all, I want to create dynamic cores in my app. Case Study --- 1. I want to create the number of cores, that should be based on number of employees. 2. How can i create the dynamic core for each employee? kindly let me know, if u know the solution.. Thanks in advance Prabhu.K -- View this message in context: http://www.nabble.com/How-to-create-Dynamic-core--tp20434246p20434246.html Sent from the Solr - User mailing list archive at Nabble.com. -- Regards, Shalin Shekhar Mangar.
