RE: Problems with SOLR Collection Distribution
Hi, When i changed the previous entries to the following, Strangely i got the error=3 this time... Anybody aware of what are these error codes 2,3 corresponds to? Is there any links for the same when one can get some info about these errors? scripts.conf user=jetty-6.1.6 solr_hostname=localhost solr_port=80 rsyncd_port=10080 data_dir=/opt/solr/data/data_tenantID_1 webapp_name=solr master_host=localhost master_data_dir=/opt/solr/data master_status_dir=/opt/solr/logs entry for postCommit: listener event=postCommit class=solr.RunExecutableListener str name=exe/opt/solr/bin/snapshooter/str str name=dirsolr/bin/str bool name=waittrue/bool arr name=argsstr-u jetty-6.1.6/str str-d /opt/solr/data/str/arr arr name=env /arr /listener The stack trace is as follows: INFO: start commit(optimize=false,waitFlush=false,waitSearcher=true) Dec 13, 2007 6:12:19 PM org.apache.solr.update.DirectUpdateHandler2 doDeletions INFO: DirectUpdateHandler2 deleting and removing dups for 1 ids Dec 13, 2007 6:12:19 PM org.apache.solr.search.SolrIndexSearcher init INFO: Opening [EMAIL PROTECTED] DirectUpdateHandler2 Dec 13, 2007 6:12:19 PM org.apache.solr.update.DirectUpdateHandler2 doDeletions INFO: DirectUpdateHandler2 docs deleted=1 Dec 13, 2007 6:12:19 PM org.apache.solr.common.SolrException log SEVERE: java.io.IOException: CreateProcess: \opt\solr\bin\snapshooter -u jetty-6.1.6 -d /opt/solr/data error=3 at java.lang.ProcessImpl.create(Native Method) at java.lang.ProcessImpl.init(ProcessImpl.java:81) at java.lang.ProcessImpl.start(ProcessImpl.java:30) at java.lang.ProcessBuilder.start(ProcessBuilder.java:451) at java.lang.Runtime.exec(Runtime.java:591) at org.apache.solr.core.RunExecutableListener.exec(RunExecutableListener.java:7 3) at org.apache.solr.core.RunExecutableListener.postCommit(RunExecutableListener. java:100) at org.apache.solr.update.UpdateHandler.callPostCommitCallbacks(UpdateHandler.j ava:100) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java :550) at com.plateau.psa.impl.EmbeddedSolr.commit(EmbeddedSolr.java:1008) at com.plateau.psa.impl.EmbeddedSolr.updateDocuments(EmbeddedSolr.java:877) -Original Message- From: Dilip.TS [mailto:[EMAIL PROTECTED] Sent: Thursday, December 13, 2007 5:44 PM To: solr-user@lucene.apache.org Subject: RE: Problems with SOLR Collection Distribution Hi, I am using the Jetty-6.1.6 server for the Embedded Solr. I made the following entries as follows: scripts.conf user=jetty-6.1.6 solr_hostname=localhost solr_port=5000 rsyncd_port=15000 data_dir=solr/data/data_tenantID_1 webapp_name=solr master_host=localhost master_data_dir=solr/data master_status_dir=solr/log and the corresponding entry for postCommit listener event=postCommit class=solr.RunExecutableListener str name=exesnapshooter/str str name=dirsolr/bin/str bool name=waitfalse/bool arr name=args str-u jetty-6.1.6/str str-d /solr/data/str /arr arr name=env /arr /listener Iam getting the following error.Am i missing something? INFO: Opening [EMAIL PROTECTED] DirectUpdateHandler2 Dec 13, 2007 5:39:57 PM org.apache.solr.update.DirectUpdateHandler2 doDeletions INFO: DirectUpdateHandler2 docs deleted=1 Dec 13, 2007 5:39:57 PM org.apache.solr.common.SolrException log SEVERE: java.io.IOException: CreateProcess: snapshooter -u jetty-6.1.6 -d /solr/data error=2 at java.lang.ProcessImpl.create(Native Method) at java.lang.ProcessImpl.init(ProcessImpl.java:81) at java.lang.ProcessImpl.start(ProcessImpl.java:30) at java.lang.ProcessBuilder.start(ProcessBuilder.java:451) at java.lang.Runtime.exec(Runtime.java:591) at org.apache.solr.core.RunExecutableListener.exec(RunExecutableListener.java:7 3) at org.apache.solr.core.RunExecutableListener.postCommit(RunExecutableListener. java:100) at org.apache.solr.update.UpdateHandler.callPostCommitCallbacks(UpdateHandler.j ava:100) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java :550) at com.plateau.psa.impl.EmbeddedSolr.commit(EmbeddedSolr.java:1008) at com.plateau.psa.impl.EmbeddedSolr.updateDocuments(EmbeddedSolr.java:877) -Original Message- From: Dilip.TS [mailto:[EMAIL PROTECTED] Sent: Thursday, December 13, 2007 4:50 PM To: SOLR Subject: Problems with SOLR Collection Distribution Hi, Iam trying to establish the SOLR Collection Distribution for my application using Embedded Solr. I have made the following entries in the scripts.conf file and included the entries for the postCommit and postOptimize operation in the solrconfig.xml file as follows: scripts.conf user=123 solr_hostname=localhost solr_port=8983 rsyncd_port=18983 data_dir=C:\solr\data\data_tenantID_1 webapp_name=solr master_host=192.111.222.555
embeddedsolr and solrj index update request
Hi all, i am using embeddedsolr and solrj to create and update a large index. however even after adding documents and commiting documents i.e. server.add(solrDocs); server.commit(); i am not able to see the new documents in search results. I have to restart the server and run the same search to see the results. is there anything i am doing wrong here? Thanks in advance for your help Sandeep This email is confidential and may also be privileged. If you are not the intended recipient please notify us immediately by telephoning +44 (0)20 7840 4300 or email [EMAIL PROTECTED] You should not copy it or use it for any purpose nor disclose its contents to any other person. Touch Local cannot accept liability for statements made which are clearly the sender's own and are not made on behalf of the firm. Touch Local Limited Registered Number: 2885607 VAT Number: GB896112114 9th Floor, 89 Albert Embankment, London SE1 7TP +44 (0)20 7840 4300
Re: SOLR X FAST
:) Ok, ok. I don´t think that this kind of investment($$$) so incredibly hilarious :) Maybe I could ask Why use Solr and not use FAST ?. It is a really big diference :) But I think you are correct. Sorry . William. On Dec 13, 2007 3:28 AM, Chris Hostetter [EMAIL PROTECTED] wrote: : Why use FAST and not use SOLR ? For example. : What will FAST offer that will justify the investment ? Am I the only one that finds these questions incredibly hilarious? particularly on this list? You should also email FAST customer service and ask them Why use Solr and not use FAST ? :) -Hoss
Re: Solr and Flex
On Dec 13, 2007, at 10:42 AM, jenix wrote: I'm using Flex for the frontend interface and Solr on backend for the search engine. I'm new to Flex and Flash and thought someone might have some code integrating the two. We've done light stuff querying solr w/ actionscript. It is pretty simple, you form your query as a url, get the url and then use AS's built in xml parser to get whatever you need. Haven't tried posting documents.
how do do most efficient: collapsing facets into top-N results
I've subclassed StandardRequestHandler to be able to show top-N results for
some of the facet-values that I'm interested in. The functionality resembles
the solr-236 field collapsing a bit, with the difference that I can
arbitrarily specify which facet-query to collapse and to what extend.
(possibility to specify N independently)
The code for this is now quite simple, but (maybe because of that) I've got
the feeling that it can be optimized quite a bit. The question is how?
first some explanation and code:
I extended the standardrequesthandler and execute
super.handleRequestBody(req,rsp) to be able to fetch the facetquery results.
From that I copy the facets that I wish to collapse to a NamedList
facet_results and execute code (see below) that basically splits a (possibly
combined) facetquery into independent queries which are added to a FQ-list.
That FQ-list is appended to the original query and FQ-list and the new query
is executed.
for(int i = 0; i facetresults.size(); i++)
{
ListQuery fqList = new ArrayListQuery();
String[] fqsplit = facetresults.getName(i).split([+]);
for(int j = 0; j fqsplit.length; j++)
{
Query fqNew = QueryParsing.parseQuery(fqsplit[j].trim(),
req.getSchema());
fqList.add(fqNew);
}
fqList.addAll(fqsExisting);
DocListAndSet resultList = new DocListAndSet();
SolrIndexSearcher s = req.getSearcher();
resultList.docList = s.getDocList(query,fqList, sort,start, rows ,0);
NamedList facetValue = new SimpleOrderedMap();
facetValue.add(results,resultList.docList);
facetresults.setVal(i, facetValue);
}
This all works okay, but I'm still thinking that there must be a better way
than executing queries over and over again, for which only the fq's are
different: Q and Sort are the same for the executed queries per facet as for
the same already exectuted overall query.
Obviously doing a intersect on the original result would by far be the
fastest solution but Mike mentioned that this wasn't doable, since the
overall sorted resultlist is not available. see:
http://www.nabble.com/showing-results-per-facet-value-efficiently-to13133815.html
Is there anything else I can do to speedup the queries?
for reference I'm now seeing 15-16ms for each exectued query which is not in
the query-cache.
This seems independent whether of not Fq's are already in the filtercache or
not, which strikes me as odd.
For example see the performance measure of the collapsed facet-queries below
(and make up 1 call to Solr). Tested on an unwarmed solr-server. 20.000
docs. intel Core 2 Duo 2ghz. 800 MB Ram assigned to Solr.
15 : ms for: _ddp_p_dc_dc_2_dc_dc:[0 TO 50]
16 : ms for: _ddp_p_dc_dc_2_dc_dc:[51 TO 100]
16 : ms for: _ddp_p_dc_dc_2_dc_dc:[101 TO 200]
15 : ms for: _ddp_p_dc_dc_2_dc_dc:[201 TO 300]
16 : ms for: idA:2140479
15 : ms for: idA:1456928
16 : ms for: idA:2601889
0 : ms for: _ddp_p_dc_dc_2_dc_dc:[0 TO 50]
0 : ms for: _ddp_p_dc_dc_2_dc_dc:[51 TO 100]
0 : ms for: _ddp_p_dc_dc_2_dc_dc:[101 TO 200]
0 : ms for: _ddp_p_dc_dc_2_dc_dc:[201 TO 300]
15 : ms for: _ddp_p_dc_dc_2_dc_dc:[0 TO 50] + idA:2140479
16 : ms for: _ddp_p_dc_dc_2_dc_dc:[0 TO 50] + idA:1456928
16 : ms for: _ddp_p_dc_dc_2_dc_dc:[0 TO 50] + idA:2601889
15 : ms for: _ddp_p_dc_dc_2_dc_dc:[51 TO 100] + idA:2140479
16 : ms for: _ddp_p_dc_dc_2_dc_dc:[51 TO 100] + idA:1456928
15 : ms for: _ddp_p_dc_dc_2_dc_dc:[51 TO 100] + idA:2601889
16 : ms for: _ddp_p_dc_dc_2_dc_dc:[101 TO 200] + idA:2140479
16 : ms for: _ddp_p_dc_dc_2_dc_dc:[101 TO 200] + idA:1456928
15 : ms for: _ddp_p_dc_dc_2_dc_dc:[101 TO 200] + idA:2601889
16 : ms for: _ddp_p_dc_dc_2_dc_dc:[201 TO 300] + idA:2140479
16 : ms for: _ddp_p_dc_dc_2_dc_dc:[201 TO 300] + idA:1456928
15 : ms for: _ddp_p_dc_dc_2_dc_dc:[201 TO 300] + idA:2601889
The strange thing here is that for example the query:
_ddp_p_dc_dc_2_dc_dc:[0 TO 50] + idA:2140479
takes 15 ms
although it's independent parts:
- _ddp_p_dc_dc_2_dc_dc:[0 TO 50]
- idA:2140479
have already been executed (they also take 15/16 ms)
so all FQ's for _ddp_p_dc_dc_2_dc_dc:[0 TO 50] + idA:2140479 must be in the
filter-cache and hence the query must execute quicker than the very first
query:
_ddp_p_dc_dc_2_dc_dc:[0 TO 50] for which the FQ wasn't in the filter-cache
at that moment.
So to summarize my 2 questions:
1. is there any way to get better performance for what 'm trying to achieve?
Perhaps a custom hitcollector or something?
2. do you have any explanation for the fact the the filter-cache doens't
seem to matter for executing the queries?
Thanks in advance for making it to the end of this post and for any help you
might give me ;-)
Geert-Jan
--
View this message in context:
http://www.nabble.com/how-do-do-most-efficient%3A-collapsing-facets-into-top-N-results-tp14318577p14318577.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr and Flex
I'm using Flex for the frontend interface and Solr on backend for the search engine. I'm new to Flex and Flash and thought someone might have some code integrating the two. -- View this message in context: http://www.nabble.com/Solr-and-Flex-tp14284703p14318528.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr and Flex
Do you have any code to share? Did you try the json option? Any performance differences or tips you'll like to share? Thank you. Jenny -- View this message in context: http://www.nabble.com/Solr-and-Flex-tp14284703p14319008.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: embeddedsolr and solrj index update request
What version solr are you running? Do you have an updateRequestJandler registered to /update? On Dec 13, 2007, at 8:36 AM, Sandeep Shetty [EMAIL PROTECTED] wrote: Hi all, i am using embeddedsolr and solrj to create and update a large index. however even after adding documents and commiting documents i.e. server.add(solrDocs); server.commit(); i am not able to see the new documents in search results. I have to restart the server and run the same search to see the results. is there anything i am doing wrong here? Thanks in advance for your help Sandeep This email is confidential and may also be privileged. If you are not the intended recipient please notify us immediately by telephoning +44 (0)20 7840 4300 or email [EMAIL PROTECTED] You should not copy it or use it for any purpose nor disclose its contents to any other person. Touch Local cannot accept liability for statements made which are clearly the sender's own and are not made on behalf of the firm. Touch Local Limited Registered Number: 2885607 VAT Number: GB896112114 9th Floor, 89 Albert Embankment, London SE1 7TP +44 (0)20 7840 4300
Re: Solr 1.3 expected release date
On Wed, 12 Dec 2007 20:04:00 -0500
Norskog, Lance [EMAIL PROTECTED] wrote:
... SOLR-303 (Distributed Search over HTTP)...
Woo-hoo!
hear hear!!!
_
{Beto|Norberto|Numard} Meijome
Your reasoning is excellent -- it's only your basic assumptions that are wrong.
I speak for myself, not my employer. Contents may be hot. Slippery when wet.
Reading disclaimers makes you go blind. Writing them is worse. You have been
Warned.
Newbie question about Solr use in web applications
I'm trying to wrap my head around the best practices way of correlating Solr documents with database items. Let's say I have a database containing people, groups, and projects (these all have different fields). I want to index these different kinds of objects with a view to eventually present search results from all three types mashed together and sorted by relevance. Using separate indices (and thus separate Solr processes) would make mashing the results together very difficult so I'm guessing I just add the separate fields to the schema along with an 'object_type' field or equivalent? Secondly should I just store the database row id for each object (while still indexing the field contents) so a query on the index returns a list of id's that I can then fetch from the database? Thanks! -- View this message in context: http://www.nabble.com/Newbie-question-about-Solr-use-in-web-applications-tp14328925p14328925.html Sent from the Solr - User mailing list archive at Nabble.com.
