Re: DataImportHandler: regex debugging
Jon,
looks like you're (just) missing the transformer=RegexTransformer in
your entity-definition, like documented here:
http://wiki.apache.org/solr/DataImportHandler#RegexTransformer
Regards
Stefan
On Wed, Feb 9, 2011 at 9:16 PM, Jon Drukman j...@cluttered.com wrote:
I am trying to use the regex transformer but it's not returning anything.
Either my regex is wrong, or I've done something else wrong in the setup of
the
entity. Is there any way to debug this? Making a change and waiting 7
minutes
to reindex the entity sucks.
entity name=boxshot
query=SELECT GROUP_CONCAT(i.url, ',') boxshot_url,
GROUP_CONCAT(i2.url, ',') boxshot_url_small FROM games g
left join image_sizes i ON g.box_image_id = i.id AND i.size_type = 39
left join image_sizes i2 on g.box_image_id = i2.id AND i2.size_type =
40
WHERE g.game_seo_title = '${game.game_seo_title}'
GROUP BY g.game_seo_title
field name=main_image regex=^(.*?), sourceColName=boxshot_url /
field name=small_image regex=^(.*?),
sourceColName=boxshot_url_small /
/entity
This returns columns that are either null, or have some comma-separated
strings.
I want the bit up to the first comma, if it exists.
Ideally I could have it log the query and the input/output
of the field statements.
Tomcat6 and Log4j
Hi,
I added “slf4j-log4j12-1.5.5.jar” and “log4j-1.2.15.jar” to
$CATALINA_HOME/webapps/solr/WEB-INF/lib ,
then deleted the library “slf4j-jdk14-1.5.5.jar” from
$CATALINA_HOME/webapps/solr/WEB-INF/lib,
then created a directory $CATALINA_HOME/webapps/solr/WEB-INF/classes.
and created $CATALINA_HOME/webapps/solr/WEB-INF/classes/log4j.properties
with the following contents :
log4j.rootLogger=INFO
log4j.appender.SOLR.logfile=org.apache.log4j.DailyRollingFileAppender
log4j.appender.SOLR.logfile.file=/home/quetelet_bdq/logs/bdq.log
log4j.appender.SOLR.logfile.DatePattern='.'-MM-dd
log4j.appender.SOLR.logfile.layout=org.apache.log4j.PatternLayout
log4j.appender.SOLR.logfile.layout.conversionPattern=%d %p [%c{3}] -
[%t] - %X{ip}: %m%n
log4j.appender.SOLR.logfile = true
I restarted solr and I got the following message in the catalina.out log :
log4j:WARN No appenders could be found for logger
(org.apache.solr.core.SolrResourceLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
What is told on this page is that this error occurs what the
log4j.properties isn't found.
Could someone help me to have it working ?
Thanks in advance,
Xavier
Re: solr render biased search result
You can do a lot with function queries. Only you know what the domain specific requirements are, so you should write application layer code to modify the Solr query based on the user profile to the one searching. Example for the 1950 movie lover you could do: q=goo bardefType=dismaxbf=map(movieyear,1950,1959,1,0)^1000.0 Or you could use a recip(abs(sub(movieyear,1955)),10,100,10)^10.0 to give more boost to the move the closer to 1955 it is. However, sometimes it is easier to pre-calculate some field values in the index and filter/boost on those. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 9. feb. 2011, at 20.44, cyang2010 wrote: Hi, I am asked that whether solr renders biased search result? For example, for this search (query all movie title by this Comedy genre), for user who indicates a preference to 1950's movies, solr renders the 1950's movies with higher score (top in the list)?Or if user is a kid, then the result will render G/PG rated movie top in the list, and render all the R rated movie bottom in the list? I know that solr can boost score based on match on a particular field. But it can't favor some value over other value in the same field. is that right? -- View this message in context: http://lucene.472066.n3.nabble.com/solr-render-biased-search-result-tp2461155p2461155.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Tomcat6 and Log4j
Have you tried to start Tomcat with
-Dlog4j.configuration=$CATALINA_HOME/webapps/solr/WEB-INF/classes/log4j.properties
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
On 10. feb. 2011, at 09.41, Xavier Schepler wrote:
Hi,
I added “slf4j-log4j12-1.5.5.jar” and “log4j-1.2.15.jar” to
$CATALINA_HOME/webapps/solr/WEB-INF/lib ,
then deleted the library “slf4j-jdk14-1.5.5.jar” from
$CATALINA_HOME/webapps/solr/WEB-INF/lib,
then created a directory $CATALINA_HOME/webapps/solr/WEB-INF/classes.
and created $CATALINA_HOME/webapps/solr/WEB-INF/classes/log4j.properties with
the following contents :
log4j.rootLogger=INFO
log4j.appender.SOLR.logfile=org.apache.log4j.DailyRollingFileAppender
log4j.appender.SOLR.logfile.file=/home/quetelet_bdq/logs/bdq.log
log4j.appender.SOLR.logfile.DatePattern='.'-MM-dd
log4j.appender.SOLR.logfile.layout=org.apache.log4j.PatternLayout
log4j.appender.SOLR.logfile.layout.conversionPattern=%d %p [%c{3}] - [%t] -
%X{ip}: %m%n
log4j.appender.SOLR.logfile = true
I restarted solr and I got the following message in the catalina.out log :
log4j:WARN No appenders could be found for logger
(org.apache.solr.core.SolrResourceLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
What is told on this page is that this error occurs what the log4j.properties
isn't found.
Could someone help me to have it working ?
Thanks in advance,
Xavier
Solr Cloud - shard replication
Hi all,
I'm attempting to set up a simple Solr Cloud, right now almost directly from
the tutorial at: http://wiki.apache.org/solr/SolrCloud
I'm attempting to set up a simple 1 shard cloud, across two servers. I'm not
sure I understand the architecture behind this, but what I'm after is two
copies of a single shard, so that if a single server goes down I will still
have a full index available.
I have two servers set up with 1 shard between them, however when I load data
(from exampledocs) into one of the servers and then run a distributed search,
when the server I load data into handles the query I get a result, but if the
other server handles it I get nothing. All queries are run using the URL for
one server, but I can see from the CLI output which server is actually handling
the request.
I'm working on the latest nightly build (apache-solr-4.0-2011-02-10_08-27-09),
although I've tried a couple of different nightly builds over the last week or
so with the same effect.
My solr/solr.xml contains:
solr persistent=false
cores adminPath=/admin/cores defaultCoreName=collection1
core name=collection1 instanceDir=. shard=shard1 /
/cores
/solr
If someone could point out what I'm doing wrong or where I need to look to
correct my configuration that'd be fantastic.
Thanks in advance.
--
Hayden
#!/usr/bin/perl
chop($_=);@s=split/ /;foreach$m(@s){if($m=='*'){$z=pop@t;$x=
pop@t;$a=eval$x$m$z;push@t,$a;}else{push@t,$m;}}print$a\n;
__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
__
Replication and newSearcher registerd poll interval
Hi, If the replication window is too small to allow a new searcher to warm and close the current searcher before the new one needs to be in place, then the slaves continuously has a high load, and potentially an OOM error. we've noticed this in our environment where we have several facets on large multivalued fields. I was wondering what the list though about modifying the replication process to skip polls (though warning to logs) when there is a searcher in the process of warming? Else as in our case it brings the slave to it's knees, workaround was to extend the poll interval, though not ideal. Cheers, Dan
Re: Tomcat6 and Log4j
Thanks for your response.
How could I do that ?
From: Jan Høydahl jan@cominvent.com
Sent: Thu Feb 10 11:01:15 CET 2011
To: solr-user@lucene.apache.org
Subject: Re: Tomcat6 and Log4j
Have you tried to start Tomcat with
-Dlog4j.configuration=$CATALINA_HOME/webapps/solr/WEB-INF/classes/log4j.properties
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
On 10. feb. 2011, at 09.41, Xavier Schepler wrote:
Hi,
I added “slf4j-log4j12-1.5.5.jar” and “log4j-1.2.15.jar” to
$CATALINA_HOME/webapps/solr/WEB-INF/lib ,
then deleted the library “slf4j-jdk14-1.5.5.jar” from
$CATALINA_HOME/webapps/solr/WEB-INF/lib,
then created a directory $CATALINA_HOME/webapps/solr/WEB-INF/classes.
and created $CATALINA_HOME/webapps/solr/WEB-INF/classes/log4j.properties
with the following contents :
log4j.rootLogger=INFO
log4j.appender.SOLR.logfile=org.apache.log4j.DailyRollingFileAppender
log4j.appender.SOLR.logfile.file=/home/quetelet_bdq/logs/bdq.log
log4j.appender.SOLR.logfile.DatePattern='.'-MM-dd
log4j.appender.SOLR.logfile.layout=org.apache.log4j.PatternLayout
log4j.appender.SOLR.logfile.layout.conversionPattern=%d %p [%c{3}] - [%t] -
%X{ip}: %m%n
log4j.appender.SOLR.logfile = true
I restarted solr and I got the following message in the catalina.out log :
log4j:WARN No appenders could be found for logger
(org.apache.solr.core.SolrResourceLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
What is told on this page is that this error occurs what the
log4j.properties isn't found.
Could someone help me to have it working ?
Thanks in advance,
Xavier
--
Tous les courriers électroniques émis depuis la messagerie
de Sciences Po doivent respecter des conditions d'usages.
Pour les consulter rendez-vous sur
http://www.ressources-numeriques.sciences-po.fr/confidentialite_courriel.htm
Re: Tomcat6 and Log4j
Add it to the CATALINA_OPTS, on Debian systems you could edit /etc/default/tomcat On Thursday 10 February 2011 12:27:59 Xavier SCHEPLER wrote: -Dlog4j.configuration=$CATALINA_HOME/webapps/solr/WEB-INF/classes/log4j.pr operties -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350
Re: Tomcat6 and Log4j
I added it to /etc/default/tomcat6. What happened is that the same error message appeared twice in /var/log/tomcat6/catalina.out. Like the same file was loaded twice. -- Tous les courriers électroniques émis depuis la messagerie de Sciences Po doivent respecter des conditions d'usages. Pour les consulter rendez-vous sur http://www.ressources-numeriques.sciences-po.fr/confidentialite_courriel.htm
Re: Tomcat6 and Log4j
Oh, now looking at your log4j.properties, i believe it's wrong. You declared
INFO as rootLogger but you use SOLR.
-log4j.rootLogger=INFO
+log4j.rootLogger=SOLR
try again
On Thursday 10 February 2011 09:41:29 Xavier Schepler wrote:
Hi,
I added “slf4j-log4j12-1.5.5.jar” and “log4j-1.2.15.jar” to
$CATALINA_HOME/webapps/solr/WEB-INF/lib ,
then deleted the library “slf4j-jdk14-1.5.5.jar” from
$CATALINA_HOME/webapps/solr/WEB-INF/lib,
then created a directory $CATALINA_HOME/webapps/solr/WEB-INF/classes.
and created $CATALINA_HOME/webapps/solr/WEB-INF/classes/log4j.properties
with the following contents :
log4j.rootLogger=INFO
log4j.appender.SOLR.logfile=org.apache.log4j.DailyRollingFileAppender
log4j.appender.SOLR.logfile.file=/home/quetelet_bdq/logs/bdq.log
log4j.appender.SOLR.logfile.DatePattern='.'-MM-dd
log4j.appender.SOLR.logfile.layout=org.apache.log4j.PatternLayout
log4j.appender.SOLR.logfile.layout.conversionPattern=%d %p [%c{3}] -
[%t] - %X{ip}: %m%n
log4j.appender.SOLR.logfile = true
I restarted solr and I got the following message in the catalina.out log :
log4j:WARN No appenders could be found for logger
(org.apache.solr.core.SolrResourceLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
What is told on this page is that this error occurs what the
log4j.properties isn't found.
Could someone help me to have it working ?
Thanks in advance,
Xavier
--
Markus Jelsma - CTO - Openindex
http://www.linkedin.com/in/markus17
050-8536620 / 06-50258350
Re: Tomcat6 and Log4j
Oh, and for sharing purposes; we use a configuration like this one. It'll have
an info and error log and stores them next to Tomcat's own logs in
/var/log/tomat on Debian systems (or whatever catalina.base is on other
distros).
log4j.rootLogger=DEBUG, info, error
log4j.appender.info=org.apache.log4j.DailyRollingFileAppender
log4j.appender.info.Threshold=INFO
log4j.appender.info.MaxFileSize=500KB
log4j.appender.info.MaxBackupIndex=20
log4j.appender.info.Append=true
log4j.appender.info.File=${catalina.base}/logs/info.log
log4j.appender.info.DatePattern='.'-MM-dd'.log'
log4j.appender.info.layout=org.apache.log4j.PatternLayout
log4j.appender.info.layout.ConversionPattern=%d %p [%c{3}] - [%t] - %X{ip}:
%m%n
log4j.appender.error=org.apache.log4j.DailyRollingFileAppender
log4j.appender.error.Threshold=ERROR
log4j.appender.error.File=${catalina.base}/logs/error.log
log4j.appender.error.DatePattern='.'-MM-dd'.log'
log4j.appender.error.layout=org.apache.log4j.PatternLayout
log4j.appender.error.layout.ConversionPattern=%d %p [%c{3}] - [%t] - %X{ip}:
%m%n
log4j.appender.error.MaxFileSize=500KB
log4j.appender.error.MaxBackupIndex=20
On Thursday 10 February 2011 12:51:13 Markus Jelsma wrote:
Oh, now looking at your log4j.properties, i believe it's wrong. You
declared INFO as rootLogger but you use SOLR.
-log4j.rootLogger=INFO
+log4j.rootLogger=SOLR
try again
On Thursday 10 February 2011 09:41:29 Xavier Schepler wrote:
Hi,
I added “slf4j-log4j12-1.5.5.jar” and “log4j-1.2.15.jar” to
$CATALINA_HOME/webapps/solr/WEB-INF/lib ,
then deleted the library “slf4j-jdk14-1.5.5.jar” from
$CATALINA_HOME/webapps/solr/WEB-INF/lib,
then created a directory $CATALINA_HOME/webapps/solr/WEB-INF/classes.
and created $CATALINA_HOME/webapps/solr/WEB-INF/classes/log4j.properties
with the following contents :
log4j.rootLogger=INFO
log4j.appender.SOLR.logfile=org.apache.log4j.DailyRollingFileAppender
log4j.appender.SOLR.logfile.file=/home/quetelet_bdq/logs/bdq.log
log4j.appender.SOLR.logfile.DatePattern='.'-MM-dd
log4j.appender.SOLR.logfile.layout=org.apache.log4j.PatternLayout
log4j.appender.SOLR.logfile.layout.conversionPattern=%d %p [%c{3}] -
[%t] - %X{ip}: %m%n
log4j.appender.SOLR.logfile = true
I restarted solr and I got the following message in the catalina.out log
:
log4j:WARN No appenders could be found for logger
(org.apache.solr.core.SolrResourceLoader).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
What is told on this page is that this error occurs what the
log4j.properties isn't found.
Could someone help me to have it working ?
Thanks in advance,
Xavier
--
Markus Jelsma - CTO - Openindex
http://www.linkedin.com/in/markus17
050-8536620 / 06-50258350
Re: Tomcat6 and Log4j
Yes thanks. This works fine :
log4j.rootLogger=INFO, SOLR
log4j.appender.SOLR=org.apache.log4j.DailyRollingFileAppender
log4j.appender.SOLR.file=/home/quetelet_bdq/logs/bdq.log
log4j.appender.SOLR.datePattern='.'-MM-dd
log4j.appender.SOLR.layout=org.apache.log4j.PatternLayout
log4j.appender.SOLR.layout.conversionPattern=%d %p [%c{3}] - [%t] - %X{ip}: %m%n
--
Tous les courriers électroniques émis depuis la messagerie
de Sciences Po doivent respecter des conditions d'usages.
Pour les consulter rendez-vous sur
http://www.ressources-numeriques.sciences-po.fr/confidentialite_courriel.htm
Re: Solr Cloud - shard replication
Hi,
SolrCloud does not currently handle the indexing side at all. So you'll need to
set up replication to tell Solr that node B should be a replica of node A.
http://wiki.apache.org/solr/SolrReplication
After you do this, you can push a document to node A, wait a minute to let it
replicate to node B, and then do a distributed (load balanced) query:
http://nodeA:8983/solr/collection1/selectq=foo bardistrib=true
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
On 10. feb. 2011, at 11.32, Hayden Stainsby wrote:
Hi all,
I'm attempting to set up a simple Solr Cloud, right now almost directly from
the tutorial at: http://wiki.apache.org/solr/SolrCloud
I'm attempting to set up a simple 1 shard cloud, across two servers. I'm not
sure I understand the architecture behind this, but what I'm after is two
copies of a single shard, so that if a single server goes down I will still
have a full index available.
I have two servers set up with 1 shard between them, however when I load data
(from exampledocs) into one of the servers and then run a distributed search,
when the server I load data into handles the query I get a result, but if the
other server handles it I get nothing. All queries are run using the URL for
one server, but I can see from the CLI output which server is actually
handling the request.
I'm working on the latest nightly build
(apache-solr-4.0-2011-02-10_08-27-09), although I've tried a couple of
different nightly builds over the last week or so with the same effect.
My solr/solr.xml contains:
solr persistent=false
cores adminPath=/admin/cores defaultCoreName=collection1
core name=collection1 instanceDir=. shard=shard1 /
/cores
/solr
If someone could point out what I'm doing wrong or where I need to look to
correct my configuration that'd be fantastic.
Thanks in advance.
--
Hayden
#!/usr/bin/perl
chop($_=);@s=split/ /;foreach$m(@s){if($m=='*'){$z=pop@t;$x=
pop@t;$a=eval$x$m$z;push@t,$a;}else{push@t,$m;}}print$a\n;
__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
__
Sharing a schema with multiple cores
Hi, I'm using Solr 1.4.1 and trying to share a schema among two cores. Here is what I did : solr.xml : solr persistent=false cores adminPath=/admin/cores defaultCoreName=prod shareSchema=true core name=prod instanceDir=prod schemaconf/schema.xml/schema /core core name=test instanceDir=test schemaconf/schema.xml/schema /core /cores /solr Then in my solr.home (where solr.xml lives) I created a conf directory and put schema.xml inside. But it doesn't work and Solr is complaining about missing schema.xml (Can't find resource 'schema.xml'). If I put schema.xml into test/conf and prod/conf it does work... So what am I missing here? Thanks, Marc.
Re: SolrCloud Feedback
On Jan 20, 2011, at 12:49 AM, Grijesh wrote: Hi Mark, I was just working on SolrCloud for my RD and I got a question in my Mind. Since in SolrCloud the configuration files are being shared on all Cloud instances and If I have different configuration files for different cores then how can I manage it by my Zookeeper managed SolrCloud. - Thanx: Grijesh You can create as many configuration sets as you want - then you just set on the collection zk node which set of config files should be used. On the Cloud wiki you will see: -Dbootstrap_confdir=./solr/conf -Dcollection.configName=myconf That will upload the config files in ./solr/conf to a config set named myconf. When there is only one config set, that is what will be used - but when there is more than one, you can set which config set to use on the collection node. - Mark Miller lucidimagination.com
solr.xml isn't loaded from classpath?
(may have double posted...apologies if it is) It seems like when solr home is absent, Solr makes an attempt to look a few other places to load its configuration. It will try to look for solrconfig.xml on the classpath as well. It doesn't seem like it makes any attempt to find solr.xml though. Why is that? Read below for the larger narrative... The gory details: Having this configuration discovery makes things really convenient for creating custom Solr web applications where you can throw all of Solr's config in your resources, create a war, deploy it to Tomcat and it happily loads. No setting of environment variables or setup required. Something like this, /someapp/src/main/resources |-solrconfig.xml |-schema.xml |-etc. The same approach is outlined here: http://netsuke.wordpress.com/2010/06/24/launching-solr-from-maven-for-rapid-development/ We're creating a multicore installation and have created a folder structure which no longer has a solrconfig.xml at the top level of the resources. /someapp/src/main/resources |-solr.xml |-core1 |-solrconfig.xml |-schema.xml |-etc. |-core2 |-solrconfig.xml |-schema.xml |-etc. And when you try to run this, Solr can't find what it needs to start up. To fix this, we manually deployed the configuration on the web server and set the solr/home environment variable on the web app's config within Tomcat. Not ideal and it makes automation awkward. Ultimately, I want a completely packaged war for a multicore instance I can drop anywhere without additional setup. Is this possible? Am I approaching this wrong? This e-mail message, and any attachments, is intended only for the use of the individual or entity identified in the alias address of this message and may contain information that is confidential, privileged and subject to legal restrictions and penalties regarding its unauthorized disclosure and use. Any unauthorized review, copying, disclosure, use or distribution is strictly prohibited. If you have received this e-mail message in error, please notify the sender immediately by reply e-mail and delete this message, and any attachments, from your system. Thank you.
Restart SolR and not Tomcat
Hello everybody, I use SolR with Tomcat, and I've this problem: I must to restart SolR without restart Tomcat and I must to do this operation on shell. I try to do this operation with this syntax but it doesn't give result: curl -u user:password http://localhost:8080/manager/text/reload?path=/solr How can I do? -- Jenny Arduini I.T.T. S.r.l. Strada degli Angariari, 25 47891 Falciano Repubblica di San Marino Tel 0549 941183 Fax 0549 974280 email: jardu...@ittweb.net http://www.ittweb.net
Re: SolrCloud Feedback
Hi, I have so far just tested the examples and got a N by M cluster running. My feedback: a) First of all, a major update of the SolrCloud Wiki is needed, to clearly state what is in which version, what are current improvement plans and get rid of outdated stuff. That said I think there are many good ideas there. b) The collection terminology is too much confused with core, and should probably be made more distinct. I just tried to configure two cores on the same Solr instance into the same collection, and that worked fine, both as distinct shards and as same shard (replica). The wiki examples give the impression that collection1 in localhost:8983/solr/collection1/select?distrib=true is some magic collection identifier, but what it really does is doing the query on the *core* named collection1, looking up what collection that core is part of and distributing the query to all shards in that collection. c) ZK is not designed to store large files. While the files in conf are normally well below the 1M limit ZK imposes, we should perhaps consider using a lightweight distributed object or k/v store for holding the /CONFIGS and let ZK store a reference only d) How are admins supposed to update configs in ZK? Install their favourite ZK editor? e) We should perhaps not be so afraid to make ZK a requirement for Solr in v4. Ideally you should interact with a 1-node Solr in the same manner as you do with a 100-node Solr. An example is the Admin GUI where the schema and solrconfig links assume local file. This requires decent tool support to make ZK interaction intuitive, such as import and export commands. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 19. jan. 2011, at 21.07, Mark Miller wrote: Hello Users, About a little over a year ago, a few of us started working on what we called SolrCloud. This initial bit of work was really a combination of laying some base work - figuring out how to integrate ZooKeeper with Solr in a limited way, dealing with some infrastructure - and picking off some low hanging search side fruit. The next step is the indexing side. And we plan on starting to tackle that sometime soon. But first - could you help with some feedback?ISome people are using our SolrCloud start - I have seen evidence of it ;) Some, even in production. I would love to have your help in targeting what we now try and improve. Any suggestions or feedback? If you have sent this before, I/others likely missed it - send it again! I know anyone that has used SolrCloud has some feedback. I know it because I've used it too ;) It's too complicated to setup still. There are still plenty of pain points. We accepted some compromise trying to fit into what Solr was, and not wanting to dig in too far before feeling things out and letting users try things out a bit. Thinking that we might be able to adjust Solr to be more in favor of SolrCloud as we go, what is the ideal state of the work we have currently done? If anyone using SolrCloud helps with the feedback, I'll help with the coding effort. - Mark Miller -- lucidimagination.com
Wikipedia table of contents.
pIs there a detailed, perhaps alphabetical amp; hierarchical table of contents for all ether wikis on the sole site?brbrbr/p pSent from Yahoo! Mail on Android/p
Re: Wikipedia table of contents.
Yes but it's not very useful: http://wiki.apache.org/solr/TitleIndex On Thursday 10 February 2011 16:14:40 Dennis Gearon wrote: pIs there a detailed, perhaps alphabetical amp; hierarchical table of contents for all ether wikis on the sole site?brbrbr/p pSent from Yahoo! Mail on Android/p -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350
Writing on master while replicating to slave
Hi, When a slave is replicating from the master instance, it appears a write lock is created. Will this lock cause issues with writing to the master while the replication is occurring or does SOLR have some queuing that occurs to prevent the actual write until the replication is complete? I've been looking around but can't seem to find anything definitive. My application's data is user centric and as a result the application does a lot of updates and commits. Additionally, we want to provide near real-time searching and so replication would have to occur aggressively. Does anybody have any strategies for handling such an application which they would be willing to share? Thanks, Shane
Re: Restart SolR and not Tomcat
Jenny, look inside the documentation of the manager application, I'm guessing you haven't activated the cross context and privileges in the server.xml to get this running. Or does it work with HTML in a browser? http://localhost:8080/manager/html paul Le 10 févr. 2011 à 16:07, Jenny Arduini a écrit : Hello everybody, I use SolR with Tomcat, and I've this problem: I must to restart SolR without restart Tomcat and I must to do this operation on shell. I try to do this operation with this syntax but it doesn't give result: curl -u user:password http://localhost:8080/manager/text/reload?path=/solr How can I do? -- Jenny Arduini I.T.T. S.r.l. Strada degli Angariari, 25 47891 Falciano Repubblica di San Marino Tel 0549 941183 Fax 0549 974280 email: jardu...@ittweb.net http://www.ittweb.net
Re: Sharing a schema with multiple cores
Ok I found the solution: First of all schema is an attribute of the core tag so it becomes: core name=prod instanceDir=prod schema=conf/schema.xml/ Also make sure the conf directory in your classpath or relative to the path from where you are launching solr. It is NOT relative to solr.xml path. Marc. On Thu, Feb 10, 2011 at 2:48 PM, Marc SCHNEIDER marc.schneide...@gmail.comwrote: Hi, I'm using Solr 1.4.1 and trying to share a schema among two cores. Here is what I did : solr.xml : solr persistent=false cores adminPath=/admin/cores defaultCoreName=prod shareSchema=true core name=prod instanceDir=prod schemaconf/schema.xml/schema /core core name=test instanceDir=test schemaconf/schema.xml/schema /core /cores /solr Then in my solr.home (where solr.xml lives) I created a conf directory and put schema.xml inside. But it doesn't work and Solr is complaining about missing schema.xml (Can't find resource 'schema.xml'). If I put schema.xml into test/conf and prod/conf it does work... So what am I missing here? Thanks, Marc.
Re: Restart SolR and not Tomcat
Her URL has /text/ in it for some reason, replace that with html like Paul has: curl -u user:password http://localhost:8080/manager/html/reload?path=/solr Alternatively if you have JMX access get the mbean with domain: Catalina name: //localhost/solr j2eeType: WebModule J2EEServer: none J2EEApplication: none beanClass: org.apache.tomcat.util.modeler.BaseModelMBean and call reload on it. Chris On Feb 10, 2011, at 7:45 AM, Paul Libbrecht wrote: Jenny, look inside the documentation of the manager application, I'm guessing you haven't activated the cross context and privileges in the server.xml to get this running. Or does it work with HTML in a browser? http://localhost:8080/manager/html paul Le 10 févr. 2011 à 16:07, Jenny Arduini a écrit : Hello everybody, I use SolR with Tomcat, and I've this problem: I must to restart SolR without restart Tomcat and I must to do this operation on shell. I try to do this operation with this syntax but it doesn't give result: curl -u user:password http://localhost:8080/manager/text/reload?path=/solr How can I do? -- Jenny Arduini I.T.T. S.r.l. Strada degli Angariari, 25 47891 Falciano Repubblica di San Marino Tel 0549 941183 Fax 0549 974280 email: jardu...@ittweb.net http://www.ittweb.net
Re: Restart SolR and not Tomcat
If I execute this comand in shell:
curl -u user:password
http://localhost:8080/manager/html/reload?path=/solr
I get this result:
!DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd;
html
head
title401 Unauthorized/title
style type=text/css
!--
BODY
{font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;font-size:12px;}
H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
PRE, TT {border: 1px dotted #525D76}
A {color : black;}A.name {color : black;}
--
/style
/head
body
h1401 Unauthorized/h1
p
You are not authorized to view this page. If you have not changed
any configuration files, please examine the file
ttconf/tomcat-users.xml/tt in your installation. That
file must contain the credentials to let you use this webapp.
/p
p
For example, to add the ttmanager-gui/tt role to a user named
tttomcat/tt with a password of tts3cret/tt, add the following to the
config file listed above.
/p
pre
lt;role rolename=manager-gui/gt;
lt;user username=tomcat password=s3cret roles=manager-gui/gt;
/pre
p
Note that for Tomcat 7 onwards, the roles required to use the manager
application were changed from the single ttmanager/tt role to the
following four roles. You will need to assign the role(s) required for
the functionality you wish to access.
/p
ul
littmanager-gui/tt - allows access to the HTML GUI and the status
pages/li
littmanager-script/tt - allows access to the text interface and the
status pages/li
littmanager-jmx/tt - allows access to the JMX proxy and the status
pages/li
littmanager-status/tt - allows access to the status pages only/li
/ul
p
The HTML interface is protected against CSRF but the text and JMX
interfaces
are not. To maintain the CSRF protection:
/p
ul
liusers with the ttmanager-gui/tt role should not be granted either
the ttmanager-script/tt or ttmanager-jmx/tt roles./li
liif the text or jmx interfaces are accessed through a browser (e.g. for
testing since these interfaces are intended for tools not
humans) then
the browser must be closed afterwards to terminate the
session./li
/ul
p
For more information - please see the
a href=/docs/manager-howto.htmlManager App HOW-TO/a.
/p
/body
/html
What do I do of wrong?
But if I show the manager with browser is all ok, and I reload the
application without problem.
Jenny Arduini
I.T.T. S.r.l.
Strada degli Angariari, 25
47891 Falciano
Repubblica di San Marino
Tel 0549 941183
Fax 0549 974280
email: jardu...@ittweb.net
http://www.ittweb.net
Il 10/02/2011 17.11, Wilkes, Chris ha scritto:
Her URL has /text/ in it for some reason, replace that with html
like Paul has:
curl -u user:password
http://localhost:8080/manager/html/reload?path=/solr
Alternatively if you have JMX access get the mbean with
domain: Catalina
name: //localhost/solr
j2eeType: WebModule
J2EEServer: none
J2EEApplication: none
beanClass: org.apache.tomcat.util.modeler.BaseModelMBean
and call reload on it.
Chris
On Feb 10, 2011, at 7:45 AM, Paul Libbrecht wrote:
Jenny,
look inside the documentation of the manager application, I'm
guessing you haven't activated the cross context and privileges in
the server.xml to get this running.
Or does it work with HTML in a browser?
http://localhost:8080/manager/html
paul
Le 10 févr. 2011 à 16:07, Jenny Arduini a écrit :
Hello everybody,
I use SolR with Tomcat, and I've this problem:
I must to restart SolR without restart Tomcat and I must to do this
operation on shell.
I try to do this operation with this syntax but it doesn't give result:
curl -u user:password
http://localhost:8080/manager/text/reload?path=/solr
How can I do?
--
Jenny Arduini
I.T.T. S.r.l.
Strada degli Angariari, 25
47891 Falciano
Repubblica di San Marino
Tel 0549 941183
Fax 0549 974280
email: jardu...@ittweb.net
http://www.ittweb.net
Re: SolrCloud Feedback
Hi Mark, hi all, I just got a customer request to conduct an analysis on the state of SolrCloud. He wants to see SolrCloud part of the next solr 1.5 release and is willing to sponsor our dev time to close outstanding bugs and open issues that may prevent the inclusion of SolrCloud in the next release. I need to give him a listing of issues and an estimation how long it will take us to fix them. I did https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=truejqlQuery=project+%3D+SOLR+AND+(summary+~+cloud+OR+description+~+cloud+OR+comment+~+cloud)+AND+resolution+%3D+Unresolved which returns me 8 bug. Do you consider this a comprehensive list of open issues or are there missing some important ones in this list? I read http://wiki.apache.org/solr/SolrCloud and it is talking about a branch of its own however when I review https://issues.apache.org/jira/browse/SOLR-1873 I get the impression that the work is already merged back into trunk, right? So what is the best to start testing the branch or trunk? TIA for any informations salu2 -- Thorsten Scherler thorsten.at.apache.org codeBusters S.L. - web based systems consulting, training and solutions http://www.codebusters.es/ -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-Feedback-tp2290048p2467091.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Restart SolR and not Tomcat
Exactly Jenny,
*you are not authorized*
means the request cannot be authorized to execute.
Means some calls failed with a security error.
manager/html/reload - for browsers by humans
manager/reload- for curl
(at least that's my experience)
paul
Le 10 févr. 2011 à 17:32, Jenny Arduini a écrit :
If I execute this comand in shell:
curl -u user:password http://localhost:8080/manager/html/reload?path=/solr
I get this result:
!DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd;
html
head
title401 Unauthorized/title
style type=text/css
!--
BODY
{font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;font-size:12px;}
H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
PRE, TT {border: 1px dotted #525D76}
A {color : black;}A.name {color : black;}
--
/style
/head
body
h1401 Unauthorized/h1
p
You are not authorized to view this page. If you have not changed
any configuration files, please examine the file
ttconf/tomcat-users.xml/tt in your installation. That
file must contain the credentials to let you use this webapp.
/p
p
For example, to add the ttmanager-gui/tt role to a user named
tttomcat/tt with a password of tts3cret/tt, add the following to the
config file listed above.
/p
pre
lt;role rolename=manager-gui/gt;
lt;user username=tomcat password=s3cret roles=manager-gui/gt;
/pre
p
Note that for Tomcat 7 onwards, the roles required to use the manager
application were changed from the single ttmanager/tt role to the
following four roles. You will need to assign the role(s) required for
the functionality you wish to access.
/p
ul
littmanager-gui/tt - allows access to the HTML GUI and the status
pages/li
littmanager-script/tt - allows access to the text interface and the
status pages/li
littmanager-jmx/tt - allows access to the JMX proxy and the status
pages/li
littmanager-status/tt - allows access to the status pages only/li
/ul
p
The HTML interface is protected against CSRF but the text and JMX
interfaces
are not. To maintain the CSRF protection:
/p
ul
liusers with the ttmanager-gui/tt role should not be granted either
the ttmanager-script/tt or ttmanager-jmx/tt roles./li
liif the text or jmx interfaces are accessed through a browser (e.g. for
testing since these interfaces are intended for tools not humans) then
the browser must be closed afterwards to terminate the session./li
/ul
p
For more information - please see the
a href=/docs/manager-howto.htmlManager App HOW-TO/a.
/p
/body
/html
What do I do of wrong?
But if I show the manager with browser is all ok, and I reload the
application without problem.
Jenny Arduini
I.T.T. S.r.l.
Strada degli Angariari, 25
47891 Falciano
Repubblica di San Marino
Tel 0549 941183
Fax 0549 974280
email: jardu...@ittweb.net
http://www.ittweb.net
Il 10/02/2011 17.11, Wilkes, Chris ha scritto:
Her URL has /text/ in it for some reason, replace that with html like
Paul has:
curl -u user:password
http://localhost:8080/manager/html/reload?path=/solr
Alternatively if you have JMX access get the mbean with
domain: Catalina
name: //localhost/solr
j2eeType: WebModule
J2EEServer: none
J2EEApplication: none
beanClass: org.apache.tomcat.util.modeler.BaseModelMBean
and call reload on it.
Chris
On Feb 10, 2011, at 7:45 AM, Paul Libbrecht wrote:
Jenny,
look inside the documentation of the manager application, I'm guessing you
haven't activated the cross context and privileges in the server.xml to get
this running.
Or does it work with HTML in a browser?
http://localhost:8080/manager/html
paul
Le 10 févr. 2011 à 16:07, Jenny Arduini a écrit :
Hello everybody,
I use SolR with Tomcat, and I've this problem:
I must to restart SolR without restart Tomcat and I must to do this
operation on shell.
I try to do this operation with this syntax but it doesn't give result:
curl -u user:password
http://localhost:8080/manager/text/reload?path=/solr
How can I do?
--
Jenny Arduini
I.T.T. S.r.l.
Strada degli Angariari, 25
47891 Falciano
Repubblica di San Marino
Tel 0549 941183
Fax 0549 974280
email: jardu...@ittweb.net
http://www.ittweb.net
RE: Restart SolR and not Tomcat
I don't know: either way works for me via cURL.
I can only say double check your typing (make sure you're passing the
user/password you think you are), and double check server.xml.
Oh, the tomcat roles were tightened up a bit in tomcat 7. If you're using
tomcat 7 (especially if you've upgraded to it recently), you might have to add
a little more detail to server.xml to make that user/password combination work
from different sources. (i.e. add the manager-gui role or add the manager-text
role to your tomcat user as appropriate). Read and understand the tomcat docs
before doing that.
Brad
-Original Message-
From: Paul Libbrecht [mailto:p...@hoplahup.net]
Sent: February-10-11 2:08 PM
To: solr-user@lucene.apache.org
Subject: Re: Restart SolR and not Tomcat
Exactly Jenny,
*you are not authorized*
means the request cannot be authorized to execute.
Means some calls failed with a security error.
manager/html/reload - for browsers by humans
manager/reload- for curl
(at least that's my experience)
paul
Le 10 févr. 2011 à 17:32, Jenny Arduini a écrit :
If I execute this comand in shell:
curl -u user:password http://localhost:8080/manager/html/reload?path=/solr
I get this result:
!DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01//EN
http://www.w3.org/TR/html4/strict.dtd;
html
head
title401 Unauthorized/title
style type=text/css
!--
BODY
{font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;font-size:12px;}
H1
{font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;}
PRE, TT {border: 1px dotted #525D76}
A {color : black;}A.name {color : black;}
--
/style
/head
body
h1401 Unauthorized/h1
p
You are not authorized to view this page. If you have not changed
any configuration files, please examine the file
ttconf/tomcat-users.xml/tt in your installation. That
file must contain the credentials to let you use this webapp.
/p
p
For example, to add the ttmanager-gui/tt role to a user named
tttomcat/tt with a password of tts3cret/tt, add the following to the
config file listed above.
/p
pre
lt;role rolename=manager-gui/gt;
lt;user username=tomcat password=s3cret roles=manager-gui/gt;
/pre
p
Note that for Tomcat 7 onwards, the roles required to use the manager
application were changed from the single ttmanager/tt role to the
following four roles. You will need to assign the role(s) required for
the functionality you wish to access.
/p
ul
littmanager-gui/tt - allows access to the HTML GUI and the status
pages/li
littmanager-script/tt - allows access to the text interface and the
status pages/li
littmanager-jmx/tt - allows access to the JMX proxy and the status
pages/li
littmanager-status/tt - allows access to the status pages only/li
/ul
p
The HTML interface is protected against CSRF but the text and JMX
interfaces
are not. To maintain the CSRF protection:
/p
ul
liusers with the ttmanager-gui/tt role should not be granted either
the ttmanager-script/tt or ttmanager-jmx/tt roles./li
liif the text or jmx interfaces are accessed through a browser (e.g. for
testing since these interfaces are intended for tools not humans) then
the browser must be closed afterwards to terminate the session./li
/ul
p
For more information - please see the
a href=/docs/manager-howto.htmlManager App HOW-TO/a.
/p
/body
/html
What do I do of wrong?
But if I show the manager with browser is all ok, and I reload the
application without problem.
Jenny Arduini
I.T.T. S.r.l.
Strada degli Angariari, 25
47891 Falciano
Repubblica di San Marino
Tel 0549 941183
Fax 0549 974280
email: jardu...@ittweb.net
http://www.ittweb.net
Il 10/02/2011 17.11, Wilkes, Chris ha scritto:
Her URL has /text/ in it for some reason, replace that with html like
Paul has:
curl -u user:password
http://localhost:8080/manager/html/reload?path=/solr
Alternatively if you have JMX access get the mbean with
domain: Catalina
name: //localhost/solr
j2eeType: WebModule
J2EEServer: none
J2EEApplication: none
beanClass: org.apache.tomcat.util.modeler.BaseModelMBean
and call reload on it.
Chris
On Feb 10, 2011, at 7:45 AM, Paul Libbrecht wrote:
Jenny,
look inside the documentation of the manager application, I'm guessing you
haven't activated the cross context and privileges in the server.xml to get
this running.
Or does it work with HTML in a browser?
http://localhost:8080/manager/html
paul
Le 10 févr. 2011 à 16:07, Jenny Arduini a écrit :
Hello everybody,
I use SolR with Tomcat, and I've this problem:
I must to restart SolR without restart Tomcat and I must to do this
operation on shell.
I try to do this operation with this syntax but it doesn't give result:
curl -u user:password
http://localhost:8080/manager/text/reload?path=/solr
How can I do?
--
Jenny
rejected email
Anyone else having problems with the Solr users list suddenly deciding everything you send is spam? For the last couple of days I've had this happening from gmail, and as far as I know I haven't changed anything that would give my mails a different spam score which is being exceeded according to the bounced message... Thanks, Erick
Re: rejected email
I tried posting from gmail this morning and had it rejected. When I resent as plaintext, it went through. On Thu, Feb 10, 2011 at 11:51 AM, Erick Erickson erickerick...@gmail.com wrote: Anyone else having problems with the Solr users list suddenly deciding everything you send is spam? For the last couple of days I've had this happening from gmail, and as far as I know I haven't changed anything that would give my mails a different spam score which is being exceeded according to the bounced message... Thanks, Erick
Re: SOLR-792 (hierarchical faceting) issue when only 1 document should be present in the pivot
I have had the same problem .. my facet pivots was returning results
something like
Cat-A (3)
Item X
Item Y
only 2 items instead of 3
or even
Cat-B (2)
no items
zero items instead of 2
so the parent level count didnt matched with the returned child pivots ..
but once I set the facet.pivot.mincount = 0 .. then it works fine .. is this
a bug or the desired behavior ..
On Wed, Nov 24, 2010 at 7:49 AM, Nicolas Peeters nicoli...@gmail.comwrote:
Hi Solr-Users,
I realized that I can get the behaviour that I expect if I put
facet.pivot.mincount to 0. However, I'm still puzzled why this needs to be
0
and not 1. There's one occurence for this document, isn't it?
With this value to 1, the print out of the pivot looks like this (where you
clearly see (1) for Value_that_can't_be_matched):
PIVOT: level1_loc_s,level2_loc_s,level3_loc_s
level1_loc_s=Greater London (8)
level2_loc_s=London (5)
level3_loc_s=Mayfair (3)
level3_loc_s=Hammersmith (2)
level2_loc_s=Greenwich (3)
level3_loc_s=Greenwich Centre (2)
level3_loc_s=Value_that_cant_be_matched (1)
level1_loc_s=Groot Amsterdam (5)
level2_loc_s=Amsterdam (3)
level3_loc_s=Jordaan (2)
level3_loc_s=Centrum (1)
level2_loc_s=Amstelveen (2)
level3_loc_s=Centrum (2)
Any expert advice on why this is the case is more than welcome!
Best regards,
Nicolas
On Wed, Nov 24, 2010 at 2:27 PM, Nicolas Peeters nicoli...@gmail.com
wrote:
Hi Solr Community,
I've been experimenting with Solr 4.0 (trunk) in order to test the
SOLR-792
feature. I have written a test that shows what I'm trying to ask.
Basically,
I'm creating a hierarchy of the area/city/neighbourhood. The problem that
I
see is that for documents that have only 1 item in a particular hierarchy
(e.g. Greater London/Greenwich/Centre (which I've called
Value_that_cant_be_matched in this example...)), these are not found by
the pivot facet. If I add a second one, then it works. I'm puzzled why
this
is the case.
This is the result of the Sytem.out that prints out the pivot facet
fields
hierarchy (see line 86)
PIVOT: level1_loc_s,level2_loc_s,level3_loc_s
level1_loc_s=Greater London (8)
level2_loc_s=London (5)
level3_loc_s=Mayfair (3)
level3_loc_s=Hammersmith (2)
level2_loc_s=Greenwich (3)
level3_loc_s=Greenwich Centre (2)
//-- why isn't there a
level3_loc_s=Value_that_cant_be_matched (1) here?
level1_loc_s=Groot Amsterdam (5)
level2_loc_s=Amsterdam (3)
level3_loc_s=Jordaan (2)
level2_loc_s=Amstelveen (2)
level3_loc_s=Centrum (2)
How can I make sure that Solr would find in the tree the single document
when I facet on this location hierarchy?
Thank you very much for your help.
Nicolas
import java.io.IOException;
import java.net.MalformedURLException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer;
import org.apache.solr.client.solrj.response.PivotField;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.common.util.NamedList;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
/**
* This is a test for hiearchical faceting based on SOLR-792 (I basically
just checkout the trunk of Solr-4.0).
*
* Unit test that shows the particular behaviour that I'm experiencing.
* I would have expected that the doc (see line 95) with as level3_loc_s
Value_that_cant_be_matched would appear in the pivot. It seems that you
actually need at least 2!
*
* @author npeeters
*/
public class HierarchicalPivotTest {
CommonsHttpSolrServer server;
@Before
public void setup() throws MalformedURLException {
// the instance can be reused
this.server = new CommonsHttpSolrServer(
http://localhost:8983/solr;);
this.server.setSoTimeout(500); // socket read timeout
this.server.setConnectionTimeout(100);
this.server.setDefaultMaxConnectionsPerHost(100);
this.server.setMaxTotalConnections(100);
this.server.setFollowRedirects(false); // defaults to false
// allowCompression defaults to false.
}
protected ListSolrInputDocument createHierarchicalOrgData() {
int id = 1;
ListSolrInputDocument docs = new
ArrayListSolrInputDocument();
docs.add(makeTestDoc(id, id++, name, Organization + id,
level1_loc_s, Groot Amsterdam, level2_loc_s, Amsterdam,
level3_loc_s, Centrum));
docs.add(makeTestDoc(id, id++, name, Organization + id,
level1_loc_s, Groot Amsterdam, level2_loc_s, Amsterdam,
level3_loc_s, Jordaan));
docs.add(makeTestDoc(id, id++, name,
edismax with windows path input?
I am using the edismax query parser -- its awesome! works well for standard dismax type queries, and allows explicit fields when necessary. I have hit a snag when people enter something that looks like a windows path: lst name=params str name=qF:\path\to\a\file/str /lst this gets parsed as: str name=rawquerystringF:\path\to\a\file/str str name=querystringF:\path\to\a\file/str str name=parsedquery+()/str Putting it in quotes makes the not-quite right query: str name=rawquerystringF:\path\to\a\file/str str name=querystringF:\path\to\a\file/str str name=parsedquery +DisjunctionMaxQuery((path:f:pathtoafile^4.0 | name:f (pathtoafile fpathtoafile)^7.0)~0.01) /str str name=parsedquery_toString +(path_path:f:pathtoafile^4.0 | name:f (pathtoafile fpathtoafile)^7.0)~0.01 /str Telling people to escape the query: q=F\:\\path\\to\\a\\file is unrealistic, but gives the proper parsed query: +DisjunctionMaxQuery((path_path:f:/path/to/a/file^4.0 | name:f path to a (file fpathtoafile)^7.0)~0.01) Any ideas on how to support this? I could try looking for things like paths in the app, and then modify the query, or maybe look at extending edismax. Perhaps when F: does not match a given field, it could auto escape the rest of the word? thanks ryan
Re: edismax with windows path input?
: extending edismax. Perhaps when F: does not match a given field, it : could auto escape the rest of the word? that's actually what yonik initially said it was suppose to do, but when i tried to add a param to let you control which fields would be supported using the : syntax i discovered it didn't work but oculdn't figure out why ... details are in the SOLR-1553 comments -Hoss
Re: Writing on master while replicating to slave
Just for the first part: There's no problem here, the write lock is to keep simultaneous *writes* from occurring, the slave reading the index doesn't enter in to it. Note that in Solr, when segments are created in an index, they are write-once. So basically what happens when a slave replicates is that the current segments that have had commits against them are replicated (they won't be written to after a commit) and Solr merrily continues writing to new segments. So the slave gets its snapshot of the index as of the last commit that happened before it polled the master. The trunk has something called NRT (near real time) that attempts to decrease the latency, but I don't know how that plays with replication. Best Erick On Thu, Feb 10, 2011 at 1:37 PM, Erick Erickson erickerick...@gmail.com wrote: Just for the first part: There's no problem here, the write lock is to keep simultaneous *writes* from occurring, the slave reading the index doesn't enter in to it. Note that in Solr, when segments are created in an index, they are write-once. So basically what happens when a slave replicates is that the current segments that have had commits against them are replicated (they won't be written to after a commit) and Solr merrily continues writing to new segments. So the slave gets its snapshot of the index as of the last commit that happened before it polled the master. The trunk has something called NRT (near real time) that attempts to decrease the latency, but I don't know how that plays with replication. Best Erick On Thu, Feb 10, 2011 at 10:33 AM, Shane Perry thry...@gmail.com wrote: Hi, When a slave is replicating from the master instance, it appears a write lock is created. Will this lock cause issues with writing to the master while the replication is occurring or does SOLR have some queuing that occurs to prevent the actual write until the replication is complete? I've been looking around but can't seem to find anything definitive. My application's data is user centric and as a result the application does a lot of updates and commits. Additionally, we want to provide near real-time searching and so replication would have to occur aggressively. Does anybody have any strategies for handling such an application which they would be willing to share? Thanks, Shane
Re: rejected email
Hmmm, never noticed that link before, thanks! Which shows you how much I can ignore that's perfectly obvious G... Works like a champ. Erick On Thu, Feb 10, 2011 at 2:05 PM, Shane Perry thry...@gmail.com wrote: I tried posting from gmail this morning and had it rejected. When I resent as plaintext, it went through. On Thu, Feb 10, 2011 at 11:51 AM, Erick Erickson erickerick...@gmail.com wrote: Anyone else having problems with the Solr users list suddenly deciding everything you send is spam? For the last couple of days I've had this happening from gmail, and as far as I know I haven't changed anything that would give my mails a different spam score which is being exceeded according to the bounced message... Thanks, Erick
Re: edismax with windows path input?
ah -- that makes sense. Yonik... looks like you were assigned to it last week -- should I take a look, or do you already have something in the works? On Thu, Feb 10, 2011 at 2:52 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : extending edismax. Perhaps when F: does not match a given field, it : could auto escape the rest of the word? that's actually what yonik initially said it was suppose to do, but when i tried to add a param to let you control which fields would be supported using the : syntax i discovered it didn't work but oculdn't figure out why ... details are in the SOLR-1553 comments -Hoss
SolrCloud (ZooKeeper)
Hi, I've completed the quickdirty tutorials of SolrCloud ( see http://wiki.apache.org/solr/SolrCloud ). The whole concept of SolrCloud and ZooKeeper look indeed very promising. I found also some info about a 'ZooKeeperComponent' - From this conponent it should be possible to configure ZooKeeper directly from the sorlconfig.xml ( see http://wiki.apache.org/solr/ZooKeeperIntegration ). Is this functionality already implemented (or is something for the future)? If so, can you please point me to a good guide/tutorial, cause I do not find much on the regular web. Or else copy me your real working ZooKeeperComponent layout how you configured ZooKeeper. Thanks for helping me,
Re: edismax with windows path input?
On Thu, Feb 10, 2011 at 2:52 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : extending edismax. Perhaps when F: does not match a given field, it : could auto escape the rest of the word? that's actually what yonik initially said it was suppose to do Hmmm, not really. essentially that FOO:BAR and FOO\:BAR would be equivalent if FOO is not the name of a real field according to the IndexSchema That part is true, but doesn't say anything about escaping. And for some unknown reason, this no longer works. foo_s:foo\-bar is a valid lucene query (with only a dash between the foo and the bar), and presumably it should be treated the same in edismax. Treating it as foo_s:foo\\-bar (a backslash and a dash between foo and bar) might cause more problems than it's worth? -Yonik http://lucidimagination.com
Re: edismax with windows path input?
On Thu, Feb 10, 2011 at 3:05 PM, Ryan McKinley ryan...@gmail.com wrote: ah -- that makes sense. Yonik... looks like you were assigned to it last week -- should I take a look, or do you already have something in the works? I got busy on other things, and don't have anything in the works. I think edismax should probably just be marked as experimental for 3.1. -Yonik http://lucidimagination.com
Re: Unable to build the SolrCloud branch - SOLR-1873
Hi, I've followed the guide worked perfect for me. ( I had to execute ant compile - not ant example, but not likely that was your problem ). 2011/1/2 siddharth sid_invinci...@yahoo.co.in I seemed to have figured out the problem. I think it was an issue with the JAVA_HOME being set. The build was failing while compiling the module solrj -- View this message in context: http://lucene.472066.n3.nabble.com/Unable-to-build-the-SolrCloud-branch-SOLR-1873-tp2180635p2180800.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: edismax with windows path input?
: essentially that FOO:BAR and FOO\:BAR would be equivalent if FOO is : not the name of a real field according to the IndexSchema : : That part is true, but doesn't say anything about escaping. And for : some unknown reason, this no longer works. that's the only part i was refering to. -Hoss
Re: SolrCloud (ZooKeeper)
On Thu, Feb 10, 2011 at 5:00 PM, Stijn Vanhoorelbeke stijn.vanhoorelb...@gmail.com wrote: I've completed the quickdirty tutorials of SolrCloud ( see http://wiki.apache.org/solr/SolrCloud ). The whole concept of SolrCloud and ZooKeeper look indeed very promising. I found also some info about a 'ZooKeeperComponent' - From this conponent it should be possible to configure ZooKeeper directly from the sorlconfig.xml ( see http://wiki.apache.org/solr/ZooKeeperIntegration ). That's not part of what has been committed to trunk. Also, if one want's to get solrconfig from zookeeper, having the zookeeper config in solrconfig is a bit chicken and eggish ;-) -Yonik http://lucidimagination.com
Re: edismax with windows path input?
foo_s:foo\-bar is a valid lucene query (with only a dash between the foo and the bar), and presumably it should be treated the same in edismax. Treating it as foo_s:foo\\-bar (a backslash and a dash between foo and bar) might cause more problems than it's worth? I don't think we should escape anything that has a valid field name. If foo_s is a field, then foo_s:foo\-bar should be used as is. If foo_s is not a field, I would want the whole thing escaped to: foo_s\:foo\\-bar before getting passed to the rest of the dismax mojo. Does that make sense? marking edismax as experimental for 3.1 makes sense! ryan
Re: edismax with windows path input?
On Thu, Feb 10, 2011 at 5:51 PM, Ryan McKinley ryan...@gmail.com wrote: foo_s:foo\-bar is a valid lucene query (with only a dash between the foo and the bar), and presumably it should be treated the same in edismax. Treating it as foo_s:foo\\-bar (a backslash and a dash between foo and bar) might cause more problems than it's worth? I don't think we should escape anything that has a valid field name. If foo_s is a field, then foo_s:foo\-bar should be used as is. If foo_s is not a field, I would want the whole thing escaped to: foo_s\:foo\\-bar before getting passed to the rest of the dismax mojo. Does that make sense? Maybe in some scenarios, but probably not in others? The clause bar\-baz will be equiv to field:bar-baz Hence it seems like the clause foo:bar\-baz should be equiv to field:foo:bar-baz (assuming foo is not a field) -Yonik http://lucidimagination.com
running optimize on master
Hi, I've read running optimize is similar to running defrag on a hard disk. Deleted docs are removed and segments are reorganized for faster searching. I have a couple questions. Is optimize necessary if I never delete documents? I build the index every hour but we don't delete in between builds. Secondly, what kind of reorganizing of segments is done to make searches faster? Thanks, Tri
Re: QTime Solr Query
Let's see what the queries are. If you're searching for single terms that don't match many docs that's one thing. If you're looking at many terms that match many documents, I'd expect larger numbers. Unless you're hitting the document cache and not searching at all Best Erick On Thu, Feb 10, 2011 at 5:08 PM, Stijn Vanhoorelbeke stijn.vanhoorelb...@gmail.com wrote: Hi, I've done some stress testing onto my solr system ( running in the ec2 cloud ). From what I've noticed during the tests, the QTime drops to just 1 or 2 ms ( on a index of ~2 million documents ). My first thought pointed me to the different Solr caches; so I've disabled all of them. Yet QTime stays low. Then the Lucence internal Field Cache came into sight. This cache is hidden deep into Lucence and is not configurable trough Solr. To cope with this I thought I would lower the memory allocated to Solr - that way a smaller cache is forced. But yet QTime stays low. Can Solr be so fast to retrieve queries in just 1/2 ms - even if I only allocate 100 Mb to Solr?
Re: running optimize on master
Does optimize merge all segments into 1 segment on the master after the build? Or after the build, there's only 1 segment. thanks, Tri From: Erick Erickson erickerick...@gmail.com To: solr-user@lucene.apache.org Sent: Thu, February 10, 2011 5:08:44 PM Subject: Re: running optimize on master Optimizing isn't necessary in your scenario, as you don't delete documents and rebuild the whole thing each time anyway. As for faster searches, this has been largely been made obsolete by recent changes in how indexes are built in the first place. Especially as you can build your index in an hour, it's likely not big enough to benefit from optimizing even under the old scenario So, unless you have some evidence that your queries are performing poorly, I would just leave the optimize step off. Best Erick On Thu, Feb 10, 2011 at 7:09 PM, Tri Nguyen tringuye...@yahoo.com wrote: Hi, I've read running optimize is similar to running defrag on a hard disk. Deleted docs are removed and segments are reorganized for faster searching. I have a couple questions. Is optimize necessary if I never delete documents? I build the index every hour but we don't delete in between builds. Secondly, what kind of reorganizing of segments is done to make searches faster? Thanks, Tri
Re: running optimize on master
Optimize will do just what you suggest, although there's a parameter whose name escapes me controlling how many segments the index is reduced to so this is configurable. It's also possible, but kind of unlikely, that the original indexing process would produce only one segment. You could tell which was the case tell this easily enough by just looking at your index directory during the build. My bet would be the optimize step.. Best Erick On Thu, Feb 10, 2011 at 8:10 PM, Tri Nguyen tringuye...@yahoo.com wrote: Does optimize merge all segments into 1 segment on the master after the build? Or after the build, there's only 1 segment. thanks, Tri From: Erick Erickson erickerick...@gmail.com To: solr-user@lucene.apache.org Sent: Thu, February 10, 2011 5:08:44 PM Subject: Re: running optimize on master Optimizing isn't necessary in your scenario, as you don't delete documents and rebuild the whole thing each time anyway. As for faster searches, this has been largely been made obsolete by recent changes in how indexes are built in the first place. Especially as you can build your index in an hour, it's likely not big enough to benefit from optimizing even under the old scenario So, unless you have some evidence that your queries are performing poorly, I would just leave the optimize step off. Best Erick On Thu, Feb 10, 2011 at 7:09 PM, Tri Nguyen tringuye...@yahoo.com wrote: Hi, I've read running optimize is similar to running defrag on a hard disk. Deleted docs are removed and segments are reorganized for faster searching. I have a couple questions. Is optimize necessary if I never delete documents? I build the index every hour but we don't delete in between builds. Secondly, what kind of reorganizing of segments is done to make searches faster? Thanks, Tri
Re: QTime Solr Query
On Thu, Feb 10, 2011 at 4:08 PM, Stijn Vanhoorelbeke stijn.vanhoorelb...@gmail.com wrote: Hi, I've done some stress testing onto my solr system ( running in the ec2 cloud ). From what I've noticed during the tests, the QTime drops to just 1 or 2 ms ( on a index of ~2 million documents ). My first thought pointed me to the different Solr caches; so I've disabled all of them. Yet QTime stays low. Then the Lucence internal Field Cache came into sight. This cache is hidden deep into Lucence and is not configurable trough Solr. To cope with this I thought I would lower the memory allocated to Solr - that way a smaller cache is forced. But yet QTime stays low. When stress-testing Solr, I usually flush the OS cache also. This is the command to do it on linux: # sync; echo 3 /proc/sys/vm/drop_caches didier Can Solr be so fast to retrieve queries in just 1/2 ms - even if I only allocate 100 Mb to Solr?
Hits when using group=true
It would be good if someone added the hits= on group=true in the log. We are using this parameter and have build a really cool SOLR log analyzer (that I am pushing to release to open source). But it is not as effective if we cannot get group=true to output hits= in the log - since 90% of our queries are group=true... There is a ticket in SOLR for this under SOLR-2337. Can someone help me identify what would be required to get this to work? Bill
Re: solr render biased search result
I am not sure I understand your question. But you can boost the result based on one value over another value. Look at bf http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_change_the_score_of_a_document_based_on_the_.2Avalue.2A_of_a_field_.28say.2C_.22popularity.22.29 On Wed, Feb 9, 2011 at 12:44 PM, cyang2010 ysxsu...@hotmail.com wrote: Hi, I am asked that whether solr renders biased search result? For example, for this search (query all movie title by this Comedy genre), for user who indicates a preference to 1950's movies, solr renders the 1950's movies with higher score (top in the list)? Or if user is a kid, then the result will render G/PG rated movie top in the list, and render all the R rated movie bottom in the list? I know that solr can boost score based on match on a particular field. But it can't favor some value over other value in the same field. is that right? -- View this message in context: http://lucene.472066.n3.nabble.com/solr-render-biased-search-result-tp2461155p2461155.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Turn off caching
do you mean queryResultCache? you can comment related paragraph in solrconfig.xml see http://wiki.apache.org/solr/SolrCaching 2011/2/8 Isan Fulia isan.fu...@germinait.com: Hi, My solrConfig file looks like config updateHandler class=solr.DirectUpdateHandler2 / requestDispatcher handleSelect=true requestParsers enableRemoteStreaming=false multipartUploadLimitInKB=2048 / /requestDispatcher requestHandler name=standard class=solr.StandardRequestHandler default=true / requestHandler name=/update class=solr.XmlUpdateRequestHandler / requestHandler name=/admin/ class=org.apache.solr.handler.admin.AdminHandlers / queryResponseWriter name=xslt class=org.apache.solr.request.XSLTResponseWriter /queryResponseWriter !--config for the admin interface -- admin defaultQuery*:*/defaultQuery /admin /config EveryTime I fire the same query so as to compare the results for different configurations , the query result time is getting reduced because of caching. So I want to turn off the cahing or clear the ache before i fire the same query . Does anyone know how to do it. -- Thanks Regards, Isan Fulia.
Re: Turn off caching
Hi, You can comment out all sections in solrconfig.xml pointing to a cache. However, there is a cache deep in Lucence - the fieldcache - that can't be commented out. This cache will always jump into the picture If I need to do such things, I restart the whole tomcat6 server to flush ALL caches. 2011/2/11 Li Li fancye...@gmail.com do you mean queryResultCache? you can comment related paragraph in solrconfig.xml see http://wiki.apache.org/solr/SolrCaching 2011/2/8 Isan Fulia isan.fu...@germinait.com: Hi, My solrConfig file looks like config updateHandler class=solr.DirectUpdateHandler2 / requestDispatcher handleSelect=true requestParsers enableRemoteStreaming=false multipartUploadLimitInKB=2048 / /requestDispatcher requestHandler name=standard class=solr.StandardRequestHandler default=true / requestHandler name=/update class=solr.XmlUpdateRequestHandler / requestHandler name=/admin/ class=org.apache.solr.handler.admin.AdminHandlers / queryResponseWriter name=xslt class=org.apache.solr.request.XSLTResponseWriter /queryResponseWriter !--config for the admin interface -- admin defaultQuery*:*/defaultQuery /admin /config EveryTime I fire the same query so as to compare the results for different configurations , the query result time is getting reduced because of caching. So I want to turn off the cahing or clear the ache before i fire the same query . Does anyone know how to do it. -- Thanks Regards, Isan Fulia.
