RE: Stop/Restart Solr
Can you please share output of following command? ps -ef | grep 'start.jar' - Jeeva -- Original Message -- From: Raheel Hasan [mailto:raheelhasan@gmail.com] Sent: October 23, 2013 3:19:46 PM GMT+05:30 To: solr-user@lucene.apache.org Subject: Re: Stop/Restart Solr Kill -9 didnt kill it... ... the process is now again listed, but with PPID=1 which I dont want to kill as many processes have this same id... On Tue, Oct 22, 2013 at 11:59 PM, Utkarsh Sengar utkarsh2...@gmail.comwrote: We use this to start/stop solr: Start: java -Dsolr.clustering.enabled=true -Dsolr.solr.home=multicore -Djetty.class.path=lib/ext/* -Dbootstrap_conf=true -DnumShards=3 -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar Stop: java -Dsolr.solr.home=multicore -Dbootstrap_conf=true -DnumShards=3 -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar --stop Thanks, -Utkarsh On Tue, Oct 22, 2013 at 10:09 AM, Raheel Hasan raheelhasan@gmail.com wrote: ok fantastic... thanks a lot guyz On Tue, Oct 22, 2013 at 10:00 PM, François Schiettecatte fschietteca...@gmail.com wrote: Yago has the right command to search for the process, that will get you the process ID specifically the first number on the output line, then do 'kill ###', if that fails 'kill -9 ###'. François On Oct 22, 2013, at 12:56 PM, Raheel Hasan raheelhasan@gmail.com wrote: its CentOS... and using jetty with solr here.. On Tue, Oct 22, 2013 at 9:54 PM, François Schiettecatte fschietteca...@gmail.com wrote: A few more specifics about the environment would help, Windows/Linux/...? Jetty/Tomcat/...? François On Oct 22, 2013, at 12:50 PM, Yago Riveiro yago.rive...@gmail.com wrote: If you are asking about if solr has a way to restart himself, I think that the answer is no. If you lost control of the remote machine someone will need to go and restart the machine ... You can try use a kvm or other remote control system -- Yago Riveiro Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, October 22, 2013 at 5:46 PM, François Schiettecatte wrote: If you are on linux/unix, use the kill command. François On Oct 22, 2013, at 12:42 PM, Raheel Hasan raheelhasan@gmail.com (mailto: raheelhasan@gmail.com) wrote: Hi, is there a way to stop/restart java? I lost control over it via SSH and connection was closed. But the Solr (start.jar) is still running. thanks. -- Regards, Raheel Hasan -- Regards, Raheel Hasan -- Regards, Raheel Hasan -- Thanks, -Utkarsh -- Regards, Raheel Hasan
RE: Stop/Restart Solr
It seems process started recently. Is there any external cron/process triggering a startup of Solr? Kill again and monitor it. - Jeeva -- Original Message -- From: Raheel Hasan [mailto:raheelhasan@gmail.com] Sent: October 23, 2013 3:29:47 PM GMT+05:30 To: solr-user@lucene.apache.org Subject: Re: Stop/Restart Solr 31173 1 0 16:45 ?00:00:08 java -jar start.jar On Wed, Oct 23, 2013 at 2:53 PM, Jeevanandam M. je...@myjeeva.com wrote: Can you please share output of following command? ps -ef | grep 'start.jar' - Jeeva -- Original Message -- From: Raheel Hasan [mailto:raheelhasan@gmail.com] Sent: October 23, 2013 3:19:46 PM GMT+05:30 To: solr-user@lucene.apache.org Subject: Re: Stop/Restart Solr Kill -9 didnt kill it... ... the process is now again listed, but with PPID=1 which I dont want to kill as many processes have this same id... On Tue, Oct 22, 2013 at 11:59 PM, Utkarsh Sengar utkarsh2...@gmail.com wrote: We use this to start/stop solr: Start: java -Dsolr.clustering.enabled=true -Dsolr.solr.home=multicore -Djetty.class.path=lib/ext/* -Dbootstrap_conf=true -DnumShards=3 -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar Stop: java -Dsolr.solr.home=multicore -Dbootstrap_conf=true -DnumShards=3 -DSTOP.PORT=8079 -DSTOP.KEY=some_value -jar start.jar --stop Thanks, -Utkarsh On Tue, Oct 22, 2013 at 10:09 AM, Raheel Hasan raheelhasan@gmail.com wrote: ok fantastic... thanks a lot guyz On Tue, Oct 22, 2013 at 10:00 PM, François Schiettecatte fschietteca...@gmail.com wrote: Yago has the right command to search for the process, that will get you the process ID specifically the first number on the output line, then do 'kill ###', if that fails 'kill -9 ###'. François On Oct 22, 2013, at 12:56 PM, Raheel Hasan raheelhasan@gmail.com wrote: its CentOS... and using jetty with solr here.. On Tue, Oct 22, 2013 at 9:54 PM, François Schiettecatte fschietteca...@gmail.com wrote: A few more specifics about the environment would help, Windows/Linux/...? Jetty/Tomcat/...? François On Oct 22, 2013, at 12:50 PM, Yago Riveiro yago.rive...@gmail.com wrote: If you are asking about if solr has a way to restart himself, I think that the answer is no. If you lost control of the remote machine someone will need to go and restart the machine ... You can try use a kvm or other remote control system -- Yago Riveiro Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, October 22, 2013 at 5:46 PM, François Schiettecatte wrote: If you are on linux/unix, use the kill command. François On Oct 22, 2013, at 12:42 PM, Raheel Hasan raheelhasan@gmail.com (mailto: raheelhasan@gmail.com) wrote: Hi, is there a way to stop/restart java? I lost control over it via SSH and connection was closed. But the Solr (start.jar) is still running. thanks. -- Regards, Raheel Hasan -- Regards, Raheel Hasan -- Regards, Raheel Hasan -- Thanks, -Utkarsh -- Regards, Raheel Hasan -- Regards, Raheel Hasan
RE: Seeking New Moderators for solr-user@lucene
Hello Hoss - My pleasure, kindly accept my moderator nomination. Regards, Jeeva -- Original Message -- From: Chris Hostetter [mailto:hossman_luc...@fucit.org] Sent: October 19, 2013 12:22:34 AM GMT+05:30 To: solr-user@lucene.apache.org Subject: Seeking New Moderators for solr-user@lucene It looks like it's time to inject some fresh blood into the solr-user@lucene moderation team. If you'd like to volunteer to be a moderator, please reply back to this thread and specify which email address you'd like to use as a moderator (if different from the one you use when sending the email) Being a moderator is really easy: you'll get a some extra emails in your inbox with MODERATE in the subject, which you skim to see if they are spam -- if they are you delete them, if not you reply all to let them get sent to the list, and authorize that person to send future messages w/o moderation. Occasionally, you'll see an explicit email to solr-user-owner@lucene from a user asking for help realted to their subscription (usually unsubscribing problems) and you and the other moderators chime in with assistance when possible. More details can be found here... https://wiki.apache.org/solr/MailingListModeratorInfo (I'll wait ~72+ hours to see who responds, and then file the appropriate jira with INFRA) -Hoss
Re: zkcli issues
guess would be that the war is not exploded yet. You have to manually unzip or start and stop solr. This logic could probably be added to those scripts (unzip it), but it's annoying to have any logic in them since everything has to be mirrored in bat and sh. - Mark On Nov 16, 2012, at 11:18 AM, Nick Chase nch...@earthlink.net wrote: I agree that it *shouldn't* be OS specific. :) Anyway, thanks for the suggestion, but that's not it. I get the same error with the script right out of the box: Error: Could not find or load main class C:\sw\apache-solr-4.0.0\example\cloud-scripts\..\solr-webapp\webapp\WEB-INF\lib\apache-solr-solrj-4.0.0.jar And anyway, it's a weird error, referencing a jar as a class, isn't it? Start up a JIRA? - Nick On 11/16/2012 10:42 AM, Mark Miller wrote: I *think* I tested the script on windows once way back. Anyway, the code itself should not be OS specific. One thing you might want to check if you are copying unix cmd line stuff - I think windows separates classpath entries with ; rather than : - so you likely to need to change that. You'd think java could have been smart enough to accept either/or at worst, but meh. For example: .:/Users/jeeva/dc-1/solr-cli-lib/* should be .;/Users/jeeva/dc-1/solr-cli-lib/* - Mark On Thu, Nov 15, 2012 at 8:53 PM, Nick Chase nch...@earthlink.net wrote: Unfortunately, this doesn't seem to solve the issue; now I'm beginning to wonder if maybe it's because I'm on Windows. Has anyone successfully run ZkCLI on Windows? Nick On 11/12/2012 2:27 AM, Jeevanandam Madanagopal wrote: Nick - Sorry, embedded links are not shown in previous email. I'm mentioning below. Handy SolrCloud ZkCLI Commands (http://www.myjeeva.com/2012/10/solrcloud-cluster-single-collection-deployment/#handy-solrcloud-cli-commands) Uploading Solr Configuration into ZooKeeper ensemble (http://www.myjeeva.com/2012/10/solrcloud-cluster-single-collection-deployment/#uploading-solrconfig-to-zookeeper) Cheers, Jeeva On Nov 12, 2012, at 12:48 PM, Jeevanandam Madanagopal je...@myjeeva.com wrote: Nick - I believe you're experiencing a difficulties with SolrCloud CLI commands for interacting ZooKeeper. Please have a look on below links, it will provide you direction. Handy SolrCloud ZkCLI Commands Uploading Solr Configuration into ZooKeeper ensemble Cheers, Jeeva On Nov 12, 2012, at 4:45 AM, Mark Miller markrmil...@gmail.com wrote: On 11/11/2012 04:47 PM, Yonik Seeley wrote: On Sun, Nov 11, 2012 at 10:39 PM, Nick Chase nch...@earthlink.net wrote: So I'm trying to use ZkCLI without success. I DID start and stop Solr in non-cloud mode, so everything is extracted and it IS finding zookeeper*.jar. However, now it's NOT finding SolrJ.
Re: Multivalued or not
Hello Peter - In Solr 3.6 multiValued is false by default. Since Schema version 1.1 onwards multiValued attribute value is false by default (field …, dynamicField …, fieldType ...) -Jeeva Blog: http://www.myjeeva.com On Nov 14, 2012, at 2:04 PM, Peter Kirk p...@alpha-solutions.dk wrote: Hi In Solr 3.6, is multivalued for fields, default true or false? It appears that it is default false for normal fields, and default true for dynamic fields - is that correct? Thanks, Peter
Re: Multivalued or not
Okay, I believe you're using Solr 3.6, here you can use schema version 1.5 However, you're currently using version 1.0, it safer to update your schema version to 1.1 then multiValued is false by default. FYI. Schema version info (from schema.xml): -- 1.0: multiValued attribute did not exist, all fields are multiValued by nature 1.1: multiValued attribute introduced, false by default 1.2: omitTermFreqAndPositions attribute introduced, true by default except for text fields 1.3: removed optional field compress feature 1.4: autoGeneratePhraseQueries attribute introduced to drive QueryParser behavior when a single string produces multiple tokens. Defaults to off for version = 1.4 1.5: omitNorms defaults to true for primitive field types (int, float, boolean, string...) - Jeeva Blog: http://www.myjeeva.com On Nov 14, 2012, at 2:54 PM, Peter Kirk p...@alpha-solutions.dk wrote: Hi, it says version 1.0 schema name=naturereserve version=1.0 /Peter -Original Message- From: Erik Hatcher [mailto:erik.hatc...@gmail.com] Sent: 14. november 2012 10:22 To: solr-user@lucene.apache.org Subject: Re: Multivalued or not But what is your schema version? See the top of schema.xml. On Nov 14, 2012, at 4:17, Peter Kirk p...@alpha-solutions.dk wrote: Hi Thanks for the reply. It is strange, because when I index to a field defined like: dynamicField indexed=true name=*_string stored=true type=string / Then the results I receive are like: arr name=category_string strWoodland/str /arr Which seems to indicate a multivalued field. If I change the field definition, so I explicitly say multivalued is false: dynamicField indexed=true name=*_string stored=true type=string multivalued=false / Then the result is like: str name=category_stringWoodland/str /Peter -Original Message- From: Jeevanandam Madanagopal [mailto:je...@myjeeva.com] Sent: 14. november 2012 10:11 To: solr-user@lucene.apache.org Subject: Re: Multivalued or not Hello Peter - In Solr 3.6 multiValued is false by default. Since Schema version 1.1 onwards multiValued attribute value is false by default (field ..., dynamicField ..., fieldType ...) -Jeeva Blog: http://www.myjeeva.com On Nov 14, 2012, at 2:04 PM, Peter Kirk p...@alpha-solutions.dk wrote: Hi In Solr 3.6, is multivalued for fields, default true or false? It appears that it is default false for normal fields, and default true for dynamic fields - is that correct? Thanks, Peter
Re: zkcli issues
Nick - I believe you're experiencing a difficulties with SolrCloud CLI commands for interacting ZooKeeper. Please have a look on below links, it will provide you direction. Handy SolrCloud ZkCLI Commands Uploading Solr Configuration into ZooKeeper ensemble Cheers, Jeeva On Nov 12, 2012, at 4:45 AM, Mark Miller markrmil...@gmail.com wrote: On 11/11/2012 04:47 PM, Yonik Seeley wrote: On Sun, Nov 11, 2012 at 10:39 PM, Nick Chase nch...@earthlink.net wrote: So I'm trying to use ZkCLI without success. I DID start and stop Solr in non-cloud mode, so everything is extracted and it IS finding zookeeper*.jar. However, now it's NOT finding SolrJ. Not sure about your specific problem in this case, but I chatted with Mark about this while at ApacheCon... it seems like we should be able to explode the WAR ourselves if necessary, eliminating the need to start Solr first. Just throwing it out there before I forgot about it ;-) -Yonik http://lucidworks.com I guess the tricky part might be knowing where to extract it. We know how to do it for the default jetty setup, but that could be reconfigured or you could be using another web container. Kind of annoying. - Mark
Re: zkcli issues
Nick - Sorry, embedded links are not shown in previous email. I'm mentioning below. Handy SolrCloud ZkCLI Commands (http://www.myjeeva.com/2012/10/solrcloud-cluster-single-collection-deployment/#handy-solrcloud-cli-commands) Uploading Solr Configuration into ZooKeeper ensemble (http://www.myjeeva.com/2012/10/solrcloud-cluster-single-collection-deployment/#uploading-solrconfig-to-zookeeper) Cheers, Jeeva On Nov 12, 2012, at 12:48 PM, Jeevanandam Madanagopal je...@myjeeva.com wrote: Nick - I believe you're experiencing a difficulties with SolrCloud CLI commands for interacting ZooKeeper. Please have a look on below links, it will provide you direction. Handy SolrCloud ZkCLI Commands Uploading Solr Configuration into ZooKeeper ensemble Cheers, Jeeva On Nov 12, 2012, at 4:45 AM, Mark Miller markrmil...@gmail.com wrote: On 11/11/2012 04:47 PM, Yonik Seeley wrote: On Sun, Nov 11, 2012 at 10:39 PM, Nick Chase nch...@earthlink.net wrote: So I'm trying to use ZkCLI without success. I DID start and stop Solr in non-cloud mode, so everything is extracted and it IS finding zookeeper*.jar. However, now it's NOT finding SolrJ. Not sure about your specific problem in this case, but I chatted with Mark about this while at ApacheCon... it seems like we should be able to explode the WAR ourselves if necessary, eliminating the need to start Solr first. Just throwing it out there before I forgot about it ;-) -Yonik http://lucidworks.com I guess the tricky part might be knowing where to extract it. We know how to do it for the default jetty setup, but that could be reconfigured or you could be using another web container. Kind of annoying. - Mark
Re: KeeperException (NodeExists for /overseer): SolrCloud Multiple Collections - is it safe ignore these exceptions?
Thanks Mark! Cheers, Jeeva On Oct 19, 2012, at 8:35 AM, Mark Miller markrmil...@gmail.com wrote: Yes, those exceptions are fine. These are cases where we try to delete the node if it's there, but don't care if it's not there - things like that. In some of these cases, ZooKeeper logs things we can't stop, even though it's expected that sometimes we will try and remove nodes that are not there or create nodes that are already there. - Mark On Thu, Oct 18, 2012 at 9:01 AM, Jeevanandam Madanagopal je...@myjeeva.com wrote: Hello - While doing prototype of SolrCloud with Multiple Collection. Each collection represents country level data. - searching within collection represents country level - local search - searching across collection represents global search Attached the graph image of SolrCoud structure. For prototype I'm running Embedded ZooKeeper ensemble (5 replicated zookeeper servers). - Searching and Indexing in respective collection works well - Search across collection works well (for global search) While joining the 'Collection2' to zookeeper ensemble I noticed the following KeeperException in the logger. Question 'is it safe to ignore these exceptions?' Exception Log snippet: Oct 18, 2012 4:54:26 PM org.apache.zookeeper.server.NIOServerCnxn$Factory run INFO: Accepted socket connection from /fe80:0:0:0:0:0:0:1%1:62700 Oct 18, 2012 4:54:26 PM org.apache.zookeeper.server.NIOServerCnxn readConnectRequest INFO: Client attempting to establish new session at /fe80:0:0:0:0:0:0:1%1:62700 Oct 18, 2012 4:54:26 PM org.apache.zookeeper.server.NIOServerCnxn finishSessionInit INFO: Established session 0x13a73521356000a with negotiated timeout 15000 for client /fe80:0:0:0:0:0:0:1%1:62700 Oct 18, 2012 4:54:26 PM org.apache.zookeeper.server.PrepRequestProcessor pRequest INFO: Got user-level KeeperException when processing sessionid:0x13a73521356000a type:create cxid:0x1 zxid:0xfffe txntype:unknown reqpath:n/a Error Path:/overseer Error:KeeperErrorCode = NodeExists for /overseer Oct 18, 2012 4:54:26 PM org.apache.zookeeper.server.PrepRequestProcessor pRequest INFO: Got user-level KeeperException when processing sessionid:0x13a73521356000a type:create cxid:0x2 zxid:0xfffe txntype:unknown reqpath:n/a Error Path:/overseer Error:KeeperErrorCode = NodeExists for /overseer Oct 18, 2012 4:54:26 PM org.apache.zookeeper.server.PrepRequestProcessor pRequest INFO: Got user-level KeeperException when processing sessionid:0x13a73521356000a type:delete cxid:0x4 zxid:0xfffe txntype:unknown reqpath:n/a Error Path:/live_nodes/mac-book-pro.local:7500_solr Error:KeeperErrorCode = NoNode for /live_nodes/mac-book-pro.local:7500_solr Oct 18, 2012 4:54:26 PM org.apache.solr.common.cloud.ZkStateReader$3 process INFO: Updating live nodes Cheers, Jeeva -- - Mark
Re: maven artifact for solr-solrj-4.0.0
Grzegorz Sobczyk - It's already available in Maven central repo link dependency groupIdorg.apache.solr/groupId artifactIdsolr-solrj/artifactId version4.0.0/version /dependency PS: use this 'http://search.maven.org' official website of maven central repository for artifact search/download Cheers, Jeeva Blog: http://www.myjeeva.com On Oct 18, 2012, at 12:30 PM, Amit Nithian anith...@gmail.com wrote: I am not sure if this repository https://repository.apache.org/content/repositories/releases/ works but the modification dates seem reasonable given the timing of the release. I suspect it'll be on maven central soon (hopefully) On Wed, Oct 17, 2012 at 11:13 PM, Grzegorz Sobczyk grzegorz.sobc...@contium.pl wrote: Hello Is there maven artifact for solrj 4.0.0 release ? When it will be available to download from http://mvnrepository.com/ ?? version 4.0.0-BETA isn't compatibile with 4.0.0 (problems with zookeeper and clusterstate.json parsing) Best regards Grzegorz Sobczyk
Re: maven artifact for solr-solrj-4.0.0
Sorry, missed the maven central repo link - http://search.maven.org/#artifactdetails|org.apache.solr|solr-solrj|4.0.0|jar Cheers, Jeeva Blog: http://www.myjeeva.com On Oct 18, 2012, at 1:59 PM, Jeevanandam Madanagopal je...@myjeeva.com wrote: Grzegorz Sobczyk - It's already available in Maven central repo link dependency groupIdorg.apache.solr/groupId artifactIdsolr-solrj/artifactId version4.0.0/version /dependency PS: use this 'http://search.maven.org' official website of maven central repository for artifact search/download Cheers, Jeeva Blog: http://www.myjeeva.com On Oct 18, 2012, at 12:30 PM, Amit Nithian anith...@gmail.com wrote: I am not sure if this repository https://repository.apache.org/content/repositories/releases/ works but the modification dates seem reasonable given the timing of the release. I suspect it'll be on maven central soon (hopefully) On Wed, Oct 17, 2012 at 11:13 PM, Grzegorz Sobczyk grzegorz.sobc...@contium.pl wrote: Hello Is there maven artifact for solrj 4.0.0 release ? When it will be available to download from http://mvnrepository.com/ ?? version 4.0.0-BETA isn't compatibile with 4.0.0 (problems with zookeeper and clusterstate.json parsing) Best regards Grzegorz Sobczyk
Re: How to expand list into multi-valued fields?
here you go specify regex transformer in entity tag of DIH config xml like below entity transformer=RegexTransformer ... / and then field column=ColumnA name=FruitField splitBy=\| / That's it! - Jeevanandam On 02-05-2012 12:35 am, invisbl wrote: I am indexing content from a RDBMS. I have a column in a table with pipe separated values, and upon indexing I would like to transform these values into multi-valued fields in SOLR's index. For example, ColumnA (From RDBMS) - apple|orange|banana I want to expand this to, SOLR Index FruitField=apple FruitField=orange FruitField=banana or number expand to, SOLR Index FruitField1=apple FruitField2=orange FruitField3=banana Please help, thank you! -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-expand-list-into-multi-valued-fields-tp3953378.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Boosting documents based on search term/phrase
Yes, you can add in last-components section on default query handler. arr name=last-components strelevator/str /arr - Jeevanandam On 02-05-2012 3:53 am, Donald Organ wrote: query elevation was exactly what I was talking about. Now is there a way to add this to the default query handler? On Tue, May 1, 2012 at 4:26 PM, Jack Krupansky j...@basetechnology.comwrote: Do you mean besides query elevation? http://wiki.apache.org/solr/**QueryElevationComponenthttp://wiki.apache.org/solr/QueryElevationComponent And besides explicit boosting by the user (the ^ suffix operator after a term/phrase)? -- Jack Krupansky -Original Message- From: Donald Organ Sent: Tuesday, May 01, 2012 3:59 PM To: solr-user Subject: Boosting documents based on search term/phrase Is there a way to boost documents based on the search term/phrase?
Re: change index/store at indexing time
Maria -
thanks for detailed explanation.
as per schema.xml; stored or indexed should be defined at schema design itself.
as per my understanding defining at runtime is not feasible.
BTW, you can have multiValued=true attribute for dynamic fields too.
- Jeevanandam
On Apr 29, 2012, at 1:06 AM, Vazquez, Maria (STM) wrote:
Thanks Jeevanandam.
That still doesn't have the same behavior as Lucene since multiple fields
with different names have to be created.
What I want is this exactly (multi-value field)
document.add(new Field(geoids, geoId, Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));
document.add(new Field(geoids, geoId, Field.Store.NO,
Field.Index.NOT_ANALYZED_NO_NORMS));
In Lucene I can save geoids first as stored and in the next line as not
stored and it will do exactly that. I want to duplicate this behavior in Solr
but I can't do it having only one field in the schema called geoids that I an
manipulate at inde time whether to store or not depending on a condition.
Thanks again for the help, hope this explanation makes it more clear in what
I'm trying to do.
Maria
On Apr 28, 2012, at 11:49 AM, Jeevanandam
je...@myjeeva.commailto:je...@myjeeva.com wrote:
Maria,
For your need please define unique pattern using dynamic field in schema.xml
Please have a look http://wiki.apache.org/solr/SchemaXml#Dynamic_fields
Hope that helps!
-Jeevanandam
Technology keeps you connected!
On Apr 28, 2012, at 10:33 PM, Vazquez, Maria (STM)
maria.vazq...@dexone.commailto:maria.vazq...@dexone.com wrote:
I can call a script for the logic part but what I want to figure out is how
to save the same field sometimes as stored and indexed, sometimes as stored
not indexed, etc. From a transformer or a script I didn't see anything where
I can modify that at indexing time.
Thanks a lot,
Maria
On Apr 27, 2012, at 18:38, Bill Bell
billnb...@gmail.commailto:billnb...@gmail.com wrote:
Yes you can. Just use a script that is called for each row.
Bill Bell
Sent from mobile
On Apr 27, 2012, at 6:38 PM, Vazquez, Maria (STM)
maria.vazq...@dexone.commailto:maria.vazq...@dexone.com wrote:
Hi,
I'm migrating a project from Lucene 2.9 to Solr 3.4.
There is a special case in the code that indexes the same field in two
different ways, which is completely legal in Lucene directly but I don't know
how to duplicate this same behavior in Solr:
if (isFirstGeo) {
document.add(new Field(geoids, geoId, Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));
isFirstGeo = false;
} else {
if (countProducts 100)
document.add(new Field(geoids, geoId, Field.Store.NO,
Field.Index.NOT_ANALYZED_NO_NORMS));
else
document.add(new Field(geoids, geoId, Field.Store.YES, Field.Index.NO));
}
Is there any way to do this in Solr in a Tranformer? I'm using the DIH to
index and I can't see a way to do this other than having three fields in the
schema like geoids_store_index, geoids_nostore_index, and
geoids_store_noindex.
Thanks a lot in advance.
Maria
Re: change index/store at indexing time
Maria,
For your need please define unique pattern using dynamic field in schema.xml
Please have a look http://wiki.apache.org/solr/SchemaXml#Dynamic_fields
Hope that helps!
-Jeevanandam
Technology keeps you connected!
On Apr 28, 2012, at 10:33 PM, Vazquez, Maria (STM) maria.vazq...@dexone.com
wrote:
I can call a script for the logic part but what I want to figure out is how
to save the same field sometimes as stored and indexed, sometimes as stored
not indexed, etc. From a transformer or a script I didn't see anything where
I can modify that at indexing time.
Thanks a lot,
Maria
On Apr 27, 2012, at 18:38, Bill Bell billnb...@gmail.com wrote:
Yes you can. Just use a script that is called for each row.
Bill Bell
Sent from mobile
On Apr 27, 2012, at 6:38 PM, Vazquez, Maria (STM)
maria.vazq...@dexone.com wrote:
Hi,
I'm migrating a project from Lucene 2.9 to Solr 3.4.
There is a special case in the code that indexes the same field in two
different ways, which is completely legal in Lucene directly but I don't
know how to duplicate this same behavior in Solr:
if (isFirstGeo) {
document.add(new Field(geoids, geoId, Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));
isFirstGeo = false;
} else {
if (countProducts 100)
document.add(new Field(geoids, geoId, Field.Store.NO,
Field.Index.NOT_ANALYZED_NO_NORMS));
else
document.add(new Field(geoids, geoId, Field.Store.YES,
Field.Index.NO));
}
Is there any way to do this in Solr in a Tranformer? I'm using the DIH to
index and I can't see a way to do this other than having three fields in
the schema like geoids_store_index, geoids_nostore_index, and
geoids_store_noindex.
Thanks a lot in advance.
Maria
Re: change index/store at indexing time
Maria,
thanks for detailed explanation.
as per schema.xml; stored or indexed should be defined at design-time.
Per my understanding defining at runtime is not feasible.
BTW, you can have multiValued=true attribute for dynamic fields too.
- Jeevanandam
On 29-04-2012 2:06 am, Vazquez, Maria (STM) wrote:
Thanks Jeevanandam.
That still doesn't have the same behavior as Lucene since multiple
fields with different names have to be created.
What I want is this exactly (multi-value field)
document.add(new Field(geoids, geoId, Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));
document.add(new Field(geoids, geoId, Field.Store.NO,
Field.Index.NOT_ANALYZED_NO_NORMS));
In Lucene I can save geoids first as stored and in the next line as
not stored and it will do exactly that. I want to duplicate this
behavior in Solr but I can't do it having only one field in the
schema
called geoids that I an manipulate at inde time whether to store or
not depending on a condition.
Thanks again for the help, hope this explanation makes it more clear
in what I'm trying to do.
Maria
On Apr 28, 2012, at 11:49 AM, Jeevanandam
je...@myjeeva.commailto:je...@myjeeva.com wrote:
Maria,
For your need please define unique pattern using dynamic field in
schema.xml
Please have a look
http://wiki.apache.org/solr/SchemaXml#Dynamic_fields
Hope that helps!
-Jeevanandam
Technology keeps you connected!
On Apr 28, 2012, at 10:33 PM, Vazquez, Maria (STM)
maria.vazq...@dexone.commailto:maria.vazq...@dexone.com wrote:
I can call a script for the logic part but what I want to figure out
is how to save the same field sometimes as stored and indexed,
sometimes as stored not indexed, etc. From a transformer or a script
I
didn't see anything where I can modify that at indexing time.
Thanks a lot,
Maria
On Apr 27, 2012, at 18:38, Bill Bell
billnb...@gmail.commailto:billnb...@gmail.com wrote:
Yes you can. Just use a script that is called for each row.
Bill Bell
Sent from mobile
On Apr 27, 2012, at 6:38 PM, Vazquez, Maria (STM)
maria.vazq...@dexone.commailto:maria.vazq...@dexone.com wrote:
Hi,
I'm migrating a project from Lucene 2.9 to Solr 3.4.
There is a special case in the code that indexes the same field in
two different ways, which is completely legal in Lucene directly but
I
don't know how to duplicate this same behavior in Solr:
if (isFirstGeo) {
document.add(new Field(geoids, geoId, Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));
isFirstGeo = false;
} else {
if (countProducts 100)
document.add(new Field(geoids, geoId, Field.Store.NO,
Field.Index.NOT_ANALYZED_NO_NORMS));
else
document.add(new Field(geoids, geoId, Field.Store.YES,
Field.Index.NO));
}
Is there any way to do this in Solr in a Tranformer? I'm using the
DIH to index and I can't see a way to do this other than having three
fields in the schema like geoids_store_index, geoids_nostore_index,
and geoids_store_noindex.
Thanks a lot in advance.
Maria
Re: should slave replication be turned off / on during master clean and re-index?
I guess you're looking for 'disabling replication poll on slave' go to 'Replication dashboard[1]', there you have options like Enable/Disable Poll, Force replication, Abort replication dashboard url: http://slave_host:port/solr/corename/admin/replication/index.jsp Poll Disabled = slave will not poll master for replication - Jeevanandam [1] http://wiki.apache.org/solr/SolrReplication#Replication_Dashboard On Apr 28, 2012, at 8:03 AM, geeky2 wrote: hello, thank you for the reply, Does a clean mean issuing a deletion query (e.g. deleteid*:*/id/delete) prior to re-indexing all of your content? I don't think the slaves will download any changes until you've committed at some point on the master. well, in this case when i say, clean (on the Master), i mean selecting the Full Import with Cleaning button from the DataImportHandler Development Console page in solr. at the top of the page, i have the check boxes selected for verbose and clean (*but i don't have the commit checkbox selected*). by doing the above process - doesn't this issue a deletion query - then start the import? and as a follow-up - when actually is the commit being done? here is my from my solrconfig.xml file on the master updateHandler class=solr.DirectUpdateHandler2 *autoCommit maxTime6/maxTime maxDocs1000/maxDocs /autoCommit* maxPendingDeletes10/maxPendingDeletes /updateHandler -- View this message in context: http://lucene.472066.n3.nabble.com/should-slave-replication-be-turned-off-on-during-master-clean-and-re-index-tp3945531p3945954.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: change index/store at indexing time
As Bill mentioned, you can use Script.
Please have a look:
http://wiki.apache.org/solr/DataImportHandler#ScriptTransformer
- Jeevanandam
On Apr 28, 2012, at 7:07 AM, Bill Bell wrote:
Yes you can. Just use a script that is called for each row.
Bill Bell
Sent from mobile
On Apr 27, 2012, at 6:38 PM, Vazquez, Maria (STM)
maria.vazq...@dexone.com wrote:
Hi,
I'm migrating a project from Lucene 2.9 to Solr 3.4.
There is a special case in the code that indexes the same field in two
different ways, which is completely legal in Lucene directly but I don't
know how to duplicate this same behavior in Solr:
if (isFirstGeo) {
document.add(new Field(geoids, geoId, Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));
isFirstGeo = false;
} else {
if (countProducts 100)
document.add(new Field(geoids, geoId, Field.Store.NO,
Field.Index.NOT_ANALYZED_NO_NORMS));
else
document.add(new Field(geoids, geoId, Field.Store.YES,
Field.Index.NO));
}
Is there any way to do this in Solr in a Tranformer? I'm using the DIH to
index and I can't see a way to do this other than having three fields in the
schema like geoids_store_index, geoids_nostore_index, and
geoids_store_noindex.
Thanks a lot in advance.
Maria
Re: Multi-words synonyms matching
usage of q and fq q = is typically the main query for the search request fq = is Filter Query; generally used to restrict the super set of documents without influencing score (more info. http://wiki.apache.org/solr/CommonQueryParameters#q) For example: q=hotel de ville === returns 100 documents q=hotel de villefq=price:[100 To *]fq=roomType:King size Bed === returns 40 documents from super set of 100 documents hope this helps! - Jeevanandam On 24-04-2012 3:08 pm, elisabeth benoit wrote: Hello, I'd like to resume this post. The only way I found to do not split synonyms in words in synonyms.txt it to use the line filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true tokenizerFactory=solr.KeywordTokenizerFactory/ in schema.xml where tokenizerFactory=solr.KeywordTokenizerFactory instructs SynonymFilterFactory not to break synonyms into words on white spaces when parsing synonyms file. So now it works fine, mairie is mapped into hotel de ville and when I send request q=hotel de ville (quotes are mandatory to prevent analyzer to split hotel de ville on white spaces), I get answers with word mairie. But when I use fq parameter (fq=CATEGORY_ANALYZED:hotel de ville), it doesn't work!!! CATEGORY_ANALYZED is same field type as default search field. This means that when I send q=hotel de ville and fq=CATEGORY_ANALYZED:hotel de ville, solr uses the same analyzer, the one with the line filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true tokenizerFactory=solr.KeywordTokenizerFactory/. Anyone as a clue what is different between q analysis behaviour and fq analysis behaviour? Thanks a lot Elisabeth 2012/4/12 elisabeth benoit elisaelisael...@gmail.com oh, that's right. thanks a lot, Elisabeth 2012/4/11 Jeevanandam Madanagopal je...@myjeeva.com Elisabeth - As you described, below mapping might suit for your need. mairie = hotel de ville, mairie mairie gets expanded to hotel de ville and mairie at index time. So mairie and hotel de ville searchable on document. However, still white space tokenizer splits at query time will be a problem as described by Markus. --Jeevanandam On Apr 11, 2012, at 12:30 PM, elisabeth benoit wrote: Have you tried the =' mapping instead? Something like hotel de ville = mairie might work for you. Yes, thanks, I've tried it but from what I undestand it doesn't solve my problem, since this means hotel de ville will be replace by mairie at index time (I use synonyms only at index time). So when user will ask hôtel de ville, it won't match. In fact, at index time I have mairie in my data, but I want user to be able to request mairie or hôtel de ville and have mairie as answer, and not have mairie as an answer when requesting hôtel. To map `mairie` to `hotel de ville` as single token you must escape your white space. mairie, hotel\ de\ ville This results in a problem if your tokenizer splits on white space at query time. Ok, I guess this means I have a problem. No simple solution since at query time my tokenizer do split on white spaces. I guess my problem is more or less one of the problems discussed in http://lucene.472066.n3.nabble.com/Multi-word-synonyms-td3716292.html#a3717215 Thanks a lot for your answers, Elisabeth 2012/4/10 Erick Erickson erickerick...@gmail.com Have you tried the =' mapping instead? Something like hotel de ville = mairie might work for you. Best Erick On Tue, Apr 10, 2012 at 1:41 AM, elisabeth benoit elisaelisael...@gmail.com wrote: Hello, I've read several post on this issue, but can't find a real solution to my multi-words synonyms matching problem. I have in my synonyms.txt an entry like mairie, hotel de ville and my index time analyzer is configured as followed for synonyms. filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ The problem I have is that now mairie matches with hotel and I would only want mairie to match with hotel de ville and mairie. When I look into the analyzer, I see that mairie is mapped into hotel, and words de ville are added in second and third position. To change that, I tried to do filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true tokenizerFactory=solr.KeywordTokenizerFactory/ (as I read in one post) and I can see now in the analyzer that mairie is mapped to hotel de ville, but now when I have query hotel de ville, it doesn't match at all with mairie. Anyone has a clue of what I'm doing wrong? I'm using Solr 3.4. Thanks, Elisabeth
Re: Auto suggest on indexed file content filtered based on user
can you please share a sample query? -Jeevanandam On 24-04-2012 1:49 pm, prakash_ajp wrote: I am trying to implement an auto-suggest feature. The search feature already exists and searches on file content in user's allotted workspace. The following is from my schema that will be used for search indexing: field name=Text type=text indexed=true stored=false multiValued=false/ field name=UserName type=string indexed=true stored=true multiValued=true/ The search result is filtered by the user name. The suggest is implemented as a searchComponent and the field 'Text' is used by the suggester and would have to be filtered the same way the search is done. The problem with this approach is, suggest works on a single field and there is no way to include the UserName field as a filter. What's the best way out from here? Thanks in advance! Jay -- View this message in context: http://lucene.472066.n3.nabble.com/Auto-suggest-on-indexed-file-content-filtered-based-on-user-tp3934565p3934565.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Auto suggest on indexed file content filtered based on user
On Apr 24, 2012, at 9:37 PM, prakash_ajp wrote: Right now, the query is a very simple one, something like q=text. Basically, it would return ['textview', 'textviewer', ..] hmm, so you're using default query field But the issue is, the 'textviewer' could be from a file that is out of bounds for this user. So, ultimately I would like to include the userName in the query. As mentioned earlier, userName is another field in the main index. and you like to filter the result set along with userName field value -- View this message in context: http://lucene.472066.n3.nabble.com/Auto-suggest-on-indexed-file-content-filtered-based-on-user-tp3934565p3935765.html Sent from the Solr - User mailing list archive at Nabble.com. in this scenario 'fq' parameter will facilitate to achieve your desire result. Please refer http://wiki.apache.org/solr/CommonQueryParameters#fq try this q=textfq=userName:prakash Let us know! -Jeevanandam
Re: Auto suggest on indexed file content filtered based on user
yes only spellcheck indexed build field is for suggest query I believe, filtering a documents on search handler using fq parameter and spell suggest are two part we are discussing here. lets say you have field for spellcheck - used to build spell dictionary field name=spell type=textSpell …. … / using copyField for populating a spell field and get dictionary created referring spellcheck handler in the default search handler at 'last-components' section, like below arr name=last-components strspellcheck/str /arr then you will be able to apply search documents filtering and spellcheck params to search handler while querying. detailed info http://wiki.apache.org/solr/SpellCheckComponent [probably you might have already went thru :) ] -Jeevanandam On Apr 25, 2012, at 12:01 AM, prakash_ajp wrote: I read on a couple of other web pages that fq is not supported for suggester. I even tried the query and it doesn't help. My understanding was, when the suggest (spellcheck) index is built, only the field chosen is considered for queries and the other fields from the main index are not available for filtering purposes once the index is created. -- View this message in context: http://lucene.472066.n3.nabble.com/Auto-suggest-on-indexed-file-content-filtered-based-on-user-tp3934565p3936144.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Using two repeater to rapidly switching Master and Slave (Replication)?
On 23-04-2012 10:28 am, A Vorderegger wrote: This setup would be highly convenient and perfect for the purpose of failing over the Master role however it does not work for me. Resolving http://slave_host:port/solr/replication?command=enablepoll I am met with: str name=statusERROR/strstr name=messageNo slave configured/str no matter what order I enable polling / replication in. I am confident that I have setup my solrconfig.xml file exactly as described. Could you please further describe how this setup is successfully achieved? Thanks in advance can you please share your repeater configuration (just replication handler definition)? It looks like, on slave host; master is enabled. and on master executing enablepoll command, will result into response lst name=responseHeader int name=status0/int int name=QTime2/int /lst str name=statusERROR/str str name=messageNo slave configured/str /response -Jeevanandam
Re: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception
On 23-04-2012 8:18 pm, sivaprasad wrote: Hi, When i am trying to index 16 millions of documents using dataimport handler, intermittently i am getting the below exception and the indexing get stopped. STACKTRACE: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost. at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1997) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2411) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2916) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:885) at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1360) at com.mysql.jdbc.MysqlIO.fetchRowsViaCursor(MysqlIO.java:4044) at com.mysql.jdbc.CursorRowProvider.fetchMoreRows(CursorRowProvider.java:396) at com.mysql.jdbc.CursorRowProvider.hasNext(CursorRowProvider.java:313) at com.mysql.jdbc.ResultSet.next(ResultSet.java:7296) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:331) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$600(JdbcDataSource.java:228) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:262) at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:77) at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:75) at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:238) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:591) at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:267) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:186) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:359) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:408) ** END NESTED EXCEPTION ** Last packet sent to the server was 2 ms ago. at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2622) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2916) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:885) at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1360) at com.mysql.jdbc.MysqlIO.fetchRowsViaCursor(MysqlIO.java:4044) at com.mysql.jdbc.CursorRowProvider.fetchMoreRows(CursorRowProvider.java:396) at com.mysql.jdbc.CursorRowProvider.hasNext(CursorRowProvider.java:313) at com.mysql.jdbc.ResultSet.next(ResultSet.java:7296) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:331) ... 11 more 2012-04-23 08:25:35,693 SEVERE [org.apache.solr.handler.dataimport.DataImporter] (Thread-21) Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception: And the db-config.xml has the below configuration. dataSource driver=com.mysql.jdbc.Driver url=jdbc:mysql://localhost:3306/phpq user=slrmgr defaultFetchSize=30 useCursorFetch=true autoReconnect=true tcpKeepAlive=true connectionTimeout=12 password=pqmgr123 batch-size=-1/ Any help on this is much appreciable. -- View this message in context: http://lucene.472066.n3.nabble.com/Full-Import-failed-org-apache-solr-handler-dataimport-DataImportHandlerException-com-mysql-jdbc-Commn-tp3932521p3932521.html Sent from the Solr - User mailing list archive at Nabble.com. Sivaprasad, just a clarification about batch size attribute, is it typo error or real in your db-config.xml Supported attribute name is batchSize=-1 (http://wiki.apache.org/solr/DataImportHandler#Configuring_JdbcDataSource) -Jeevanandam
Re: How to escape “” character in regex in Solr schema.xml?
try this one pattern=(?lt;=[^.!?\\s][^.!?]*(?:[.!?](?![']?\s|$)[^.!?]*)*)[.!?]+(?=\\s|$) I tested locally, solr start perfectly. now please test with data. -Jeevanandam On 19-04-2012 9:29 am, smooth almonds wrote: Using Solr 3.5.0 and in my schema.xml I'm using the following to mark the end of sentences and replace the end punctuation with a symbolic token: charFilter class=quot;solr.PatternReplaceCharFilterFactoryquot; pattern=quot;(?lt;=[^.!?\\s][^.!?]*(?:[.!?](?![']?\s|$)[^.!?]*)*)[.!?]+(?=\\s|$)quot; replacement=quot; monkeysentencequot;/ I'm not sure if that will even work for what I want, but first I need to solve the problem of escaping the '' character in the first '?=' lookbehind. I get the following error: org.xml.sax.SAXParseException: The value of attribute pattern associated with an element type null must not contain the '' character. I've tried using a '\' as in: pattern=(?\=[^.!?\\s][^.!?]*(?:[.!?](?![']?\s|$)[^.!?]*)*)[.!?]+(?=\\s|$) But I get the same error. -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-escape-character-in-regex-in-Solr-schema-xml-tp3921961p3921961.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: How to escape “” character in regex in Solr schema.xml?
previously given pattern will solve the '' char issue. however you will get following exception in the log Caused by: java.util.regex.PatternSyntaxException: Look-behind group does not have an obvious maximum length near index 48 (?=[^.!?\\s][^.!?]*(?:[.!?](?![']?\s|$)[^.!?]*)*)[.!?]+(?=\\s|$) ^ so revisit your regex pattern particularly position 48 -Jeevanandam On 19-04-2012 7:06 pm, Jeevanandam wrote: try this one pattern=(?lt;=[^.!?\\s][^.!?]*(?:[.!?](?![']?\s|$)[^.!?]*)*)[.!?]+(?=\\s|$) I tested locally, solr start perfectly. now please test with data. -Jeevanandam On 19-04-2012 9:29 am, smooth almonds wrote: Using Solr 3.5.0 and in my schema.xml I'm using the following to mark the end of sentences and replace the end punctuation with a symbolic token: charFilter class=quot;solr.PatternReplaceCharFilterFactoryquot; pattern=quot;(?lt;=[^.!?\\s][^.!?]*(?:[.!?](?![']?\s|$)[^.!?]*)*)[.!?]+(?=\\s|$)quot; replacement=quot; monkeysentencequot;/ I'm not sure if that will even work for what I want, but first I need to solve the problem of escaping the '' character in the first '?=' lookbehind. I get the following error: org.xml.sax.SAXParseException: The value of attribute pattern associated with an element type null must not contain the '' character. I've tried using a '\' as in: pattern=(?\=[^.!?\\s][^.!?]*(?:[.!?](?![']?\s|$)[^.!?]*)*)[.!?]+(?=\\s|$) But I get the same error. -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-escape-character-in-regex-in-Solr-schema-xml-tp3921961p3921961.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Wrong categorization with DIH
Ramo - Please share DIH configuration with us. -Jeevanandam On Apr 19, 2012, at 7:46 PM, Ramo Karahasan wrote: Does anyone has an idea what's going wrong here? Thanks, Ramo -Ursprüngliche Nachricht- Von: Gora Mohanty [mailto:g...@mimirtech.com] Gesendet: Dienstag, 17. April 2012 11:34 An: solr-user@lucene.apache.org Betreff: Re: Wrong categorization with DIH On 17 April 2012 14:47, Ramo Karahasan ramo.karaha...@googlemail.com wrote: Hi, i currently face the followin issue: Testing the following sql statement which is also used in SOLR (DIH) leads to a wrong categorization in solr: select p.title as title, p.id, p.category_id, p.pic_thumb, c.name as category, c.id as category_id from product p, category c WHERE p.category_id = c.id AND p.id = 3091328 This returns in my sql client: Apple MacBook Pro MD313D/A 33,8 cm (13,3 Zoll) Notebook (Intel Core i5-2435M, 2,4GHz, 4GB RAM, 500GB HDD, Intel HD 3000, Mac OS), 3091328, 1003, http://m-d.ww.cdn.com/images/I/41teWbp-uAL._SL75_.jpg, Computer, 1003 As you see, the categoryid 1003 points to Computer Via the solr searchadmin i get the following result when searchgin for id:3091328 str name=categorySport/str int name=category_id1003/int [...] Please share with us the rest of the DIH configuration file, i.e., the part where these data are saved to the Solr index. Regards, Gora
Re: PolySearcher in Solr
Please have a look http://wiki.apache.org/solr/DistributedSearch -Jeevanandam On Apr 19, 2012, at 9:14 PM, Ramprakash Ramamoorthy wrote: Dear all, I came across this while browsing through lucy http://lucy.apache.org/docs/perl/Lucy/Search/PolySearcher.html Does solr have an equivalent of this? My usecase is exactly the same (reading through multiple indices in a single shard and perform a distribution across shards). If not can someone give me a hint? I tried swapping readers for a single searcher, but didn't help. -- With Thanks and Regards, Ramprakash Ramamoorthy, Project Trainee, Zoho Corporation. +91 9626975420
Re: Wrong categorization with DIH
Ramo -
Are you using all the selected columns from the query?
select p.title as title, p.id, p.category_id,
p.pic_thumb, c.name as category, c.id as category_id from product p,
category c ...
I see following attributes 'p.id', 'p.category_id' 'p.pic_thumb' doesn't
have alias defined.
Pointers:
- Select only required field in the sql query
- Ensure sql alias name and attribute name in the schema.xml should match
or
- If you like to do explicit mapping for every column in DIH config as follow
field column=SQL-COLUMN-NAME-HERE name=SOLR-SCHEMA-ATTRIBUTE-NAME-HERE /
Detailed Info refer this: http://wiki.apache.org/solr/DataImportHandler
-Jeevanandam
On Apr 19, 2012, at 9:37 PM, Ramo Karahasan wrote:
Hi,
my config is just the following:
dataConfig
dataSource type=JdbcDataSource
driver=com.mysql.jdbc.Driver
url=jdbc:mysql://xx/asdx
user=
password=/
document
entity name=id
query=select p.title as title, p.id, p.category_id,
p.pic_thumb, c.name as category, c.id as category_id from product p,
category c WHERE p.category_id = c.id AND '${dataimporter.request.clean}'
!= 'false' OR updated_at '${dataimporter.last_index_time}'
/entity
/document
/dataConfig
I'm doing it as described on:
http://wiki.apache.org/solr/DataImportHandlerDeltaQueryViaFullImport
Any ideas?
Best regars,
Ramo
-Ursprüngliche Nachricht-
Von: Jeevanandam Madanagopal [mailto:je...@myjeeva.com]
Gesendet: Donnerstag, 19. April 2012 17:44
An: solr-user@lucene.apache.org
Betreff: Re: Wrong categorization with DIH
Ramo -
Please share DIH configuration with us.
-Jeevanandam
On Apr 19, 2012, at 7:46 PM, Ramo Karahasan wrote:
Does anyone has an idea what's going wrong here?
Thanks,
Ramo
-Ursprüngliche Nachricht-
Von: Gora Mohanty [mailto:g...@mimirtech.com]
Gesendet: Dienstag, 17. April 2012 11:34
An: solr-user@lucene.apache.org
Betreff: Re: Wrong categorization with DIH
On 17 April 2012 14:47, Ramo Karahasan ramo.karaha...@googlemail.com
wrote:
Hi,
i currently face the followin issue:
Testing the following sql statement which is also used in SOLR (DIH)
leads to a wrong categorization in solr:
select p.title as title, p.id, p.category_id, p.pic_thumb, c.name as
category, c.id as category_id from product p, category c WHERE
p.category_id = c.id AND p.id = 3091328
This returns in my sql client:
Apple MacBook Pro MD313D/A 33,8 cm (13,3 Zoll) Notebook (Intel Core
i5-2435M, 2,4GHz, 4GB RAM, 500GB HDD, Intel HD 3000, Mac OS),
3091328, 1003, http://m-d.ww.cdn.com/images/I/41teWbp-uAL._SL75_.jpg,
Computer,
1003
As you see, the categoryid 1003 points to Computer
Via the solr searchadmin i get the following result when searchgin
for
id:3091328
str name=categorySport/str
int name=category_id1003/int
[...]
Please share with us the rest of the DIH configuration file, i.e., the
part where these data are saved to the Solr index.
Regards,
Gora
Re: Can you suggest a method or pattern to consistently promote a document with any query?
Chris - Take a look - QueryElevationComponent http://wiki.apache.org/solr/QueryElevationComponent -Jeevanandam On Apr 18, 2012, at 10:46 PM, Chris Warner wrote: Hi, folks, Perhaps I'm overlooking an obvious solution to a common desire... I'd like to return a specific document with every query, as the first result. As well, I'd like to have that document be the first result in a *:* query. I'm looking into index time boosting using the boost attribute on the appropriate doc. I haven't tested this yet, and I'm not sure this would do anything for the *:* queries. Thanks for any suggested reading or patterns... Best, Chris -- chris_war...@yahoo.com
Re: Can you suggest a method or pattern to consistently promote a document with any query?
Chris - If you have defined 'last-components' in search handler, forceElevation=true may not required. It gets invoked in search life cycle arr name=last-components strelevator/str /arr -Jeevanandam On Apr 18, 2012, at 11:37 PM, Chris Warner wrote: Thanks to those who responded. A more thorough reading of the wiki and I see the need for forceElevation=true in the elevate query. Cheers, Chris - Original Message - From: Otis Gospodnetic otis_gospodne...@yahoo.com To: solr-user@lucene.apache.org solr-user@lucene.apache.org; Chris Warner chris_war...@yahoo.com Cc: Sent: Wednesday, April 18, 2012 10:23 AM Subject: Re: Can you suggest a method or pattern to consistently promote a document with any query? Chris, I haven't checked if Elevate Component has an easy way to push a specific doc for *all* queries, but have a look http://wiki.apache.org/solr/QueryElevationComponent Otis Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html - Original Message - From: Chris Warner chris_war...@yahoo.com To: solr-user@lucene.apache.org solr-user@lucene.apache.org Cc: Sent: Wednesday, April 18, 2012 1:16 PM Subject: Can you suggest a method or pattern to consistently promote a document with any query? Hi, folks, Perhaps I'm overlooking an obvious solution to a common desire... I'd like to return a specific document with every query, as the first result. As well, I'd like to have that document be the first result in a *:* query. I'm looking into index time boosting using the boost attribute on the appropriate doc. I haven't tested this yet, and I'm not sure this would do anything for the *:* queries. Thanks for any suggested reading or patterns... Best, Chris -- chris_war...@yahoo.com
Re: making query in query result
Halil - I'm describing scenario with sample query below: query 1: (cat:electronics) - lets say it returns 25 docs in search result query 2: (features:power) - will be applied on above result i.e. 'query 1'(25 docs) so final result refined to 16 docs in search result. If above scenario matches your need, please try like this: q=(cat%3Aelectronics) fq=(features:power) fq means Filter Query -Jeevanandam On 17-04-2012 4:22 pm, halil wrote: Hi List, I want to make query in a query result whish is done previously. I googled the net but couldnot find anything. How can I do that? I need a starting point. thanks in advance, -halil agin.
Re: Multi-words synonyms matching
Elisabeth - As you described, below mapping might suit for your need. mairie = hotel de ville, mairie mairie gets expanded to hotel de ville and mairie at index time. So mairie and hotel de ville searchable on document. However, still white space tokenizer splits at query time will be a problem as described by Markus. --Jeevanandam On Apr 11, 2012, at 12:30 PM, elisabeth benoit wrote: Have you tried the =' mapping instead? Something like hotel de ville = mairie might work for you. Yes, thanks, I've tried it but from what I undestand it doesn't solve my problem, since this means hotel de ville will be replace by mairie at index time (I use synonyms only at index time). So when user will ask hôtel de ville, it won't match. In fact, at index time I have mairie in my data, but I want user to be able to request mairie or hôtel de ville and have mairie as answer, and not have mairie as an answer when requesting hôtel. To map `mairie` to `hotel de ville` as single token you must escape your white space. mairie, hotel\ de\ ville This results in a problem if your tokenizer splits on white space at query time. Ok, I guess this means I have a problem. No simple solution since at query time my tokenizer do split on white spaces. I guess my problem is more or less one of the problems discussed in http://lucene.472066.n3.nabble.com/Multi-word-synonyms-td3716292.html#a3717215 Thanks a lot for your answers, Elisabeth 2012/4/10 Erick Erickson erickerick...@gmail.com Have you tried the =' mapping instead? Something like hotel de ville = mairie might work for you. Best Erick On Tue, Apr 10, 2012 at 1:41 AM, elisabeth benoit elisaelisael...@gmail.com wrote: Hello, I've read several post on this issue, but can't find a real solution to my multi-words synonyms matching problem. I have in my synonyms.txt an entry like mairie, hotel de ville and my index time analyzer is configured as followed for synonyms. filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ The problem I have is that now mairie matches with hotel and I would only want mairie to match with hotel de ville and mairie. When I look into the analyzer, I see that mairie is mapped into hotel, and words de ville are added in second and third position. To change that, I tried to do filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true tokenizerFactory=solr.KeywordTokenizerFactory/ (as I read in one post) and I can see now in the analyzer that mairie is mapped to hotel de ville, but now when I have query hotel de ville, it doesn't match at all with mairie. Anyone has a clue of what I'm doing wrong? I'm using Solr 3.4. Thanks, Elisabeth
Re: Solr is indexing but not showing results
Srini - This text datatype comes as sample configuration in SOLR distribution. Check this, it may suit your need! fieldType name=text class=solr.TextField positionIncrementGap=100 autoGeneratePhraseQueries=true analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt enablePositionIncrements=true / filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.KeywordMarkerFilterFactory protected=protwords.txt/ filter class=solr.PorterStemFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt enablePositionIncrements=true / filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=0 catenateNumbers=0 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.KeywordMarkerFilterFactory protected=protwords.txt/ filter class=solr.PorterStemFilterFactory/ /analyzer /fieldType -Jeevanandam On Apr 10, 2012, at 12:08 AM, Walter Underwood wrote: You will need to define or customize a field type for text. The example schema.xml file that is installed with Solr 3.5 has a several kinds of text fields, text_general and text_en are good places to start. You can use one of those, then customize it. wunder On Apr 9, 2012, at 11:27 AM, srini wrote: Hi Thanks for your reply. As per your suggestion I changed XML field type to text. field name=XML type=string indexed=true stored=true required=true/ but when I start solr it is throwing following exception. SEVERE: org.apache.solr.common.SolrException: Unknown fieldtype 'text' specified on field XML Any suggestions!!(Thanks for your reply) -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-is-indexing-but-not-showing-results-tp3897176p3897626.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr is indexing but not showing results
I agree partially, it actually depends. For instance during index time few of the synonyms mapping may or may not expand (for e.g.. frequent data index population from different source). So good apply at index time as well as query time to achieve complete ratio. Mostly of the time I did similar settings to meet customer requirements. For example: - Below sample text datatype with synonyms at index query time (below config has similar analyzer structure of tokenizer filter; so we can keep commonly one analyzer config too.) fieldType name=text class=solr.TextField positionIncrementGap=100 autoGeneratePhraseQueries=true analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt enablePositionIncrements=true / filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.KeywordMarkerFilterFactory protected=protwords.txt/ filter class=solr.PorterStemFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt enablePositionIncrements=true / filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=0 catenateNumbers=0 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.KeywordMarkerFilterFactory protected=protwords.txt/ filter class=solr.PorterStemFilterFactory/ /analyzer /fieldType -Jeevanandam On Apr 10, 2012, at 12:18 AM, Walter Underwood wrote: That is not a good configuration. Synonyms should be expanded at index time, not query time. --wunder On Apr 9, 2012, at 11:43 AM, Jeevanandam Madanagopal wrote: Srini - This text datatype comes as sample configuration in SOLR distribution. Check this, it may suit your need! fieldType name=text class=solr.TextField positionIncrementGap=100 autoGeneratePhraseQueries=true analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt enablePositionIncrements=true / filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=1 catenateNumbers=1 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.KeywordMarkerFilterFactory protected=protwords.txt/ filter class=solr.PorterStemFilterFactory/ /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.SynonymFilterFactory synonyms=synonyms.txt ignoreCase=true expand=true/ filter class=solr.StopFilterFactory ignoreCase=true words=stopwords.txt enablePositionIncrements=true / filter class=solr.WordDelimiterFilterFactory generateWordParts=1 generateNumberParts=1 catenateWords=0 catenateNumbers=0 catenateAll=0 splitOnCaseChange=1/ filter class=solr.LowerCaseFilterFactory/ filter class=solr.KeywordMarkerFilterFactory protected=protwords.txt/ filter class=solr.PorterStemFilterFactory/ /analyzer /fieldType -Jeevanandam On Apr 10, 2012, at 12:08 AM, Walter Underwood wrote: You will need to define or customize a field type for text. The example schema.xml file that is installed with Solr 3.5 has a several kinds of text fields, text_general and text_en are good places to start. You can use one of those, then customize it. wunder On Apr 9, 2012, at 11:27 AM, srini wrote: Hi Thanks for your reply. As per your suggestion I changed XML field type to text. field name=XML type=string indexed=true stored=true required=true/ but when I start solr it is throwing following exception. SEVERE: org.apache.solr.common.SolrException: Unknown fieldtype 'text' specified on field XML Any suggestions!!(Thanks for your reply) -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-is-indexing-but-not-showing-results-tp3897176p3897626.html Sent from the Solr - User mailing list archive at Nabble.com.
